Disclaimers ----------- The files in this directory are provided by IBM on an "AS IS" basis without warranty of any kind. Introduction ------------ The files in this directory contain generally useful utilities that use GPFS specific APIs to scan filesystem metadata. The source code is provided to give examples of how to use some of the APIs. Execution of these commands requires root authority. Utilities --------- tsgetusage - Fast scan of all files in a filesystem and calculate how much disk space is used by each unique user and/or group. tsinode - Fast scan of all files in a filesystem and display one line of ls information about each one. tslistall - Fast scan of all files in a filesystem and display one line of inodenumber pathname type ctime mtime tsfindinode - Fast parallel scan of all directories in a filesystem listing inodenumber type pathname for each inode specified tsreaddir - Fast parallel scan of all directories in a filesystem listing inodenumber type pathname for each file (used by tslistall) tstimes - Fast scan of all files in a filesystem and display one line of inode mtime ctime for each file (used by tslistall) tsbackup - Fast parallel backup utility using multiple TSM clients and a single TSM server. Parallelism is achieved via the use of multi-threading and the use of multiple TSM client processes. This utility makes use of the GPFS inode scan APIs. ------------------------------------------------------------------------------ tsgetusage - Fast scan of all files in a filesystem and calculate how much disk space is used by each unique user and/or group. Usage: tsgetusage [-g][-h][-q][-v][-u][-H hashTableSize] fsPath where: fsPath path to file system -u outputs data in diskusg format for acctdisk -g collect stats for groups -H sets the hash table size (default 1027) -h prints this message -q outputs only the resulting stats -v print information about each file Using the -u option will produce a file with a format like /usr/sbin/acct/diskusg that is used for other types of filesystems. This can be used as input to the /usr/sbin/acct/acctdisk program, and then merged with the other disk accounting data using /usr/sbin/acct/acctmerg. For more information about the AIX Accounting System, the preparation of daily and monthly reports, and the accounting files, see the AIX Accounting Overview. ------------------------------------------------------------------------------ tsinode - Fast scan of all files in a filesystem and display one line of information about each one. Usage: tsinode fsPath [startinode [endinode]] where: fsPath path to file system startinode first inode number to be listed endinode last inode number to be listed Hint: For the best performance on a filesystem with millions of inodes, run several copies of tsinode with different inode ranges and concatenate the output files. ------------------------------------------------------------------------------ tsfindinode - Fast parallel scan of all directories in a filesystem listing: inodenumber pathname Usage: tsfindinode {-i {inum | inumfile}} [-t nThreads] [--] fsPath where: fsPath path to file system -i inum adds one inode number to the list (this can be used multiple times) -i inumfile names a file containing inode numbers to be added to the list (this can be used multiple times) -t set the number of threads (default = 16) -- marker so that names starting with "-" are not treated like options. This program recursively traverses the directory structure to list all accessible files and directories in the file system whose inode numbers match those specified in the list of inodes. Each file prints one line on stdout containing the file's inode number, and the pathname to the file. The number and names are separated by a tab character. The starting directory is listed with the pathname given as the argument. All other lines have pathnames relative to the starting directory. The program uses the GPFS directory scan interface to quickly read the directory entries without calling "stat" on each file. The directories are scanned in parallel by multiple threads which results in an unsorted output file. Inode numbers that are not found in the tree are printed to stderr with the name "(notfound)". Example: >tsfindinode -i 23612 -i 1452 -i 47666 /gpfs 23612 include/stdio.h 1452 include/curses.h 47666 (notfound) > ------------------------------------------------------------------------------ tslistall - Fast scan of all files in a filesystem and display one line per file with the following information: inodenumber/generation mtime ctime type pathname Usage: tslistall fsPath [tmpdirname] where: fsPath path to file system tmpdirname is a directory where intermediate files for sort can be generated. The default directory id /tmp. This is necessary for GPFS filesystem that have large numbers of files. This is a shell script that invokes tsreaddir and tstimes then merges the resulting files. The output is printed to stdout with 1 line per file and is sorted by inode number. By saving the output of this command, you can compare it with a later list to find out what has changed in the filesystem since the old list was made. Example: >tslistall /gpfs 0000000003/0000000001 1012390401.913049600 1012390401.913049600 DIR /gpfs 0000001452/0000000003 1024702059.694025000 1024702059.694025000 LNK include/curses.h 0000001812/0000000000 1024702015.294047000 1024702015.294047000 DIR include 0000019578/0000000000 1026797433.109149000 1026794560.584238000 REG mail.rc 0000023612/0000000017 1024702090.705801000 1024702090.705801000 REG include/stdio.h 0000023874/0000000000 1024702024.982645000 1024702024.982645000 REG include/string.h > ------------------------------------------------------------------------------ tsreaddir - Fast parallel scan of all directories in a filesystem listing inodenumber type pathname Usage: tsreaddir [-d] [-n] [-t nThreads] [--] fsPath where: fsPath path to file system -d list only directories. -n do not list directories. -t set the number of threads (default = 16) -- marker so that names starting with "-" are not treated like options. This program recursively traverses the directory structure to list all accessible files and directories in the file system. Each file prints one line on stdout containing the file's inode number, the generation number, the file's type and the pathname to the file. The fields are separated by tab characters. If directories are listed, the starting directory is listed with the pathname given as the argument. All other lines have pathnames relative to the starting directory. The file types are "DIR" for directories, "REG" for regular files, "LNK" for symbolic links and "UNK" for all others. If the -d option is specified only directories are listed, and if the -n option is specified directories are not included in the output list. The program uses the GPFS directory scan interface to quickly read the directory entries without calling "stat" on each file. The directories are scanned in parallel by multiple threads which results in an unsorted output file. Example: >tsreaddir /gpfs 0000000003/0000000001 DIR /gpfs 0000001812/0000000000 DIR include 0000023612/0000000017 REG include/stdio.h 0000023874/0000000000 REG include/string.h 0000001452/0000000003 LNK include/curses.h 0000019578/0000000000 REG mail.rc > ------------------------------------------------------------------------------ tstimes - Fast scan of all files in a filesystem and display one line of inode mtime ctime for each file Usage: tstimes fsPath where: fsPath path to file system This program performs a gpfs inode scan on the specified file system to obtain the mtime and ctime for each file. The results are output on stdout, with one line per file containg the inode number, the generation number, the mtime seconds and nanoseconds GMT, and the ctime seconds and nanoseconds GMT. The output file is sorted by the inode numbers. The fields are separated by tab characters. Example: >tstimes /gpfs 0000000003/0000000001 1012390401.913049600 1012390401.913049600 0000001452/0000000003 1024702059.694025000 1024702059.694025000 0000001812/0000000000 1024702015.294047000 1024702015.294047000 0000019578/0000000000 1026797433.109149000 1026794560.584238000 0000023612/0000000017 1024702090.705801000 1024702090.705801000 0000023874/0000000000 1024702024.982645000 1024702024.982645000 > ------------------------------------------------------------------------------ mmbackup/tsbackup (see documentation for mmbackup in the GPFS Admin Guide) tsbackup - Fast parallel backup utility using multiple TSM clients and a single TSM server. Parallelism is achieved via the use of multi-threading and the use of multiple TSM client processes. This utility makes use of the GPFS inode scan APIs. Usage: mmbackup Device -n ControlFile [-t {full | incremental}] [-r IOrate] or mmbackup Device -R [-r IOrate] Parts: tsbackup.C (sample code in this directory) tsbackup.h (sample code in this directory) mmbackup.sh (script in /usr/lpp/mmfs/bin) mmexectsmcmd.sh (script in /usr/lpp/mmfs/bin)