source: gpfs_3.1_ker2.6.20/lpp/mmfs/samples/debugtools/README

===============================================================================
filehist       - Shell script wrapper for generating file space histograms
                 for all currently mounted GPFS filesystems
                 (uses /usr/lpp/mmfs/samples/util/tsinode
                  and  /usr/lpp/mmfs/samples/debugtools/tsfilehist.awk)
                 Generates one file for each filesystem called $fsname.filesum

tsfilehist.awk - Awk script for generating file space histograms on GPFS
                 filesystems. (Used by filehist script)

===============================================================================
trsum.awk      - Awk script for analyzing a GPFS trace and reporting various
                 statistics.

                 Example to show application calls and IO performed.
                     (The trace will wrap and only retain the records just
                      before the stop. See mmtrace -? for other options to
                      make the trace output larger.)

                   mmtrace trace=io
                   #run test program
                   mmtrace stop  #produces file with prefix /tmp/mmfs/trcrpt.
                   cd /tmp/mmfs
                   for file in trcrpt.*; do
                     echo $file
                     trsum.awk $file > trsum.${file#*trcrpt.}
                   done &

                 The trsum output shows system call entry/exit and elapsed time
                 along with arguments for each operation.
                 (trsum.awk details=0 ...  to turn this part off)

                 At the end of the report are summaries and performance histograms.

===============================================================================
fsstruct.awk   - Error report analysis on AIX clusters.
                 Build a file with the names of all nodes in the GPFS cluster:
                   mmcommon getNodeList|awk '{print $3}' > $WCOLL
                 Gather errpt -a output from all nodes
                   for node in $(<$WCOLL); do
                     mmdsh -L $node errpt -a 2>/dev/null
                   done > errpt.all &
                 Then summarize the errpt.all file with one line per event:
                   fsstruct.awk errpt.all  | sort -k1,1 > gpfs.err.summary
                   fsstruct.awk all=1 errpt.all  | sort -k1,1 > all.err.summary

===============================================================================
fsstructlx.awk - Error report analysis on Linux clusters.
                 Build a file with the names of all nodes in the GPFS cluster:
                   mmcommon getNodeList|awk '{print $3}' > $WCOLL
                 Gather /var/log/messages* from all nodes
                   for node in $(<$WCOLL); do
                     mmdsh -L $node 'cat $(ls -tr /var/log/messages*)' 2>/dev/null
                   done > syslogs.all &
                 Then summarize the syslogs.all file with one line per event:
                   fsstructlx.awk syslogs.all | sort -k1,1 > gpfs.err.summary
                   fsstructlx.awk all=1 syslogs.all | sort -k1,1 > all.err.summary

===============================================================================