GPFS Performance Report (Uses bonnie++ command)

rock, wade, 阿碩

Machine Informance

Node	8 nodes (1 server , 7 client provide disks)
CPU	Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)
Memory	2GB DDR2 667 (each node)
Disk	320G+160G (each node), All nodes: (320G+ 160G) * 7 = 3.36T
NIC	Intel Corporation 82566DM Gigabit Network Connection
Switch	D-link 24 port GE switch

Command Introduction

Offical Web

​http://www.iozone.org/
Documentation：​PDF、​Postscript gzip、​MS Word Doc

Description

IOzone是一套檔案系統的 benchmark。

Installation

sudo apt-get install iozone3 

Synopsis

iozone [-s filesize_Kb] [-r record_size_Kb] [-f [path]filename] [-h]
             [-i test] [-E] [-p] [-a] [-A] [-z] [-Z] [-m] [-M] [-t children]
             [-l min_number_procs] [-u max_number_procs] [-v] [-R] [-x] [-o]
             [-d microseconds] [-F path1 path2...] [-V pattern] [-j stride]
             [-T] [-C] [-B] [-D] [-G] [-I] [-H depth] [-k depth] [-U mount_point]
             [-S cache_size] [-O] [-L cacheline_size] [-K] [-g maxfilesize_Kb]
             [-n minfilesize_Kb] [-N] [-Q] [-P start_cpu] [-e] [-c] [-b Excel.xls]
             [-J milliseconds] [-X write_telemetry_filename] [-w] [-W]
             [-Y read_telemetry_filename] [-y minrecsize_Kb] [-q maxrecsize_Kb]
             [-+u] [-+m cluster_filename] [-+d] [-+x multiplier] [-+p # ]
             [-+r] [-+t] [-+X] [-+Z]

           -a  Auto mode
           -A  Auto2 mode
           -b Filename  Create Excel worksheet file
           -B  Use mmap() files
           -c  Include close in the timing calculations
           -C  Show bytes transferred by each child in throughput testing
           -d #  Microsecond delay out of barrier
           -D  Use msync(MS_ASYNC) on mmap files
           -e  Include flush (fsync,fflush) in the timing calculations
           -E  Run extension tests
           -f filename  to use
           -F filenames  for each process/thread in throughput test
           -g #  Set maximum file size (in Kbytes) for auto mode (or #m or #g)
           -G  Use msync(MS_SYNC) on mmap files
           -h  help
           -H #  Use POSIX async I/O with # async operations
           -i #  Test to run (0=write/rewrite, 1=read/re-read, 2=random-read/write
                 3=Read-backwards, 4=Re-write-record, 5=stride-read, 6=fwrite/re-fwrite
                 7=fread/Re-fread, 8=random_mix, 9=pwrite/Re-pwrite, 10=pread/Re-pread
                 11=pwritev/Re-pwritev, 12=preadv/Re-preadv)
           -I  Use VxFS VX_DIRECT, O_DIRECT,or O_DIRECTIO for all file operations
           -j #  Set stride of file accesses to (# * record size)
           -J #  milliseconds of compute cycle before each I/O operation
           -k #  Use POSIX async I/O (no bcopy) with # async operations
           -K  Create jitter in the access pattern for readers
           -l #  Lower limit on number of processes to run
           -L #  Set processor cache line size to value (in bytes)
           -m  Use multiple buffers
           -M  Report uname -a output
           -n #  Set minimum file size (in Kbytes) for auto mode (or #m or #g)
           -N  Report results in microseconds per operation
           -o  Writes are synch (O_SYNC)
           -O  Give results in ops/sec.
           -p  Purge on
           -P #  Bind processes/threads to processors, starting with this cpu
           -q #  Set maximum record size (in Kbytes) for auto mode (or #m or #g)
           -Q  Create offset/latency files
           -r #  record size in Kb
              or -r #k .. size in Kb
              or -r #m .. size in Mb
              or -r #g .. size in Gb
           -R  Generate Excel report
           -s #  file size in Kb
              or -s #k .. size in Kb
              or -s #m .. size in Mb
              or -s #g .. size in Gb
           -S #  Set processor cache size to value (in Kbytes)
           -t #  Number of threads or processes to use in throughput test
           -T  Use POSIX pthreads for throughput tests
           -u #  Upper limit on number of processes to run
           -U  Mount point to remount between tests
           -v  version information
           -V #  Verify data pattern write/read
           -w  Do not unlink temporary file
           -W  Lock file when reading or writing
           -x  Turn off stone-walling
           -X filename  Write telemetry file. Contains lines with (offset reclen compute_time) in ascii
           -y #  Set minimum record size (in Kbytes) for auto mode (or #m or #g)
           -Y filename  Read  telemetry file. Contains lines with (offset reclen compute_time) in ascii
           -z  Used in conjunction with -a to test all possible record sizes
           -Z  Enable mixing of mmap I/O and file I/O
           -+K Sony special. Manual control of test 8.
           -+m  Cluster_filename   Enable Cluster testing
           -+d  File I/O diagnostic mode. (To troubleshoot a broken file I/O subsystem)
           -+u  Enable CPU utilization output (Experimental)
           -+x # Multiplier to use for incrementing file and record sizes
           -+p # Percentage of mix to be reads
           -+r Enable O_RSYNC|O_SYNC for all testing.
           -+t Enable network performance test. Requires -+m 
           -+n No retests selected.
           -+k Use constant aggregate data set size.
           -+q Delay in seconds between tests.
           -+l Enable record locking mode.
           -+L Enable record locking mode, with shared file.
           -+B Sequential mixed workload.
           -+D Enable O_DSYNC mode.
           -+A #  Enable madvise. 0 = normal, 1=random, 2=sequential
                                  3=dontneed, 4=willneed
           -+V Enable shared file. No locking.
           -+X Enable short circuit mode for filesystem testing ONLY
               ALL Results are NOT valid in this mode.
           -+Z Enable old data set compatibility mode. WARNING.. Published
               hacks may invalidate these results and generate bogus, high
               values for results.

GPFS Performance

Parameters Comments

• records sizes =4KB to 16MB
• file sizes = 64KB to 16GB

Compare with different recordsize

Writer Report

	16KB	128KB	256KB	512KB	1MB	2MB	4MB	8MB	16MB	32MB	64MB	128MB	256MB	512MB	1GB	2GB	4GB	8GB	16GB
4	183899	561388	780757	819060	867768	709151	722018	66022	98084	0	0	0	0	0	0	0	0	0	0
8	507534	681128	1007946	921007	1135353	64140	911234	963085	39168	0	0	0	0	0	0	0	0	0	0
16	500370	842584	992428	1024109	1179665	772856	128389	1006009	90290	0	0	0	0	0	0	0	0	0	0
32	533049	794491	1230499	1150614	48238	69252	1341613	1207908	101242	0	0	0	0	0	0	0	0	0	0
64	556091	920608	1369084	1672585	1756707	69369	1144460	1140806	87574	62245	55415	55464	53076	54488	54706	52839	53292	52977	53019
128	1058568		1765482	1570856	1835473	1708237	1141617	903500	102208	52044	61418	59490	54913	55247	53107	52879	52817	53791	53072
256	1231341			1475062	1296378	1662371	79833	1149576	84296	65247	59252	60519	54215	52875	53087	52963	53023	53035	52861
512	1730210				1007970	1745776	1535286	778557	39674	51691	48634	56133	54275	56336	53116	53333	53181	52937	53379
1024	1984640					1447339	1268157	1010125	100517	73908	59416	55873	53971	58759	53729	53535	53146	53008	53173
2048	1220492						1033584	931124	93311	65882	59456	56138	54319	54684	53009	52852	52833	52949	53029
4096	871282							875016	101678	70289	54114	55192	55079	55954	53763	52916	52851	53277	52794
8192	776940								109738	49325	54305	59157	54447	52906	56208	52853	53488	53020	52801
16384	96733									54134	55499	55904	56976	52692	53040	53396	52891	53487	52808

• records size = 4KB