GPFS Performance Report (Uses dd command)

rock, rock@…

0. Machine Informance

Node	8 nodes (1 server , 7 client provide disks)
CPU	Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)
Memory	2GB DDR2 667 (each node)
Disk	320G+160G (each node), All nodes: (320G+ 160G) * 7 = 3.36T
NIC	Intel Corporation 82566DM Gigabit Network Connection
Switch	D-link 24 port GE switch

1. 8 Nodes, Enable Replicate, Default Parameters

• Test Command: dd

gpfs-server:/usr/lpp/mmfs/samples/perf# dd if=/dev/zero of=/home/gpfs_mount/dd.test.16G bs=1G count=16 
16+0 records in 
16+0 records out 
17179869184 bytes (17 GB) copied, 329.694 seconds, 52.1 MB/s

• Server Statistics Informance

gpfs-server:/home/rock# dstat -c -d -M gpfs -n -N eth1 -l 5 
----total-cpu-usage---- -dsk/total- --gpfs-i/o- --net/eth1- ---load-avg--- 
usr sys idl wai hiq siq|_read _writ|_read write|_recv _send|_1m_ _5m_ 15m_ 
  0   0  99   0   0   1|3558B 8599B|   0     0 |   0     0 | 0.3  0.3  0.3 
  0   4  90   0   1   5|   0    50k|   0    51M|1270k   93M| 0.3  0.3  0.3 
  0   4  90   0   1   5|   0    25k|   0    52M|1961k  142M| 0.5  0.3  0.3 
  0   4  90   0   1   5|   0    42k|   0    57M|1281k   93M| 1.1  0.5  0.4 
  0   4  90   0   1   5|   0  3277B|   0    56M|1291k   95M| 1.1  0.5  0.4 
  0   4  89   0   1   5|   0     0 |   0    57M|1725k    - | 1.1  0.5  0.4 

• Client Statistics Informance

gpfs04:~# dstat -c -d -M gpfs -n -N eth0 -l 5 
----total-cpu-usage---- -dsk/total- --gpfs-i/o- --net/eth0- ---load-avg--- 
usr sys idl wai hiq siq|_read _writ|_read write|_recv _send|_1m_ _5m_ 15m_ 
  0   0 100   0   0   0| 418B   26k|   0     0 |   0     0 | 0.1  0.1  0.1 
  0   1  93   6   0   1|   0    15M|   0     0 |  16M  355k| 0.1  0.1  0.1 
  0   0  91   7   0   1|   0    14M|   0     0 |  15M  331k| 0.1  0.1  0.1 
  0   1  89  10   0   1|   0    15M|   0     0 |  16M  358k| 0.1  0.1  0.1 
  0   1  92   7   0   1|   0    15M|   0     0 |  15M  350k| 0.1  0.1  0.1 
  0   1  93   6   0   1|   0    15M|   0     0 |  15M  348k| 0.1  0.1  0.1 
  0   1  92   6   0   1|   0    17M|   0     0 |  17M  386k| 0.1  0.1  0.1 

2. 8 Nodes, No Replicate, Default Parameter

• Test Command: dd

  gpfs-server:/usr/lpp/mmfs/samples/perf# dd if=/dev/zero of=/home/gpfs_mount/dd.test.16G_2 bs=1G count=16 
  16+0 records in 
  16+0 records out 
  17179869184 bytes (17 GB) copied, 165.948 seconds, 104 MB/s 
  

• Server Statistics Informance

  gpfs04:~# dstat -c -d -M gpfs -n -N eth0 -l 5 
  ----total-cpu-usage---- -dsk/total- --gpfs-i/o- --net/eth0- ---load-avg--- 
  usr sys idl wai hiq siq|_read _writ|_read write|_recv _send|_1m_ _5m_ 15m_ 
    0   0 100   0   0   0| 419B   19k|   0     0 |   0     0 | 0.1  0.1  0.1 
    0   0  93   6   0   1|   0    15M|   0     0 |  16M  359k| 0.1  0.1  0.1 
    0   0  93   6   0   1|   0    15M|   0     0 |  16M  359k| 0.1  0.1  0.1 
    0   0  94   5   0   1|   0    14M|   0     0 |  15M  335k| 0.1  0.1  0.1 
    0   0  92   6   0   1|   0    14M|   0     0 |  15M  331k| 0.2  0.1  0.1 
    0   1  90   8   0   1|   0    14M|   0     0 |  15M  333k| 0.2  0.1  0.1 
    0   0  92   7   0   1|   0    12M|   0     0 |  13M  296k| 0.2  0.1  0.1 
  

• Client Statistics Informance

  gpfs-server:/home/rock# dstat -c -d -M gpfs -n -N eth1 -l 5 
  ----total-cpu-usage---- -dsk/total- --gpfs-i/o- --net/eth1- ---load-avg--- 
  usr sys idl wai hiq siq|_read _writ|_read write|_recv _send|_1m_ _5m_ 15m_ 
    0   0  99   0   0   1|3562B 8607B|   0     0 |   0     0 | 0.8  0.4  0.3 
    0   5  88   0   1   5|   0     0 |   0   101M|1227k   89M| 0.8  0.5  0.3 
    0   4  90   0   1   5|   0    49k|   0   107M|1899k  136M| 0.9  0.5  0.3 
    0   5  89   0   1   4|   0     0 |   0   103M|1202k   86M| 0.9  0.5  0.3 
    0   4  89   0   1   5|   0     0 |   0   108M|1859k  135M|   1  0.5  0.3 
    1   5  88   0   1   5|   0     0 |   0   104M|1204k   84M|   1  0.5  0.3 
    0   5  89   0   1   5|   0     0 |   0   107M|1912k  137M| 1.1  0.5  0.3 
  

3. 8 Nodes, Replicate, Adjust Parameters

• Test Command: dd

  gpfs-server:/home/gpfs_mount# dd if=/dev/zero of=/home/gpfs_mount/dd.test.16G_5 bs=1G count=16 
  16+0 records in 
  16+0 records out 
  17179869184 bytes (17 GB) copied, 317.796 seconds, 54.1 MB/s
  

4. 8 Nodes, No Replicate, Adjust Parameters

• Test Command: dd

  gpfs-server:/home/gpfs_mount# dd if=/dev/zero of=/home/gpfs_mount/dd.test.16G_4 bs=1G count=16 
  16+0 records in 
  16+0 records out 
  17179869184 bytes (17 GB) copied, 158.117 seconds, 109 MB/s
  

5. Compare

Replicate & No Parameters	No Replicate & No Parameters	Replicate & Adjust Parameters	No Replicate & Adjust Parameters
52.1 MB/s	105 MB/s	54.1 MB /s	109 MB/s