GPFS Performance Report (Uses bonnie++ command)

rock, wade

Machine Informance

Node	8 nodes (1 server , 7 client provide disks)
CPU	Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)
Memory	2GB DDR2 667 (each node)
Disk	320G+160G (each node), All nodes: (320G+ 160G) * 7 = 3.36T
NIC	Intel Corporation 82566DM Gigabit Network Connection
Switch	D-link 24 port GE switch

Commande Introduction

Offical Web

​http://www.coker.com.au/bonnie++/
​http://www.coker.com.au/bonnie++/readme.html

Description

bonnie++ 是一套用來測試硬碟驅動程式及檔案系統效能的 benchmark，你可以決定那些測試是重要的，並且用它來比較兩個相異系統的差別。作者並不想將這些測試結果導向一個單一數字，因為他認為單一數字對比較這些東西並沒有什麼幫助。

Installation

sudo apt-get install bonnie++

Synopsis

 bonnie++         [-d         dir]         [-s         size(Mb)[:chunk-size(b)]]         [-n         num-
 ber-to-stat(*1024)[:max-size[:min-size][:num-directories]]] [-m machine-name]  [-r  ram-size-in-Mb]  [-x
 number-of-tests] [-u uid-to-use:gid-to-use] [-g gid-to-use] [-q] [-f] [-b] [-p processes | -y]

-d 要測試的目錄
-s 測試的檔案大小(MB)
-n 測試的檔案數(Byte)
-m 機器的名稱，自行命名，只會顯示在螢幕上，不影響結果
-r 指定記憶體大小(MB)
-x 測試次數
-u 你所要使用的 user-id，不建議使用 root
-g 你所要使用的 group-id
-q 安靜模式，只會顯示結果，其它額外的訊息會被隱藏
-f 快速模式，會跳過 per-char IO 測試
-b 不要寫入緩衝，fsync (即時寫入)
-p 有多少的 processes 要來服務 semaphores
-y 在每次測試前等待 semaphore

GPFS Performance

1. 8 Nodes, No Replicate, Adjust Parameters

gpfs-server:/home/gpfs/demo# bonnie++ -d /home/gpfs_mount/ -s 16000:1048576  -r 1024 -u root
Using uid:0, gid:0.
Writing with putc()...done                                             
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
gpfs-s 16000M:1024k 58779  95 105491   8 21720   4 50114  83 74658   7  55.4   7
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   295   0  1424   1   694   1   309   1  1820   3   414   1
gpfs-server,16000M:1024k,58779,95,105491,8,21720,4,50114,83,74658,7,55.4,7,16,295,0,1424,1,694,1,309,1,1820,3,414,1

2. 8 Nodes, Replicate, Adjust Parameters

bonnie++ -d /home/gpfs_mount/ -s 16000:1048576  -r 1024 -u root
Using uid:0, gid:0.
Writing with putc()...done
Writing intelligently...done
Rewriting...done
Reading with getc()...done
Reading intelligently...done
start 'em...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version  1.03       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
gpfs-s 16000M:1024k 52540  84 52879   4 20592   3 53244  91 79776   8  51.4   5
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   317   1  1171   1   658   1   366   1  1657   2   427   1
gpfs-server,16000M:1024k,52540,84,52879,4,20592,3,53244,91,79776,8,51.4,5,16,317,1,1171,1,658,1,366,1,1657,2,427,1

Parameters Comments

• machine name = gpfs-s
• rem size = 1024
• size = 16GB
• chunk-size = 1024k

Compare

寫入-per char(K/sec)		寫入-block(K/sec)	覆寫(K/sec)	讀取-per char(K/sec)	讀取-block(K/sec)	隨機讀取(K/sec)
8 Nodes, No Replicate, Adjust Parameters	58779	105491	21720	50114	74658	55.4
8 Nodes, Replicate, Adjust Parameters	52540	52879	20592	53244	79776	51.4

Running Information

gpfs-server:/home/gpfs_mount# ls -al
total 16384036
drwxr-xr-x  2 root root      32768 2008-03-04 15:21 .
drwxr-xr-x 14 root root       4096 2008-02-29 14:38 ..
-rw-------  1 root root 1073741824 2008-03-04 15:25 Bonnie.13706.000
-rw-------  1 root root 1073741824 2008-03-04 15:26 Bonnie.13706.001
-rw-------  1 root root 1073741824 2008-03-04 15:26 Bonnie.13706.002
-rw-------  1 root root 1073741824 2008-03-04 15:27 Bonnie.13706.003
-rw-------  1 root root 1073741824 2008-03-04 15:22 Bonnie.13706.004
-rw-------  1 root root 1073741824 2008-03-04 15:22 Bonnie.13706.005
-rw-------  1 root root 1073741824 2008-03-04 15:23 Bonnie.13706.006
-rw-------  1 root root 1073741824 2008-03-04 15:23 Bonnie.13706.007
-rw-------  1 root root 1073741824 2008-03-04 15:23 Bonnie.13706.008
-rw-------  1 root root 1073741824 2008-03-04 15:23 Bonnie.13706.009
-rw-------  1 root root 1073741824 2008-03-04 15:23 Bonnie.13706.010
-rw-------  1 root root 1073741824 2008-03-04 15:23 Bonnie.13706.011
-rw-------  1 root root 1073741824 2008-03-04 15:24 Bonnie.13706.012
-rw-------  1 root root 1073741824 2008-03-04 15:24 Bonnie.13706.013
-rw-------  1 root root 1073741824 2008-03-04 15:24 Bonnie.13706.014
-rw-------  1 root root  671088640 2008-03-04 15:24 Bonnie.13706.015
gpfs-server:/home/gpfs_mount# du -c
16384032        .
16384032        total