wiki:lustre_bonnie++

Version 15 (modified by chris, 16 years ago) (diff)

--

Lustre File System Performance Report (Uses bonnie++ command)



Machine Informance

Node 4 nodes (1 server , 3 client provide disks)
CPU Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)
Memory 2GB DDR2 667 (each node)
Disk 320G+160G (each node), All nodes: (320G+ 160G) * 3 = 1.4T
NIC Intel Corporation 82566DM Gigabit Network Connection
Switch ZyXEL 8 port GE switch



Offical Web

http://www.coker.com.au/bonnie++/
http://www.coker.com.au/bonnie++/readme.html

Lustre Performance

  • without NFS tunning, kernel io scheduler = default
    [root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576  -r 1024 -u root -m first
    Using uid:0, gid:0.
    Writing with putc()...done
    Writing intelligently...done
    Rewriting...done
    Reading with getc()...done
    Reading intelligently...done
    start 'em...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    first  16000M:1024k 44277  73 44487   6 20676   7 44695  90 49761  12  60.2  17
                        ------Sequential Create------ --------Random Create--------
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16  6706  35  2971  31  5324  36  6314  37  2995  33  5812  36
    first,16000M:1024k,44277,73,44487,6,20676,7,44695,90,49761,12,60.2,17,16,6706,35,2971,31,5324,36,6314,37,2995,33,5812,36
    
    
  • with NFS tunning, kernel io scheduler=default
    [root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576  -r 1024 -u root -m first
    Using uid:0, gid:0.
    Writing with putc()...done
    Writing intelligently...done
    Rewriting...done
    Reading with getc()...done
    Reading intelligently...done
    start 'em...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    first  16000M:1024k 44359  72 51748   7 23290   8 46002  92 57361  15  64.5  18
                        ------Sequential Create------ --------Random Create--------
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16  6859  38  2978  33  5514  37  6494  36  3013  32  5577  35
    first,16000M:1024k,44359,72,51748,7,23290,8,46002,92,57361,15,64.5,18,16,6859,38,2978,33,5514,37,6494,36,3013,32,5577,35
    
    
  • with NFS tunning, kernel io scheduler = deadline
    [root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576  -r 1024 -u root
    Using uid:0, gid:0.
    Writing with putc()...done
    Writing intelligently...done
    Rewriting...done
    Reading with getc()...done
    Reading intelligently...done
    start 'em...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    lustre 16000M:1024k 51892  90 54387   8 24917   9 47740  95 63936  17  69.9  22
                        ------Sequential Create------ --------Random Create--------
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16  6745  31  3013  32  6486  38  6850  33  3079  30  6327  38
    lustre,16000M:1024k,51892,90,54387,8,24917,9,47740,95,63936,17,69.9,22,16,6745,31,3013,32,6486,38,6850,33,3079,30,6327,38
    
  • without nfs tunning, kernel io scheduler = cfq
    [root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576  -r 1024 -u root
    Using uid:0, gid:0.
    Writing with putc()...done
    Writing intelligently...done
    Rewriting...done
    Reading with getc()...done
    Reading intelligently...done
    start 'em...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    lustre 16000M:1024k 50986  80 46204   8 21749   8 46982  95 56664  15  65.2  21
                        ------Sequential Create------ --------Random Create--------
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16  8117  37  3062  31  6504  38  7041  35  2963  33  5579  34
    lustre,16000M:1024k,50986,80,46204,8,21749,8,46982,95,56664,15,65.2,21,16,8117,37,3062,31,6504,38,7041,35,2963,33,5579,34
    
  • with nfs tunning, kernel io scheduler = cfq
    [root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576  -r 1024 -u root
    Using uid:0, gid:0.
    Writing with putc()...done
    Writing intelligently...done
    Rewriting...done
    Reading with getc()...done
    Reading intelligently...done
    start 'em...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    lustre 16000M:1024k 51603  85 48624   7 22740   8 46542  97 59096  16  67.7  21
                        ------Sequential Create------ --------Random Create--------
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16  6940  33  2944  34  5327  41  6141  35  2999  33  5549  39
    lustre,16000M:1024k,51603,85,48624,7,22740,8,46542,97,59096,16,67.7,21,16,6940,33,2944,34,5327,41,6141,35,2999,33,5549,39
    
  • with nfs tunning, kernel io scheduler = as
    [root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576  -r 1024 -u root
    Using uid:0, gid:0.
    Writing with putc()...done
    Writing intelligently...done
    Rewriting...done
    Reading with getc()...done
    Reading intelligently...done
    start 'em...done...done...done...
    Create files in sequential order...done.
    Stat files in sequential order...done.
    Delete files in sequential order...done.
    Create files in random order...done.
    Stat files in random order...done.
    Delete files in random order...done.
    Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                        -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
    Machine   Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
    lustre 16000M:1024k 48623  78 51374   8 23598   9 47865  96 60014  16  68.6  21
                        ------Sequential Create------ --------Random Create--------
                        -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
                  files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                     16  8914  36  3153  31  7046  38  8020  39  3108  32  6087  39
    lustre,16000M:1024k,48623,78,51374,8,23598,9,47865,96,60014,16,68.6,21,16,8914,36,3153,31,7046,38,8020,39,3108,32,6087,39
    

Attachments (3)

Download all attachments as: .zip