| Version 15 (modified by chris, 16 years ago) (diff) |
|---|
Lustre File System Performance Report (Uses bonnie++ command)
Machine Informance
| Node | 4 nodes (1 server , 3 client provide disks) |
| CPU | Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node) |
| Memory | 2GB DDR2 667 (each node) |
| Disk | 320G+160G (each node), All nodes: (320G+ 160G) * 3 = 1.4T |
| NIC | Intel Corporation 82566DM Gigabit Network Connection |
| Switch | ZyXEL 8 port GE switch |
Offical Web
http://www.coker.com.au/bonnie++/
http://www.coker.com.au/bonnie++/readme.html
Lustre Performance
- without NFS tunning, kernel io scheduler = default
[root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576 -r 1024 -u root -m first Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP first 16000M:1024k 44277 73 44487 6 20676 7 44695 90 49761 12 60.2 17 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 6706 35 2971 31 5324 36 6314 37 2995 33 5812 36 first,16000M:1024k,44277,73,44487,6,20676,7,44695,90,49761,12,60.2,17,16,6706,35,2971,31,5324,36,6314,37,2995,33,5812,36 - with NFS tunning, kernel io scheduler=default
[root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576 -r 1024 -u root -m first Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP first 16000M:1024k 44359 72 51748 7 23290 8 46002 92 57361 15 64.5 18 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 6859 38 2978 33 5514 37 6494 36 3013 32 5577 35 first,16000M:1024k,44359,72,51748,7,23290,8,46002,92,57361,15,64.5,18,16,6859,38,2978,33,5514,37,6494,36,3013,32,5577,35 - with NFS tunning, kernel io scheduler = deadline
[root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576 -r 1024 -u root Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP lustre 16000M:1024k 51892 90 54387 8 24917 9 47740 95 63936 17 69.9 22 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 6745 31 3013 32 6486 38 6850 33 3079 30 6327 38 lustre,16000M:1024k,51892,90,54387,8,24917,9,47740,95,63936,17,69.9,22,16,6745,31,3013,32,6486,38,6850,33,3079,30,6327,38 - without nfs tunning, kernel io scheduler = cfq
[root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576 -r 1024 -u root Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP lustre 16000M:1024k 50986 80 46204 8 21749 8 46982 95 56664 15 65.2 21 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 8117 37 3062 31 6504 38 7041 35 2963 33 5579 34 lustre,16000M:1024k,50986,80,46204,8,21749,8,46982,95,56664,15,65.2,21,16,8117,37,3062,31,6504,38,7041,35,2963,33,5579,34 - with nfs tunning, kernel io scheduler = cfq
[root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576 -r 1024 -u root Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP lustre 16000M:1024k 51603 85 48624 7 22740 8 46542 97 59096 16 67.7 21 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 6940 33 2944 34 5327 41 6141 35 2999 33 5549 39 lustre,16000M:1024k,51603,85,48624,7,22740,8,46542,97,59096,16,67.7,21,16,6940,33,2944,34,5327,41,6141,35,2999,33,5549,39 - with nfs tunning, kernel io scheduler = as
[root@lustre bonnie++-1.03c]# ./bonnie++ -d /mnt/client_fs01/ -s 16000:1048576 -r 1024 -u root Using uid:0, gid:0. Writing with putc()...done Writing intelligently...done Rewriting...done Reading with getc()...done Reading intelligently...done start 'em...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.03c ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size:chnk K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP lustre 16000M:1024k 48623 78 51374 8 23598 9 47865 96 60014 16 68.6 21 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 8914 36 3153 31 7046 38 8020 39 3108 32 6087 39 lustre,16000M:1024k,48623,78,51374,8,23598,9,47865,96,60014,16,68.6,21,16,8914,36,3153,31,7046,38,8020,39,3108,32,6087,39
Attachments (3)
- Lustre with diff kernel_scheduler.PNG (19.3 KB) - added by chris 16 years ago.
- Lustre create with diff kernel_scheduler.PNG (20.4 KB) - added by chris 16 years ago.
- fake multi-clients.PNG (15.6 KB) - added by chris 16 years ago.
Download all attachments as: .zip
