Build Lustre Storage Cluster for 3D Fly platform

0.Introduction

Some domain-applications usually need to computing huge data. It usually meet a bottleneck problem of performance. It will lead I/O performance to drag total performance. Thereforce, we use Lustre high performance stroage to improve I/O performance. You can refer to Lustre web site ​http://wiki.lustre.org/index.php?title=Main_Page

1.Machine information & Software

• Hardware

Node	8 nodes (1 server , 7 client provide disks)
CPU	Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)
Memory	2GB DDR2 667 (each node)
Disk	All nodes: 320G * 7 = 2.24T
NIC	Intel Corporation 82566DM Gigabit Network Connection
Switch	D-link 24 port GE switch

• Software

OS	OpenSuSE 10.2
DRBL	1.9.0-35
Amira	Testing Version
Kernel	2.6.18

2.Config OpenSuSE

• Install packages for Lustre
• Install packages for DRBL

3.Install Lustre

• Build kernel

  $ sudo su
  $ cd /usr/src
  $ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2
  $ tar jxvf linux-2.6.18.tar.bz2
  $ cd 2.6.18
  $ cp /boot/config-xxx .config
  $ make oldconfig ; make menuconfig
  $ make -j 5
  $ make modules
  $ make modules_install
  $ make install
  $ mkinitrd
  $ vim /boot/grub/menu.list
  

• recompiler NIC driver

  wget http://trac.nchc.org.tw/grid/attachment/wiki/Problem/e1000-7.6.15.4.tar.gz?format=raw
  tar e1000-7.6.15.4.tar.gz
  cd e1000-7.6.15.4/src
  make install
  

• Bonding NIC (bond0 -> eth1 & eth2)

  $ cd /etc/sysconfig/network
  $ vim ifcfg-eth1 
  $ vim ifeth-eth2
  $ vim ifcfg-bond0
  $ vim /etc/rc.d/after.local
  $ vim /etc/module/modules.conf.local
  $ vim /etc/rc.d/after.local
  
  

• Build Lustre Modules

  $ configure --with-linux=/usr/src/2.6.18
  $ make install
  $ depmod -a
  $ vim /etc/rc.d/after.local
  

4.Build DRBL Environment

$ 
check node_root/etc/rc.d/module.conf.local
check nodes/192.168.1.X/etc/rc.d/module.conf.local
rm /tftfpood/node_root/sysconf/network/if

5.Build Lustre Storage Cluster

• setup autologin

  = fly =
  mkfs.lustre --fsname=flyfs --mdt --mgs /dev/sdb1
  mount -t lustre /dev/sdb1 /home/mdt
  = fly01 ~ fly07 =
  mkfs.lustre --fsname=flyfs --ost  --mgsnode=fly@tcp0 /dev/sdb
  mount -t lustre /dev/sdb /home/ost
  = fly =
  mkdir /home/flyfs
  mount -t lustre fly@tcp0:/flyfs /home/flyfs
  
  

6.Install Amira

7.Test Amira

Reference