wiki:3D_Lustre

Version 3 (modified by rock, 16 years ago) (diff)

--

Build Lustre Storage Cluster for 3D Fly platform

0.Introduction

Some domain-applications usually need to computing huge data. It usually meet a bottleneck problem of performance. It will lead I/O performance to drag total performance. Thereforce, we use Lustre high performance stroage to improve I/O performance. You can refer to Lustre web site http://wiki.lustre.org/index.php?title=Main_Page

1.Machine information & Software

  • Hardware
Node 8 nodes (1 server , 7 client provide disks)
CPU Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz (each node)
Memory 2GB DDR2 667 (each node)
Disk All nodes: 320G * 7 = 2.24T
NIC Intel Corporation 82566DM Gigabit Network Connection
Switch D-link 24 port GE switch
  • Software
OSOpenSuSE 10.2
DRBL1.9.0-35
AmiraTesting Version
Kernel2.6.18

2.Config OpenSuSE

  • Install packages for Lustre
  • Install packages for DRBL

3.Install Lustre

  • Build kernel
    $ sudo su
    $ cd /usr/src
    $ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.18.tar.bz2
    $ tar jxvf linux-2.6.18.tar.bz2
    $ cd 2.6.18
    $ cp /boot/config-xxx .config
    $ make oldconfig ; make menuconfig
    $ make -j 5
    $ make modules
    $ make modules_install
    $ make install
    $ mkinitrd
    $ vim /boot/grub/menu.list
    
  • recompiler NIC driver
    wget http://trac.nchc.org.tw/grid/attachment/wiki/Problem/e1000-7.6.15.4.tar.gz?format=raw
    tar e1000-7.6.15.4.tar.gz
    cd e1000-7.6.15.4/src
    make install
    
  • Bonding NIC (bond0 -> eth1 & eth2)
    $ cd /etc/sysconfig/network
    $ vim ifcfg-eth1 
    $ vim ifeth-eth2
    $ vim ifcfg-bond0
    $ vim /etc/rc.d/after.local
    $ vim /etc/module/modules.conf.local
    $ vim /etc/rc.d/after.local
    
    
  • Build Lustre Modules
    $ configure --with-linux=/usr/src/2.6.18
    $ make install
    $ depmod -a
    $ vim /etc/rc.d/after.local
    

4.Build DRBL Environment

$ 
check node_root/etc/rc.d/module.conf.local
check nodes/192.168.1.X/etc/rc.d/module.conf.local
rm /tftfpood/node_root/sysconf/network/if

5.Build Lustre Storage Cluster

  • setup autologin
    = fly =
    mkfs.lustre --fsname=flyfs --mdt --mgs /dev/sdb1
    mount -t lustre /dev/sdb1 /home/mdt
    = fly01 ~ fly07 =
    mkfs.lustre --fsname=flyfs --ost  --mgsnode=fly@tcp0 /dev/sdb
    mount -t lustre /dev/sdb /home/ost
    = fly =
    mkdir /home/flyfs
    mount -t lustre fly@tcp0:/flyfs /home/flyfs
    
    

6.Install Amira

7.Test Amira

Reference

Attachments (4)

Download all attachments as: .zip