【Build Xen & Lustre enabled kernel】

• Our OS: Dbian etch / ubuntu 8.04
• Our Method: Xen 和 Lustre 都需 patch kernel 來達到彼此的功能。
  我們使用較簡單但較笨的方法來 Merge Xen 和 Lustre ，主要步驟可分為三步驟：
  (1)安裝 XEN image (2)產生 XEN 和 Lustre 相容的 kernel (3)建立 Lustre 模組。

1.Requirement packages

• 1.1 Kernel Packages

  sudo aptitude install build-essential libncurses5-dev kernel-package
  

• 1.2 XEN Packages

  $ sudo apt-get install iproute bridge-utils python-twisted gcc binutils make zlib1g-dev python-dev transfig bzip2 \
      screen ssh debootstrap libcurl3-dev libncurses5-dev x-dev libsdl1.2-dev bin86 bcc python-pam patch latex-make \
      qemu qemu-launcher qemuctl graphviz graphviz-dev lvm2 libvncserver-dev libjpeg62-dev libjpeg62 gettext \
      python-xml gawk tetex-extra tetex-base openssl libssl-dev mercurial libc6-dev-i386
  

• 1.3 Lustre Packages

  $ sudo aptitude install bzip2 debhelper dpatch libsnmp-dev module-assistant quilt libxml2 python-xml
  

2.Download XEN & Lustre

• 2.1 Download XEN

  $ cd /usr/src
  $ sudo wget http://bits.xensource.com/oss-xen/release/3.1.4/xen-3.1.4.tar.gz
  $ sudo tar zxvf xen-3.1.4.tar.gz
  

• 2.2 Download Lustre
  • ​Lustre

    $ cd /usr/src
    $ tar zxvf lustre-1.6.6.tar.gz
    

3.Build Xen image

$ cd /usr/src/xen-3.2.1
$ sudo make -j 4 world
$ sudo make install

4.Build Xen & Lustre Kernel

• 4.1 Use Lustre's patch file to patch xen kernel

  $ cd /usr/src/xen-3.1.4/linux-2.6.18-xen
  $ sudo ln -s /usr/src/lustre-1.6.6/lustre/kernel_patches/series/2.6.18-vanilla.series series
  $ sudo ln -s /usr/src/lustre-1.6.6/lustre/kernel_patches/patches/ .
  $ sudo quilt -av push
  $ sudo cp /boot/config-2.6.18.8-xen .config
  $ sudo make menuconfig
  

• 4.2 Install XEN & Lustre patched kernel

  $ sudo make ; sudo make modules ; sudo make install ; sudo make modules_install
  $ sudo mkinitramfs -o /boot/initrd.img-2.6.18.8 2.6.18.8
  $ sudo update-grub
  $ sudo reboot
  

5.Make Lustre modules

$ cd /usr/src/lustre-1.6.5.1
$ sudo ./configure --with-linux=/usr/src/linux-2.6.18-xen.hg/  --with-linux-obj=/usr/src/xen-3.2.1/build-linux-2.6.18-xen_x86_64/
$ sudo make -j 4
$ sudo make install
$ sduo depmod -a

### Load modules in booting ###
$ sudo vim /etc/modules
loop max_loop=255
ldiskfs
lnet networks=tcp
$ sudo reboot

6.Push DRBL(option)

• Refer to ​https://trac.nchc.org.tw/grid/wiki/jazz/DRBL_Xen

7.Test Xen & Lustre

### For each client ###
$ sudo vim /etc/modules
loop max_loop=255
ldiskfs
lnet networks=tcp

$ sudo vim /etc/xen/xend-config.sxp
$ sudo xend start
$ sudo xm list

### Startup Lustre ###
#### Steup-> Meta Data & Management Server (virt) ####
$ sudo mkfs.lustre --fsname=testfs --mdt --mgs /dev/sdb1
$ sudo mount -t lustre /dev/sdb1 /home/mdt

#### Setup-> Object Storage Machine (virt01) ####
$ sudo mkfs.lustre --fsname=testfs --ost --mgsnode=virt@tcp0 /dev/sda
$ sudo mount -t lustre /dev/sda /home/ost
$ sudo dmesg
Lustre: flyfs-OST0000: received MDS connection from 140.110.X.X@tcp

#### Setup-> Object Storage Machine (virt02) ####
$ sudo mkfs.lustre --fsname=testfs --ost --mgsnode=virt@tcp0 /dev/sda
$ sudo mount -t lustre /dev/sda /home/ost

#### Setup-> Client (virt03) ####
$ sudo mount -t lustre virt@tcp0:/testfs /home/testfs
$ mount
$ df -h

### Auto mount ###
$ mount -l -t lustre
/dev/sdb1 on /home/mdt type lustre (rw) [flyfs-MDT0000]
/dev/sdb2 on /home/ost type lustre (rw) [flyfs-OST0000]

$ vim /etc/fstab 
LABEL=flyfs-MDT0000 /home/mdt   lustre  defaults,_netdev,noauto 0 0
LABEL=flyfs-OST0000 /home/ost   lustre  defaults,_netdev,noauto 0 0

8.Install xen-tools

• ​xen-tools

  $ wget http://www.xen-tools.org/software/xen-tools/xen-tools-3.9.tar.gz
  $ cd xen-tools-3.9
  $ sudo make install
  $ sudo vim /etc/xen-tools/xen-tools.conf
  

Reference