Build Xen & Lustre enabled kernel

• Our OS: Dbian etch
• Our Method: We used xen-patched kernel 2.6.18, then used Lustre patched file to patch it. We used xen tar ball to compile with this kernel to build Xen images amd used this kernel to reboot. Finally we used Lustre tar ball to build Lustre modules.

1.Requirement packages

• Build kernel packages:

  sudo aptitude install build-essential libncurses5-dev kernel-package
  

• Xen's packages:

  $ sudo apt-get install iproute bridge-utils python-twisted gcc binutils make zlib1g-dev python-dev transfig bzip2 \
      screen ssh debootstrap libcurl3-dev libncurses5-dev x-dev libsdl1.2-dev bin86 bcc python-pam patch latex-make \
      qemu qemu-launcher qemuctl graphviz graphviz-dev lvm2 libvncserver-dev libjpeg62-dev libjpeg62 gettext python-xml \
      gawk tetex-extra tetex-base openssl libssl-dev mercurial
  

• Lustre's packages:

  $ sudo aptitude install bzip2 debhelper dpatch libsnmp-dev module-assistant
  
  = Install libxml =
  $ sudo wget http://http.us.debian.org/debian/pool/main/libx/libxml2/libxml2_2.6.27.dfsg-2_i386.deb
  $ sudo dpkg -i libxml2_2.6.27.dfsg-2_i386.deb
  
  = Install pyxml =
  $ sudo wget PyXML-0.8.4.tar.gz
  $ sudo python setup.py build
  $ sudo python setup.py install
  

2.Patch & install xen images

• Download Xen-patched 2.6.18 kernel & Xen tar ball

  $ cd /usr/src
  $ sudo wget http://bits.xensource.com/oss-xen/release/3.2.1/xen-3.2.1.tar.gz
  $ sudo tar zxvf xen-3.2.1.tar.gz
  $ sudo hg clone http://xenbits.xensource.com/linux-2.6.18-xen.hg
  
  = Download Debian Lustre's deb file (http://packages.debian.org/source/sid/lustre) =
  $ wget http://debian.linux.org.tw/debian/pool/main/l/lustre/linux-patch-lustre_1.6.5-1_all.deb
  $ wget http://debian.linux.org.tw/debian/pool/main/l/lustre/lustre-source_1.6.5-1_all.deb
  $ dpkg -i linux-patch-lustre_1.6.5-1_all.deb lustre-source_1.6.5-1_all.deb
  
  = Use Lustre's patch file to patch xen kernel =
  $ cd linux-2.6.18-xen.hg
  $ sudo ln -s /usr/src/kernel-patches/lustre/series/2.6.18-vanilla.series /usr/src/linux-2.6.18-xen.hg/series
  $ sudo ln -s /usr/src/kernel-patches/lustre/patches/ /usr/src/linux-2.6.18-xen.hg/.
  $ sudo quilt -av push
  

3.Make Xen image

$ cd /usr/src/xen-3.2.1
$ sudo make world
（SCSI disk I/O must build-in）
$ sudo make install
$ sduo mkinitramfs -o /boot/initrd.img-2.6.18.8-xen 2.6.18.8-xen
$ sudo update-grub

$ cd /usr/src/linux-2.6.18-xen.hg

$ sudo reboot
(choose xen image to booting)

4.Make Lustre modules

$ sudo cp /boot/config-2.6.18.8-xen /usr/src/linux-2.6.18-xen.hg/.config
$ cd /usr/src/linux-2.6.18-xen.hg

= Because XEN rename 2.6.18.8 to 2.6.18.8-xen, so we rebuild kernel for corrent 2.6.18.8. Then Lustre will read correct kernel version =
$ sudo make menuconfig (or sudo make config)
$ sudo make -j 5
$ sudo make install
$ sudo make modules_install
$ sduo mkinitramfs -o /boot/initrd.img-2.6.18.8 2.6.18.8
$ sudo update-grub
$ sudo reboot
(choose new kernel to booting)

= Now, we start build and install lustre module and tools =
$ cd /usr/src
$ sudo tar jxvf lustre.tar.bz2
$ cd module/lustre
$ sudo ./configure -with-linux=/usr/src/linux-2.6.18-xen.hg
$ sudo make
$ sudo make install
$ sduo depmod -a

= Load modules in booting =
$ sudo vim /etc/modules
loop max_loop=255
ldiskfs
lnet networks=tcp

5.Push DRBL(If you don't push DRBL, you can skip it)

• refer to ​https://trac.nchc.org.tw/grid/wiki/jazz/DRBL_Xen

6.Test Xen & Lustre

= For each client =
$ sudo vim /etc/modules
loop max_loop=255
ldiskfs
lnet networks=tcp

$ sudo vim /etc/xen/xend-config.sxp
$ sudo xend start
$ sudo xm list

= Startup Lustre =
== Steup-> Meta Data & Management Server (virt) ==
$ sudo mkfs.lustre --fsname=testfs --mdt --mgs /dev/sdb1
== Setup-> Object Storage Machine (virt01) ==
$ sudo mkfs.lustre --fsname=testfs --ost --mgsnode=virt@tcp0 /dev/sda
== Setup-> Object Storage Machine (virt02) ==
$ sudo mkfs.lustre --fsname=testfs --ost --mgsnode=virt@tcp0 /dev/sda
== Setup-> Client (virt03) ==
$ sudo mount -t lustre virt@tcp0:/testfs /home/testfs

Reference