[[PageOutline]]
= 【Build Xen & Lustre enabled kernel】 =
 * '''Our OS''': Dbian etch / ubuntu 8.04
 * '''Our Method''': Xen 和 Lustre 都需 patch kernel 來達到彼此特有的功能。[[BR]]我們使用較簡單但較笨的方法來 Merge Xen 和 Lustre ，主要步驟可分為三步驟：[[BR]]'''''(1)安裝 XEN image (2)產生 XEN 和 Lustre 相容的 kernel (3)建立 Lustre 模組'''''。
[[BR]]

== 1.Requirement packages ==
 * __''1.1 Kernel Packages''__
{{{
sudo aptitude install build-essential libncurses5-dev kernel-package
}}}
 * ''__1.2 XEN Packages__''
{{{
$ sudo apt-get install iproute bridge-utils python-twisted gcc binutils make zlib1g-dev python-dev transfig bzip2 \
    screen ssh debootstrap libcurl3-dev libncurses5-dev x-dev libsdl1.2-dev bin86 bcc python-pam patch latex-make \
    qemu qemu-launcher qemuctl graphviz graphviz-dev lvm2 libvncserver-dev libjpeg62-dev libjpeg62 gettext \
    python-xml gawk tetex-extra tetex-base openssl libssl-dev mercurial libc6-dev-i386
}}}

 * __''1.3 Lustre Packages''__
{{{
$ sudo aptitude install bzip2 debhelper dpatch libsnmp-dev module-assistant quilt libxml2 python-xml
}}}
[[BR]]

== 2.Download XEN & Lustre ==
 * __''2.1 Download XEN''__
{{{
$ cd /usr/src
$ sudo wget http://bits.xensource.com/oss-xen/release/3.1.4/xen-3.1.4.tar.gz
$ sudo tar zxvf xen-3.1.4.tar.gz
}}}
 * __''2.2 Download Lustre''__
   * [http://www.sun.com/software/products/lustre/get.jsp Lustre]
{{{
$ cd /usr/src
$ tar zxvf lustre-1.6.5.1.tar.gz
}}}
[[BR]]

== 3.Build Xen Image ==
{{{
$ cd /usr/src/xen-3.2.1
$ sudo make -j 4 world
$ sudo make install
$ sudo rm /boot/vmlinuz-2.6.18.8-xen /boot/System.map-2.6.18.8-xen ; sudo rm -fr /lib/modules/2.6.18.8-xen/
}}}
[[BR]]

== 4.Build Xen & Lustre Kernel ==
 * __''4.1 Use Lustre's patch file to patch Xen kernel''__ 
{{{
$ cd /usr/src/xen-3.1.4/linux-2.6.18.8-xen
$ sudo ln -s /usr/src/lustre-1.6.5.1/lustre/kernel_patches/series/2.6.18-vanilla.series series
$ sudo ln -s /usr/src/lustre-1.6.5.1/lustre/kernel_patches/patches/ .
$ sudo quilt -av push
$ sudo cp /boot/config-2.6.18.8-xen .config

// Config  Kernel Option : (1)建議將未來需要的 driver build-in (2)在Local version - append to kernel release 加上 -xen //
$ sudo make menuconfig
}}}
 * __''4.2 Install XEN & Lustre patched kernel''__
{{{
$ sudo make ; sudo make modules ; sudo make install ; sudo make modules_install
$ sudo mkinitramfs -o /boot/initrd.img-2.6.18.8-xen 2.6.18.8-xen
$ sudo update-grub
$ sudo reboot
}}}
[[BR]]

== 5.Make Lustre modules ==
{{{
$ cd /usr/src/lustre-1.6.5.1
$ ln -s  /usr/src/xen-3.1.4/linux-2.6.18.8-xen/ /usr/src/linux
$ sudo ./configure --with-linux=/usr/src/xen-3.1.4/linux-2.6.18.8-xen/
$ sudo make -j 4
$ sudo make install
$ sduo depmod -a
$ sudo reboot
}}}
[[BR]]

== 6.Push DRBL(option) ==
 * Refer to https://trac.nchc.org.tw/grid/wiki/jazz/DRBL_Xen
[[BR]]

== 7.Test Xen & Lustre ==
 * __''7.1 Config Lustre module''__
{{{
$ sudo vim /etc/modules
ldiskfs
lnet networks=tcp

$ sudo sync ; sudo reboot
}}}
 * __''7.2 Config Xend''__
{{{
$ sudo vim /boot/grub/menu.lst
netloop.nloopbacks=150

$ sudo vim /etc/xen/xend-config.sxp
$ sudo xend start
$ sudo xm list
}}}
 * __''7.3 Setup Lsutre''__
   * ''Meta Data & Management Server''
{{{
$ sudo mkfs.lustre --fsname=testfs --mdt --mgs /dev/sdb1
$ sudo mount -t lustre /dev/sdb1 /home/mdt
}}}
   * ''Object Storage Machine''
{{{
$ sudo mkfs.lustre --fsname=testfs --ost --mgsnode=virt@tcp0 /dev/sda
$ sudo mount -t lustre /dev/sda /home/ost
$ sudo dmesg
Lustre: flyfs-OST0000: received MDS connection from 140.X.X.X@tcp
}}}
   * ''Mount Lustre FS''
{{{
$ sudo mount -t lustre virt@tcp0:/testfs /home/testfs
$ mount
$ df -h
}}}
   * ''Auto mount Lustre FS''
{{{
$ mount -l -t lustre
/dev/sdb1 on /home/mdt type lustre (rw) [flyfs-MDT0000]
/dev/sdb2 on /home/ost type lustre (rw) [flyfs-OST0000]

$ vim /etc/fstab 
LABEL=flyfs-MDT0000 /home/mdt   lustre  defaults,_netdev,noauto 0 0
LABEL=flyfs-OST0000 /home/ost   lustre  defaults,_netdev,noauto 0 0
}}}
[[BR]]

== 8.Install xen-tools ==
 * [http://www.xen-tools.org/software/xen-tools xen-tools]
{{{
$ wget http://www.xen-tools.org/software/xen-tools/xen-tools-3.9.tar.gz
$ cd xen-tools-3.9
$ sudo make install
$ sudo vim /etc/xen-tools/xen-tools.conf
}}}

[[BR]]

== Problem ==
 * Lustre 1.6.6 無法相容與 XEN ，再 make lustre modules 會有錯誤產生，建議改用 Lustre 1.6.5.1
[[BR]]

== Reference ==
 * [http://www.xen.org/ XEN]
 * [http://wiki.lustre.org/index.php?title=Main_Page Lustre] 
 * [http://drbl.nchc.org.tw/ DRBL]
 * [http://www.xen-tools.org/software/xen-tools xen-tools]