[[PageOutline]]

= Massive Deployment of Xen enabled Virtual Cluster using Diskless Remote Boot Linux =

== Test Environment ==

 * VMWare Server 1.0.3
 * VMWare Image debian 4.0r1 as DRBL Server
   * [[Image(drbl_xen_vmx.jpg)]]
 * VMWare Image without disk as DRBL Client
   * [[Image(drbl_xen_client_vmx.jpg)]]

== Pre-installation ==

 * First, install Xen Hypervisor and Xen enabled Linux Kernel and then reboot
{{{
root@drbl-xen-srv:~# uname -a
Linux drbl-xen-srv 2.6.18-6-686 #1 SMP Sat May 24 10:24:42 UTC 2008 i686 GNU/Linux
root@drbl-xen-srv:~# apt-get install xen-hypervisor-3.0.3-1-i386-pae linux-image-2.6.18-6-xen-vserver-686 xen-tools libc6-xen bridge-utils
root@drbl-xen-srv:~# update-grub
root@drbl-xen-srv:~# reboot
}}}
   * Note1: If you did not install libc6-xen, "4gb seg fixup" error message will keep showing up while booting.
   * [[Image(xen_4gb_seg_fixup.png)]]
   * Note2: amd64 architecture does not have libc6-xen package (i386 only)
 * Choose Xen to boot at GRUB menu
   * [[Image(xen_grub.png)]]

== Install and Configure DRBL ==

 * Download GPG-KEY-DRBL and add GPG key via '''apt-key'''
{{{
root@drbl-xen-srv:~# wget http://drbl.nchc.org.tw/GPG-KEY-DRBL
root@drbl-xen-srv:~# apt-key add GPG-KEY-DRBL
}}}
 * Add DRBL repository URL to /etc/apt/sources.list.d/ then install '''drbl''' package
{{{
root@drbl-xen-srv:~# echo "deb http://free.nchc.org.tw/drbl-core drbl stable" > /etc/apt/sources.list.d/drbl-sources.list
root@drbl-xen-srv:~# apt-get update
root@drbl-xen-srv:~# apt-get install drbl
}}}
 * Use '''drblsrv''' command to setup DRBL server. Here we choose ''linux-image-2.6.18-6-486'' as default DRBL PXE kernel image
{{{
root@drbl-xen-srv:~# /opt/drbl/sbin/drblsrv -i 
[....skip....]
Install kernel for clients... ...
In ayo repository, searching the latest kernel ...
*****************************************************.
Now run: /opt/drbl/sbin/drblsrv-offline -c -d -a -l en_US -k /var/cache/apt/archives/linux-image-2.6.18-6-486_2.6.18.dfsg.1-18etch5_i386.deb "" ""
Kernel package /var/cache/apt/archives/linux-image-2.6.18-6-486_2.6.18.dfsg.1-18etch5_i386.deb is used for client...
*****************************************************.
[....skip....]
}}}
 * Use '''drblpush''' command to deploy DRBL Client Image
{{{
root@drbl-xen-srv:~# /opt/drbl/sbin/drblpush -i
}}}
 * Use Thin Client VMWare Image to test DRBL
   * [[Image(drbl_xen_client_boot.jpg)]]

== Configure PXELinux for Xen Network Booting ==

 * To load Xen-based kernels you have to use the SYSLINUX multiboot module ''mboot.c32''. So we copy mboot.c32 and Xen hypervisor to TFTP root directory of DRBL.
{{{
root@drbl-xen-srv:~# dpkg -S mboot.c32
syslinux: /usr/lib/syslinux/mboot.c32
root@drbl-xen-srv:~# cp /usr/lib/syslinux/mboot.c32 /tftpboot/nbi_img/.
root@drbl-xen-srv:~# cp /boot/xen-3.0.3-1-i386-pae.gz /tftpboot/nbi_img/.
}}}
 * Use '''drblsrv-offline''' command to generate initial ramdisk of xen-based kernel which is needed for PXE network booting. You should find ''initrd-pxe.2.6.18-6-xen-vserver-686.img'' and ''vmlinuz-2.6.18-6-xen-vserver-686'' in /tftpboot/nbi_img/ folder.
{{{
root@drbl-xen-srv:~# /opt/drbl/sbin/drblsrv-offline -s `uname -r`
root@drbl-xen-srv:~# find /tftpboot/nbi_img/ -name "*-xen*"
/tftpboot/nbi_img/initrd-pxe.2.6.18-6-xen-vserver-686.img
/tftpboot/nbi_img/vmlinuz-2.6.18-6-xen-vserver-686
}}}
   * Note: ''initrd-pxe.2.6.18-6-xen-vserver-686.img'' is different from ''/boot/initrd.img-2.6.18-6-xen-vserver-686''. In ''initrd-pxe.2.6.18-6-xen-vserver-686.img'', there are NFS mounting scripts which is needed for each DRBL client.
 * Since Xen-based kernel can not boot with Xen hypervisor, we have to switch vmlinuz-pxe and initrd-pxe.img back to normal linux kernl.
{{{
root@drbl-xen-srv:~# /opt/drbl/sbin/drblsrv-offline -c -d -a -l en_US -k /var/cache/apt/archives/linux-image-2.6.18-6-486_2.6.18.dfsg.1-18etch5_i386.deb
}}}
   * Note: You can simply remove and recreate the symbolic links named by vmlinuz-pxe and initrd-pxe.img in /tftpboot/nbi_img
{{{
root@drbl-xen-srv:~# cd /tftpboot/nbi_img
root@drbl-xen-srv:/tftpboot/nbi_img# rm initrd-pxe.img vmlinuz-pxe
root@drbl-xen-srv:/tftpboot/nbi_img# ln -s initrd-pxe.2.6.18-6-486.img initrd-pxe.img
root@drbl-xen-srv:/tftpboot/nbi_img# ln -s vmlinuz-2.6.18-6-486 vmlinuz-pxe
}}}
 * Add a new Xen option to pxelinux.cfg
{{{
root@drbl-xen-srv:~# cat << EOF >> /tftpboot/nbi_img/pxelinux.cfg/default
> label xen cluster
>   MENU DEFAULT
>   # MENU HIDE
>   MENU LABEL Xen 3.0.1-1-i386-pae
>   # MENU PASSWD
>   kernel mboot.c32
>   append xen-3.0.3-1-i386-pae.gz --- vmlinuz-2.6.18-6-xen-vserver-686 root=/dev/sda1 ro console=tty0 --- initrd-pxe.2.6.18-6-xen-vserver-686.img
>   TEXT HELP
>   Run Xen 3.0 in each DRBL Client
>   ENDTEXT
> EOF
}}}
 * Use '''dcs''' to switch PXE menu
{{{
drbl-xen-srv:~# /opt/drbl/sbin/dcs more switch-pxe-menu
}}}
 * Use Thin Client VMWare Image to test DRBL
   * [[Image(drbl_xen_client_boot_2.jpg)]]
   * [[Image(drbl_xen_client_boot_3.jpg)]]

== Enable Xend in default rcX.d ==

 * You can run '''/etc/init.d/xend start''' manually to start xen service for each DRBL clients via '''drbl-doit''' command.
{{{
drbl-xen-srv:~# /opt/drbl/bin/drbl-doit /etc/init.d/xend start
}}}
 * You can check the status of xend by running '''xm list'''
{{{
drbl-xen-srv:~# /opt/drbl/bin/drbl-doit /usr/sbin/xm list
}}}
 * To save your time, we recommand you to add xend to default rcX.d by running '''update-rc.d xend defaults'''
{{{
drbl-xen-srv:~# /opt/drbl/bin/drbl-doit /usr/sbin/update-rc.d xend defaults
}}}

== Reference ==

 * [http://syslinux.zytor.com/wiki/index.php/Xen Syslinux Wiki]
 * [http://os-drive.com/files/docbook/xen-pxeboot.html Xen PXE Boot Howto]
 * [http://www.tjd.phlegethon.org/software/ Bootloader support for Xen]
 * If you are using debian, beware of bug [http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=391152 #391152]