wiki:jazz/DRBL_Xen

Version 22 (modified by jazz, 16 years ago) (diff)

--

Massive Deployment of Xen enabled Virtual Cluster using Diskless Remote Boot Linux

Test Environment

  • VMWare Server 1.0.3
  • VMWare Image debian 4.0r1 as DRBL Server
  • VMWare Image without disk as DRBL Client

Pre-installation

  • First, install Xen Hypervisor and Xen enabled Linux Kernel and then reboot
    root@drbl-xen-srv:~# uname -a
    Linux drbl-xen-srv 2.6.18-6-686 #1 SMP Sat May 24 10:24:42 UTC 2008 i686 GNU/Linux
    root@drbl-xen-srv:~# apt-get install xen-hypervisor-3.0.3-1-i386-pae linux-image-2.6.18-6-xen-vserver-686 xen-tools libc6-xen
    root@drbl-xen-srv:~# update-grub
    root@drbl-xen-srv:~# reboot
    
    • Note1: If you did not install libc6-xen, "4gb seg fixup" error message will keep showing up while booting.
    • Note2: amd64 architecture does not have libc6-xen package (i386 only)
  • Choose Xen to boot at GRUB menu

Install and Configure DRBL

  • Download GPG-KEY-DRBL and add GPG key via apt-key
    root@drbl-xen-srv:~# wget http://drbl.nchc.org.tw/GPG-KEY-DRBL
    root@drbl-xen-srv:~# apt-key add GPG-KEY-DRBL
    
  • Add DRBL repository URL to /etc/apt/sources.list.d/ then install drbl package
    root@drbl-xen-srv:~# echo "deb http://free.nchc.org.tw/drbl-core drbl stable" > /etc/apt/sources.list.d/drbl-sources.list
    root@drbl-xen-srv:~# apt-get update
    root@drbl-xen-srv:~# apt-get install drbl
    
  • Use drblsrv command to setup DRBL server. Here we choose linux-image-2.6.18-6-486 as default DRBL PXE kernel image
    root@drbl-xen-srv:~# /opt/drbl/sbin/drblsrv -i 
    [....skip....]
    Install kernel for clients... ...
    In ayo repository, searching the latest kernel ...
    *****************************************************.
    Now run: /opt/drbl/sbin/drblsrv-offline -c -d -a -l en_US -k /var/cache/apt/archives/linux-image-2.6.18-6-486_2.6.18.dfsg.1-18etch5_i386.deb "" ""
    Kernel package /var/cache/apt/archives/linux-image-2.6.18-6-486_2.6.18.dfsg.1-18etch5_i386.deb is used for client...
    *****************************************************.
    [....skip....]
    
  • Use drblpush command to deploy DRBL Client Image
    root@drbl-xen-srv:~# /opt/drbl/sbin/drblpush -i
    
  • Use Thin Client VMWare Image to test DRBL

Configure PXELinux for Xen Network Booting

  • To load Xen-based kernels you have to use the SYSLINUX multiboot module mboot.c32. So we copy mboot.c32 and Xen hypervisor to TFTP root directory of DRBL.
    root@drbl-xen-srv:~# dpkg -S mboot.c32
    syslinux: /usr/lib/syslinux/mboot.c32
    root@drbl-xen-srv:~# cp /usr/lib/syslinux/mboot.c32 /tftpboot/nbi_img/.
    root@drbl-xen-srv:~# cp /boot/xen-3.0.3-1-i386-pae.gz /tftpboot/nbi_img/.
    
  • Use drblsrv-offline command to generate initial ramdisk of xen-based kernel which is needed for PXE network booting. You should find initrd-pxe.2.6.18-6-xen-vserver-686.img and vmlinuz-2.6.18-6-xen-vserver-686 in /tftpboot/nbi_img/ folder.
    root@drbl-xen-srv:~# /opt/drbl/sbin/drblsrv-offline -s `uname -r`
    root@drbl-xen-srv:~# find /tftpboot/nbi_img/ -name "*-xen*"
    /tftpboot/nbi_img/initrd-pxe.2.6.18-6-xen-vserver-686.img
    /tftpboot/nbi_img/vmlinuz-2.6.18-6-xen-vserver-686
    
    • Note: initrd-pxe.2.6.18-6-xen-vserver-686.img is different from /boot/initrd.img-2.6.18-6-xen-vserver-686. In initrd-pxe.2.6.18-6-xen-vserver-686.img, there are NFS mounting scripts which is needed for each DRBL client.
  • Since Xen-based kernel can not boot with Xen hypervisor, we have to switch vmlinuz-pxe and initrd-pxe.img back to normal linux kernl.
    root@drbl-xen-srv:~# /opt/drbl/sbin/drblsrv-offline -c -d -a -l en_US -k /var/cache/apt/archives/linux-image-2.6.18-6-486_2.6.18.dfsg.1-18etch5_i386.deb
    
    • Note: You can simply remove and recreate the symbolic links named by vmlinuz-pxe and initrd-pxe.img in /tftpboot/nbi_img
      root@drbl-xen-srv:~# cd /tftpboot/nbi_img
      root@drbl-xen-srv:/tftpboot/nbi_img# rm initrd-pxe.img vmlinuz-pxe
      root@drbl-xen-srv:/tftpboot/nbi_img# ln -s initrd-pxe.2.6.18-6-486.img initrd-pxe.img
      root@drbl-xen-srv:/tftpboot/nbi_img# ln -s vmlinuz-2.6.18-6-486 vmlinuz-pxe
      
  • Add a new Xen option to pxelinux.cfg
    root@drbl-xen-srv:~# cat << EOF >> /tftpboot/nbi_img/pxelinux.cfg/default
    > label xen cluster
    >   MENU DEFAULT
    >   # MENU HIDE
    >   MENU LABEL Xen 3.0.1-1-i386-pae
    >   # MENU PASSWD
    >   kernel mboot.c32
    >   append xen-3.0.3-1-i386-pae.gz --- vmlinuz-2.6.18-6-xen-vserver-686 root=/dev/sda1 ro console=tty0 --- initrd-pxe.2.6.18-6-xen-vserver-686.img
    >   TEXT HELP
    >   Run Xen 3.0 in each DRBL Client
    >   ENDTEXT
    > EOF
    
  • Use dcs to switch PXE menu
    drbl-xen-srv:~# /opt/drbl/sbin/dcs more switch-pxe-menu
    
  • Use Thin Client VMWare Image to test DRBL

Enable Xend in default rcX.d

  • You can run /etc/init.d/xend start manually to start xen service for each DRBL clients via drbl-doit command.
    drbl-xen-srv:~# /opt/drbl/bin/drbl-doit /etc/init.d/xend start
    
  • You can check the status of xend by running xm list
    drbl-xen-srv:~# /opt/drbl/bin/drbl-doit /usr/sbin/xm list
    
  • To save your time, we recommand you to add xend to default rcX.d by running update-rc.d xend defaults
    drbl-xen-srv:~# /opt/drbl/bin/drbl-doit /usr/sbin/update-rc.d xend defaults
    

Reference

Attachments (7)

Download all attachments as: .zip