2009-04-28

• 雲端運算基礎課程

DRBL

• 測試 Michael Smith 的 Clonezilla SE Web UI
  • 測試環境: Debian Lenny

    $ lsb_release -a
    Distributor ID: Debian
    Description:   Debian GNU/Linux 5.0.1 (lenny)
    Release:     5.0.1
    Codename:    lenny
    

  • 基本套件相依：根據 README 文件，需要　php-gd, php-mysql, oddjobd, mysql-server, apache, drbl/clonezilla

    $ sudo apt-get install php5-gd php5-mysql mysql-server drbl
    

  • 其中 ​oddjob 因為是 Redhat 的人開發的，所以目前仍需要額外自行下載安裝，我用 checkinstall 來產生 deb 套件。在 ./configure 過程中發現相依一些套件包含 D-Bus-1 , SELinux Header, libxml-2.0,　libpam0g-dev　套件裡的 /usr/include/security/pam_modules.h 跟 python2.5-dev 裡的 /usr/include/python2.5/Python.h (用 apt-file 去找)

    $ sudo apt-get install libdbus-1-dev libxml2-dev libselinux1-dev libpam0g-dev python2.5-dev
    $ wget http://people.redhat.com/nalin/oddjob/oddjob-0.29.1-1.tar.gz
    $ tar zxvf oddjob-0.29.1-1.tar.gz
    $ cd oddjob-0.29.1-1/
    ~/oddjob-0.29.1-1$ ./configure
    ~/oddjob-0.29.1-1$ sudo apt-get install checkinstall
    ~/oddjob-0.29.1-1$ checkinstall
    
    checkinstall 1.6.1, Copyright 2002 Felipe Eduardo Sanchez Diaz Duran
               This software is released under the GNU GPL.
    
    *****************************************
    **** Debian package creation selected ***
    *****************************************
    
    This package will be built according to these values:
    
    0 -  Maintainer: [ jazz@drbl ]
    1 -  Summary: [ The oddjob package provides a means for unprivileged applications ]
    2 -  Name:    [ oddjob-0.29.1 ]
    3 -  Version: [ 1 ]
    4 -  Release: [ 1 ]
    5 -  License: [ GPL ]
    6 -  Group:   [ checkinstall ]
    7 -  Architecture: [ amd64 ]
    8 -  Source location: [ oddjob-0.29.1-1 ]
    9 -  Alternate source location: [  ]
    10 - Requires: [  ]
    11 - Provides: [ oddjob-0.29.1 ]
    
    Enter a number to change any of them or press ENTER to continue:
    
    ~/oddjob-0.29.1-1$ sudo dpkg --info oddjob-0.29.1_0.29.1-1nchc_amd64.deb
     Package: oddjob-0.29.1
     Priority: extra
     Section: optional
     Installed-Size: 768
     Maintainer: Jazz Yao-Tsung Wang
     Architecture: amd64
     Version: 0.29.1-1nchc
     Depends: libdbus-1-dev, libxml2-dev, libselinux1-dev, libpam0g-dev, python2.5-dev
     Provides: oddjob-0.29.1
     Description: invoke and control applications by issuing requests using the D-Bus
      The oddjob package provides a means for unprivileged applications
      to invoke and take limited control of privileged applications by
      issuing requests using the D-Bus system message bus.
    

Hadoop

• 測試 ​Cloudera 的 Hadoop Debian Package

  ~# lsb_release -a
  Distributor ID: Ubuntu
  Description:   Ubuntu 8.10
  Release:     8.10
  Codename:     intrepid
  
  ~# cat > /etc/apt/sources.list.d/hadoop.list << EOF
  deb http://archive.cloudera.com/debian intrepid contrib
  EOF
  
  ~# apt-get update
  ~# apt-get install hadoop
  ~# /opt/drbl/sbin/drblsrv -i
  ~# /opt/drbl/sbin/drblpush -i
  

• 編輯 DRBL 上的 hadoop-site.xml

  ~# cat > /etc/hadoop/conf/hadoop-site.xml << EOF
  <?xml version="1.0"?>
  <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
  
  <!-- Put site-specific property overrides in this file. -->
  
  <configuration>
    <property>
      <name>fs.default.name</name>
      <value>hdfs://192.168.166.254:9000/</value>
      <description> </description>
    </property>
    <property>
      <name>mapred.job.tracker</name>
      <value>192.168.166.254:9001</value>
      <description>  </description>
    </property>
    <property>
      <name>hadoop.tmp.dir</name>
      <value>/var/run/hadoop/${user.name}</value>
      <description> </description>
    </property>
  </configuration>
  EOF
  

• 啟動 Namenode, Secondary Namenode, Jobtracker 在 DRBL Server 並且用 drbl-client-service 指令讓每個 DRBL Client 預設都當 Task Tracker 跟 Datanode。最後執行 re-deploy

  ~# update-rc.d hadoop-namenode defaults
  ~# update-rc.d hadoop-secondarynamenode defaults
  ~# update-rc.d hadoop-jobtracker defaults
  ~# /etc/init.d/hadoop-namenode start
  ~# /etc/init.d/hadoop-secondarynamenode start
  ~# /etc/init.d/hadoop-tasktracker start
  ~# /opt/drbl/sbin/drbl-client-service tasktracker on
  ~# /opt/drbl/sbin/drbl-client-service datanode on
  ~# /opt/drbl/sbin/drblpush -c /etc/drbl/drblpush.conf