wiki:jazz/DRBL_Hadoop
close Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_fs.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.

Version 2 (modified by jazz, 16 years ago) (diff)

--

Deploy Hadoop to PC Classroom using DRBL

  • java is required for Hadoop, so you need to install java runtime or jdk first. 
    ~$ echo "deb http://free.nchc.org.tw/debian/ etch non-free" > /tmp/etch-non-free.list
~$ sudo mv /tmp/etch-non-free.list /etc/apt/sources.list.d/.
~$ sudo apt-get update
~$ sudo apt-get install sun-java5-jdk
  • download Hadoop 0.18.2 
    ~$ wget http://ftp.twaren.net/Unix/Web/apache/hadoop/core/hadoop-0.18.2/hadoop-0.18.2.tar.gz
~$ tar zxvf hadoop-0.18.2.tar.gz
  • setup JAVA_HOME environment variable 
    ~$ echo "export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun" >> ~/.bash_profile
~$ source ~/.bash_profile
  • edit hadoop-0.18.2/conf/hadoop-env.sh

    • hadoop-0.18.2/conf/hadoop-env.sh

      old new  
      66# remote nodes.
      77
      88# The java implementation to use.  Required.
      9 # export JAVA_HOME=/usr/lib/j2sdk1.5-sun
       9export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun
       10export HADOOP_HOME=/home/jazz/hadoop-0.18.2
       11export HADOOP_CONF_DIR=$HADOOP_HOME/conf
      1012
      1113# Extra Java CLASSPATH elements.  Optional.
      1214# export HADOOP_CLASSPATH=