| 1 | == Hadoop Hands-on Labs (1) == |
| 2 | |
| 3 | * download hadoop-0.18.2 |
| 4 | {{{ |
| 5 | $ cd ~ |
| 6 | $ wget http://ftp.twaren.net/Unix/Web/apache/hadoop/core/hadoop-0.18.2/hadoop-0.18.2.tar.gz |
| 7 | $ tar zxvf hadoop-0.18.2.tar.gz |
| 8 | }}} |
| 9 | * 1. Hadoop 會用 SSH 進行內部連線,因此需要做 SSH Key exchange |
| 10 | {{{ |
| 11 | ~$ ssh-keygen |
| 12 | ~$ cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys |
| 13 | }}} |
| 14 | * 2. 需要 JAVA_HOME 環境變數才能執行 hadoop namenode |
| 15 | {{{ |
| 16 | $ echo "export JAVA_HOME=/usr/lib/jvm/java-6-sun" >> ~/.bash_profile |
| 17 | $ cd ~/hadoop-0.18.2 |
| 18 | }}} |
| 19 | * 3. 編輯 conf/hadoop-evn.sh (HADOOP_HOME要設定到你的hadoop安裝目錄) |
| 20 | {{{ |
| 21 | export JAVA_HOME=/usr/lib/jvm/java-6-sun |
| 22 | export HADOOP_HOME=/home/jazz/hadoop-0.18.2/ |
| 23 | export HADOOP_CONF_DIR=$HADOOP_HOME/conf |
| 24 | }}} |
| 25 | * 4. 編輯 conf/hadoop-site.xml 在 configuration 那一段加入以下設定 |
| 26 | {{{ |
| 27 | <property> |
| 28 | <name>fs.default.name</name> |
| 29 | <value>hdfs://localhost:9000/</value> |
| 30 | <description> |
| 31 | The name of the default file system. Either the literal string |
| 32 | "local" or a host:port for NDFS. |
| 33 | </description> |
| 34 | </property> |
| 35 | <property> |
| 36 | <name>mapred.job.tracker</name> |
| 37 | <value>localhost:9001</value> |
| 38 | <description> |
| 39 | The host and port that the MapReduce job tracker runs at. If |
| 40 | "local", then jobs are run in-process as a single map and |
| 41 | reduce task. |
| 42 | </description> |
| 43 | </property> |
| 44 | |
| 45 | }}} |
| 46 | * 5. 啟動hadoop 的兩道指令 |
| 47 | {{{ |
| 48 | ~/hadoop-0.18.2$ bin/hadoop namenode -format |
| 49 | ~/hadoop-0.18.2$ bin/start-all.sh |
| 50 | }}} |
| 51 | * 6. 完成後可以看到以下三個網頁 |
| 52 | * http://localhost:50030/ |
| 53 | * http://localhost:50060/ |
| 54 | * http://localhost:50070/ |
| 55 | |
| 56 | * 7. 也可以放的東西上hdfs去看看 |
| 57 | {{{ |
| 58 | ~/hadoop-0.18.2$ bin/hadoop dfs -put conf conf |
| 59 | ~/hadoop-0.18.2$ bin/hadoop dfs -ls |
| 60 | Found 1 items |
| 61 | drwxr-xr-x - jazz supergroup 0 2008-11-04 15:56 /user/jazz/conf |
| 62 | }}} |