wiki:III140503/Lab8

◢ <實作七> | <回課程大綱> ▲ | <實作九> ◣

實作八 Lab 8

在完全分散模式下編譯 MapReduce 程式
Compiling Hadoop MapReduce Java Program in Hadoop Cluster
以下練習,請連線至 hadoop.3du.me 操作。底下的 userXX 等於您的用戶名稱。

Practice 1 : Word Count (Basic)

  • 上傳內容到 HDFS 內
    upload data to HDFS
    $ mkdir lab8_input
    $ echo "I like NCTU Cloud Course." > lab8_input/input1
    $ echo "I like nctu Cloud Course, and we enjoy this course." > lab8_input/input2
    $ hadoop fs -put lab8_input lab8_input
    $ hadoop fs -ls lab8_input
    Found 2 items
    -rw-r--r--   2 hXXXX supergroup         26 2011-04-19 10:07 /user/hXXXX/lab8_input/input1
    -rw-r--r--   2 hXXXX supergroup         52 2011-04-19 10:07 /user/hXXXX/lab8_input/input2
    
  • 運作程式
    Compile WordCount.java and run it by hadoop jar command
$ mkdir MyJava
$ ln -s /opt/hadoop/hadoop-*-core.jar hadoop-core.jar
$ javac -classpath hadoop-core.jar -d MyJava WordCount.java
$ jar -cvf wordcount.jar -C MyJava .
$ hadoop jar wordcount.jar WordCount lab8_input/ lab8_out1/
$ hadoop fs -cat lab8_out1/part-00000
  • lab8_out1 執行結果
    You should see results like this :
    Cloud 2
    Course, 1
    Course. 1
    I 2
    NCTU  1
    and 1
    course. 1
    enjoy 1
    like  2
    nctu  1
    this  1
    we  1
    

Last modified 8 years ago Last modified on May 4, 2014, 1:16:47 PM