實做四: Hadoop 程式編譯

練習 1 : Word Count 初級版

• 上傳內容到hdfs內

$ cd /opt/hadoop
$ mkdir lab4_input
$ echo "I like NCHC Cloud Course." > lab4_input/input1
$ echo "I like nchc Cloud Course, and we enjoy this course." > lab4_input/input2
$ bin/hadoop fs -put lab4_input lab4_input
$ bin/hadoop fs -ls lab4_input

• 下載 WordCount.java​ 並存到/opt/hadoop;

  $ wget http://trac.nchc.org.tw/cloud/attachment/wiki/jazz/Hadoop_Lab6/WordCount.java?format=raw
  $ mv WordCount.java\?format\=raw WordCount.java
  

• 運作程式

$ mkdir MyJava
$ javac -classpath hadoop-*-core.jar -d MyJava WordCount.java
$ jar -cvf wordcount.jar -C MyJava .
$ bin/hadoop jar wordcount.jar WordCount lab4_input/ lab4_out1/
$ bin/hadoop fs -cat lab4_out1/part-00000

---

練習 2 : Word Count 進階版

$ echo "\." >pattern.txt && echo "\," >>pattern.txt
$ bin/hadoop fs -put pattern.txt ./
$ mkdir MyJava2

• 下載 WordCount2.java​ 並存到/opt/hadoop;

  $ wget http://trac.nchc.org.tw/cloud/attachment/wiki/jazz/Hadoop_Lab6/WordCount2.java?format=raw
  $ mv WordCount2.java\?format\=raw WordCount2.java
  

$ javac -classpath hadoop-*-core.jar -d MyJava2 WordCount2.java
$ jar -cvf wordcount2.jar -C MyJava2 .
$ bin/hadoop jar wordcount2.jar WordCount2 lab4_input lab4_out2 -skip pattern.txt
$ bin/hadoop fs -cat lab4_out2/part-00000
$ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false lab4_input lab4_out3 -skip pattern.txt
$ bin/hadoop fs -cat lab4_out3/part-00000