wiki:jazz/Hadoop_Lab6

實做六

  • restart your Hadoop machine done on yesterday.
    • on node1
      $ cd /opt/hadoop
      $ bin/hadoop start-dfs.sh
      $ ssh node02 "bin/hadoop start-mapred.sh"
      
    • please check hadoop status.

Example 1 :

  • upload input to hdfs
$ cd /opt/hadoop
$ bin/hadoop dfs -mkdir input
$ echo "I like NCHC Cloud Course." > input1
$ echo "I like nchc Cloud Course, and we enjoy this course." > input2
$ bin/hadoop dfs -put input1 input
$ bin/hadoop dfs -put input2 input
$ bin/hadoop dfs -ls input
  • run this code
$ mkdir MyJava
$ javac -classpath hadoop-*-core.jar -d MyJava WordCount.java
$ jar -cvf wordcount.jar -C MyJava .
$ bin/hadoop jar wordcount.jar WordCount input/ output/
$ bin/hadoop dfs -cat output/part-00000

Example 2 :

$ echo "\." >pattern.txt && echo "\," >>pattern.txt
$ bin/hadoop dfs -put pattern.txt ./
$ mkdir MyJava2
$ javac -classpath hadoop-*-core.jar -d MyJava2 WordCount2.java
$ jar -cvf wordcount2.jar -C MyJava2 .
$ bin/hadoop jar wordcount2.jar WordCount2 input output2 -skip pattern.txt
$ bin/hadoop dfs -cat output2/part-00000
$ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false input output3 -skip pattern.txt
$ bin/hadoop dfs -cat output3/part-00000
Last modified 15 years ago Last modified on Sep 14, 2009, 1:22:05 AM

Attachments (2)

Download all attachments as: .zip