Changes between Version 1 and Version 2 of NCTU110329/Lab6


Ignore:
Timestamp:
Apr 19, 2011, 10:53:52 AM (14 years ago)
Author:
jazz
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • NCTU110329/Lab6

    v1 v2  
    1515
    1616{{{
    17 $ cd /opt/hadoop
    18 $ mkdir lab4_input
    19 $ echo "I like NCHC Cloud Course." > lab4_input/input1
    20 $ echo "I like nchc Cloud Course, and we enjoy this course." > lab4_input/input2
    21 $ bin/hadoop fs -put lab4_input lab4_input
    22 $ bin/hadoop fs -ls lab4_input
     17$ mkdir lab6_input
     18$ echo "I like NCTU Cloud Course." > lab6_input/input1
     19$ echo "I like nctu Cloud Course, and we enjoy this course." > lab6_input/input2
     20$ hadoop fs -put lab6_input lab6_input
     21$ hadoop fs -ls lab6_input
     22Found 2 items
     23-rw-r--r--   2 hXXXX supergroup         26 2011-04-19 10:07 /user/hXXXX/lab6_input/input1
     24-rw-r--r--   2 hXXXX supergroup         52 2011-04-19 10:07 /user/hXXXX/lab6_input/input2
    2325}}}
    2426
    25  * 下載 [http://secuse.nchc.org.tw/class/WordCount.java WordCount.java] 並存到/opt/hadoop;[[BR]]Download [http://trac.nchc.org.tw/cloud/raw-attachment/wiki/WordCount/WordCount.java WordCount.java] and save to /opt/hadoop
     27 * 下載 [http://hadoop.nchc.org.tw/WordCount.java WordCount.java] 並存到家目錄;[[BR]]Download [http://hadoop.nchc.org.tw/WordCount.java WordCount.java] and save to your home directory
    2628{{{
    27 ~$ cd /opt/hadoop
    28 /opt/hadoop$ wget http://trac.nchc.org.tw/cloud/raw-attachment/wiki/WordCount/WordCount.java
     29~$ wget http://hadoop.nchc.org.tw/WordCount.java
    2930}}}
    3031
    … …  
    3334{{{
    3435$ mkdir MyJava
    35 $ javac -classpath hadoop-*-core.jar -d MyJava WordCount.java
     36$ ln -s /usr/lib/hadoop/hadoop-*-core.jar hadoop-core.jar
     37$ javac -classpath hadoop-core.jar -d MyJava WordCount.java
    3638$ jar -cvf wordcount.jar -C MyJava .
    37 $ bin/hadoop jar wordcount.jar WordCount lab4_input/ lab4_out1/
    38 $ bin/hadoop fs -cat lab4_out1/part-00000
     39$ hadoop jar wordcount.jar WordCount lab6_input/ lab6_out1/
     40$ hadoop fs -cat lab6_out1/part-00000
    3941}}}
    4042
    41  * lab4_out1 執行結果 [[BR]]You should see results like this :
     43 * lab6_out1 執行結果 [[BR]]You should see results like this :
    4244{{{
    4345#!text
    … …  
    4648Course. 1
    4749I       2
    48 NCHC    1
     50NCTU    1
    4951and     1
    5052course. 1
    5153enjoy   1
    5254like    2
    53 nchc    1
     55nctu    1
    5456this    1
    5557we      1
    … …  
    6163{{{
    6264$ echo "\." >pattern.txt && echo "\," >>pattern.txt
    63 $ bin/hadoop fs -put pattern.txt ./
    64 $ mkdir MyJava2
     65$ hadoop fs -put pattern.txt .
     66$ mkdir -p MyJava2
    6567}}}
    6668
    6769
    68  * 下載 [raw-attachment:wiki:Hadoop_Lab4:WordCount2.java WordCount2.java] 並存到/opt/hadoop;[[BR]]Download [raw-attachment:wiki:Hadoop_Lab4:WordCount2.java WordCount2.java] to /opt/hadoop
     70 * 下載 [http://hadoop.nchc.org.tw/WordCount2.java WordCount2.java] 並存到家目錄;[[BR]]Download [http://hadoop.nchc.org.tw/WordCount2.java WordCount2.java] to home directory
    6971{{{
    70 ~$ cd /opt/hadoop
    71 /opt/hadoop$ wget http://trac.nchc.org.tw/cloud/raw-attachment/wiki/Hadoop_Lab4/WordCount2.java
     72~$ wget http://hadoop.nchc.org.tw/WordCount2.java
    7273}}}
    7374
    7475{{{
    75 $ javac -classpath hadoop-*-core.jar -d MyJava2 WordCount2.java
     76$ javac -classpath hadoop-core.jar -d MyJava2 WordCount2.java
    7677$ jar -cvf wordcount2.jar -C MyJava2 .
    77 $ bin/hadoop jar wordcount2.jar WordCount2 lab4_input lab4_out2 -skip pattern.txt
    78 $ bin/hadoop fs -cat lab4_out2/part-00000
     78$ hadoop jar wordcount2.jar WordCount2 lab6_input lab6_out2 -skip pattern.txt
     79$ hadoop fs -cat lab6_out2/part-00000
    7980}}}
    8081
    81  * lab4_out2 執行結果[[BR]]You should see results like this:
     82 * lab6_out2 執行結果[[BR]]You should see results like this:
    8283{{{
    8384#!text
    … …  
    8586Course  2
    8687I       2
    87 NCHC    1
     88NCTU    1
    8889and     1
    8990course  1
    9091enjoy   1
    9192like    2
    92 nchc    1
     93nctu    1
    9394this    1
    9495we      1
    … …  
    9798 * Let's given case insensitive and ignore pattern for this example
    9899{{{
    99 /opt/hadoop$ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false lab4_input lab4_out3 -skip pattern.txt
    100 /opt/hadoop$ bin/hadoop fs -cat lab4_out3/part-00000
     100$ hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false lab6_input lab6_out3 -skip pattern.txt
     101$ hadoop fs -cat lab6_out3/part-00000
    101102}}}
    102103
    103  * lab4_out3 執行結果[[BR]]You should see results like this:
     104 * lab6_out3 執行結果[[BR]]You should see results like this:
    104105{{{
    105106#!text
    … …  
    110111i       2
    111112like    2
    112 nchc    2
     113nctu    2
    113114this    1
    114115we      1