Changes between Version 1 and Version 2 of III110813/Lab10


Ignore:
Timestamp:
Oct 21, 2011, 3:26:03 PM (13 years ago)
Author:
jazz
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • III110813/Lab10

    v1 v2  
    2020~$ for ((i=60;i<80;i++)); do echo /user/$(whoami)/sample-$i.fa; done > sample-04.txt
    2121~$ for ((i=80;i<100;i++)); do echo /user/$(whoami)/sample-$i.fa; done > sample-05.txt
    22 ~$ hadoop fs -mkdir lab9_input
    23 ~$ hadoop fs -put sample-0* lab9_input
     22~$ hadoop fs -mkdir lab10_input
     23~$ hadoop fs -put sample-0* lab10_input
    2424}}}
    2525 * 檢查輸入檔案
    2626{{{
    2727~$ hadoop fs -ls
    28 ~$ hadoop fs -ls lab9_input
     28~$ hadoop fs -ls lab10_input
    2929}}}
    3030
     
    8181 * 讓我們用 Hadoop Streaming 的方式來執行 testmapper.sh
    8282{{{
    83 ~$ hadoop jar hadoop-streaming.jar -input lab9_input -output lab9_out1 -mapper testmapper.sh -file testmapper.sh
     83~$ hadoop jar hadoop-streaming.jar -input lab10_input -output lab10_out1 -mapper testmapper.sh -file testmapper.sh
    8484}}}
    85  * 觀察 lab9_out1 的結果,看與本機執行有何不同呢?
     85 * 觀察 lab10_out1 的結果,看與本機執行有何不同呢?
    8686{{{
    87 ~$ hadoop fs -cat /user/$(whoami)/lab9_out1/part-00000 | head
     87~$ hadoop fs -cat /user/$(whoami)/lab10_out1/part-00000 | head
    8888hadoop@hadoop104:/var/lib/hadoop/cache/hadoop/mapred/local/taskTracker/jobcache/job_201106041247_1820/attempt_201106041247_1820_m_000002_0/work> hadoop fs -get /user/h998/sample-60.fa /tmp/h998/sample-60.fa 
    8989hadoop@hadoop104:/var/lib/hadoop/cache/hadoop/mapred/local/taskTracker/jobcache/job_201106041247_1820/attempt_201106041247_1820_m_000002_0/work> hadoop fs -get /user/h998/sample-61.fa /tmp/h998/sample-61.fa 
     
    148148 * 接著用 hadoop streaming 來執行
    149149{{{
    150 ~$ hadoop jar hadoop-streaming.jar -input lab9_input -output lab9_out2 -mapper mapper.sh -file mapper.sh
     150~$ hadoop jar hadoop-streaming.jar -input lab10_input -output lab10_out2 -mapper mapper.sh -file mapper.sh
    151151}}}