Changes between Version 1 and Version 2 of III110813/Lab10
- Timestamp:
- Oct 21, 2011, 3:26:03 PM (13 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
III110813/Lab10
v1 v2 20 20 ~$ for ((i=60;i<80;i++)); do echo /user/$(whoami)/sample-$i.fa; done > sample-04.txt 21 21 ~$ for ((i=80;i<100;i++)); do echo /user/$(whoami)/sample-$i.fa; done > sample-05.txt 22 ~$ hadoop fs -mkdir lab 9_input23 ~$ hadoop fs -put sample-0* lab 9_input22 ~$ hadoop fs -mkdir lab10_input 23 ~$ hadoop fs -put sample-0* lab10_input 24 24 }}} 25 25 * 檢查輸入檔案 26 26 {{{ 27 27 ~$ hadoop fs -ls 28 ~$ hadoop fs -ls lab 9_input28 ~$ hadoop fs -ls lab10_input 29 29 }}} 30 30 … … 81 81 * 讓我們用 Hadoop Streaming 的方式來執行 testmapper.sh 82 82 {{{ 83 ~$ hadoop jar hadoop-streaming.jar -input lab 9_input -output lab9_out1 -mapper testmapper.sh -file testmapper.sh83 ~$ hadoop jar hadoop-streaming.jar -input lab10_input -output lab10_out1 -mapper testmapper.sh -file testmapper.sh 84 84 }}} 85 * 觀察 lab 9_out1 的結果,看與本機執行有何不同呢?85 * 觀察 lab10_out1 的結果,看與本機執行有何不同呢? 86 86 {{{ 87 ~$ hadoop fs -cat /user/$(whoami)/lab 9_out1/part-00000 | head87 ~$ hadoop fs -cat /user/$(whoami)/lab10_out1/part-00000 | head 88 88 hadoop@hadoop104:/var/lib/hadoop/cache/hadoop/mapred/local/taskTracker/jobcache/job_201106041247_1820/attempt_201106041247_1820_m_000002_0/work> hadoop fs -get /user/h998/sample-60.fa /tmp/h998/sample-60.fa 89 89 hadoop@hadoop104:/var/lib/hadoop/cache/hadoop/mapred/local/taskTracker/jobcache/job_201106041247_1820/attempt_201106041247_1820_m_000002_0/work> hadoop fs -get /user/h998/sample-61.fa /tmp/h998/sample-61.fa … … 148 148 * 接著用 hadoop streaming 來執行 149 149 {{{ 150 ~$ hadoop jar hadoop-streaming.jar -input lab 9_input -output lab9_out2 -mapper mapper.sh -file mapper.sh150 ~$ hadoop jar hadoop-streaming.jar -input lab10_input -output lab10_out2 -mapper mapper.sh -file mapper.sh 151 151 }}}