◢ <實作八> | <回課程大綱> ▲ | <實作十> ◣

實作九 Lab 9

Hadoop Streaming 搭配不同程式語言練習
Hadoop Streaming in different Language

以下練習，請連線至 hadoop.classcloud.org 操作。底下的 hXXXX 等於您的用戶名稱。

• 或者連線至 ​https://hadoop.classcloud.org 改用 web shell 執行

搭配現存二進位執行檔

Existing Binary

~$ hadoop fs -put /etc/hadoop/conf lab9_input
~$ hadoop jar hadoop-streaming.jar -input lab9_input -output lab9_out1 -mapper /bin/cat -reducer /usr/bin/wc
~$ hadoop fs -cat lab9_out1/part-00000

搭配 Bash Shell Script

~$ echo "sed -e \"s/ /\n/g\" | grep ." > streamingMapper.sh
~$ echo "uniq -c | awk '{print \$2 \"\t\" \$1}'" > streamingReducer.sh
~$ chmod a+x streamingMapper.sh
~$ chmod a+x streamingReducer.sh
~$ hadoop jar hadoop-streaming.jar -input lab9_input -output lab9_out2 -mapper streamingMapper.sh -reducer streamingReducer.sh -file streamingMapper.sh -file streamingReducer.sh
~$ hadoop fs -cat lab9_out2/part-00000