{{{
#!html
<div style="text-align: center; color:#151B8D"><big style="font-weight: bold;"><big><big>
ITRI HBase 進階課程 
</big></big></big></div> <div style="text-align: center; color:#7E2217"><big style="font-weight: bold;"><big>
1. Streaming 
</big></big></div>
}}}

[wiki:waue/2011/0426 回課程大綱 << ] Stream 範例


== Hadoop Streaming with commands ==

 * 範例一：使用 cat 當 mapper，使用 wc 當 reducer
{{{
hadoop@lucid:~$ cd /opt/hadoop
hadoop@lucid:/opt/hadoop$ bin/start-all.sh
hadoop@lucid:/opt/hadoop$ bin/hadoop fs -put conf input
hadoop@lucid:/opt/hadoop$ bin/hadoop jar ./contrib/streaming/hadoop-0.20.2-streaming.jar -input input -output output -mapper /bin/cat -reducer /usr/bin/wc
}}}
 * 範例二：使用 Bash Shell Script 當 Mapper 與 Reducer
{{{
hadoop@lucid:/opt/hadoop$ echo "sed -e \"s/ /\n/g\" | grep ." > streamingMapper.sh
hadoop@lucid:/opt/hadoop$ echo "uniq -c | awk '{print \$2 \"\t\" \$1}'" > streamingReducer.sh
hadoop@lucid:/opt/hadoop$ chmod a+x streamingMapper.sh
hadoop@lucid:/opt/hadoop$ chmod a+x streamingReducer.sh
hadoop@lucid:/opt/hadoop$ bin/hadoop fs -rmr input output
hadoop@lucid:/opt/hadoop$ bin/hadoop fs -put conf input
hadoop@lucid:/opt/hadoop$ bin/hadoop jar ./contrib/streaming/hadoop-0.20.2-streaming.jar -input input -output output -mapper streamingMapper.sh -reducer streamingReducer.sh -file streamingMapper.sh -file streamingReducer.sh
}}}