實作一: Hadoop Standard Alone Mode
範例一:使用 cat 當 mapper,使用 wc 當 reducer
- 用 Standard Alone Mode 跑 hadoop code ( using streaming example )
- core-site.xml, hdfs-site.xml, mapred-site.xml 皆空白即可
hadoop@lucid:~$ cd /opt/hadoop hadoop@lucid:/opt/hadoop$ bin/hadoop jar ./contrib/streaming/hadoop-0.20.2-streaming.jar -input input -output output -mapper /bin/cat -reducer /usr/bin/wc
- 比較:使用 單機安裝模式
hadoop@lucid:~$ cd /opt/hadoop // 設定 core-site.xml, hdfs-site.xml, mapred-site.xml hadoop@lucid:/opt/hadoop$ bin/hadoop namenode -format hadoop@lucid:/opt/hadoop$ bin/start-all.sh hadoop@lucid:/opt/hadoop$ bin/hadoop fs -put conf input hadoop@lucid:/opt/hadoop$ bin/hadoop jar ./contrib/streaming/hadoop-0.20.2-streaming.jar -input input -output output -mapper /bin/cat -reducer /usr/bin/wc
Last modified 13 years ago
Last modified on Jul 18, 2011, 12:26:59 PM