wiki:Hadoop_StandardAlone

Version 2 (modified by waue, 13 years ago) (diff)

--

實作一: Hadoop Standard Alone Mode

範例一:使用 cat 當 mapper,使用 wc 當 reducer

  • 用 Standard Alone Mode 跑 hadoop code ( using streaming example )
    hadoop@lucid:~$ cd /opt/hadoop
    hadoop@lucid:/opt/hadoop$ bin/hadoop jar ./contrib/streaming/hadoop-0.20.2-streaming.jar -input input -output output -mapper /bin/cat -reducer /usr/bin/wc
    
  • 比較:使用 單機安裝模式
    hadoop@lucid:~$ cd /opt/hadoop
    // 設定 core-site.xml, hdfs-site.xml, mapred-site.xml
    hadoop@lucid:/opt/hadoop$ bin/hadoop namenode -format
    hadoop@lucid:/opt/hadoop$ bin/start-all.sh
    hadoop@lucid:/opt/hadoop$ bin/hadoop fs -put conf input
    hadoop@lucid:/opt/hadoop$ bin/hadoop jar ./contrib/streaming/hadoop-0.20.2-streaming.jar -input input -output output -mapper /bin/cat -reducer /usr/bin/wc