Changes between Version 27 and Version 28 of jazz/08-11-05


Ignore:
Timestamp:
Nov 5, 2008, 2:34:41 PM (16 years ago)
Author:
jazz
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • jazz/08-11-05

    v27 v28  
    5151 * hadoop-stream:
    5252   * 目前處理 binary 的能力仍有限，因此建議使用在純文字處理上。
    53    *
    54 {{{
     53   * 如果保留原始 hadoop-site.xml 的 configure 描述(沒有加任何 <property>)，預設是使用 local filesystem
     54{{{
     55~/hadoop-0.18.2$ cat conf/hadoop-site.xml
     56<?xml version="1.0"?>
     57<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
     58
     59<!-- Put site-specific property overrides in this file. -->
     60
     61<configuration>
     62
     63</configuration>
     64
    5565~/hadoop-0.18.2$ echo "sed -e \"s/ /\n/g\" | grep ." > streamingMapper.sh
    5666~/hadoop-0.18.2$ echo "uniq -c | awk '{print $2 \"\t\" $1}'" > streamingReducer.sh
    57 ~/hadoop-0.18.2$ bin/hadoop jar ./contrib/streaming/hadoop-0.18.2-streaming.jar -input conf -output out1 -mapper streamingMapper.sh -reducer streamingReducer.sh -file ./streamingMapper.sh -file ./streamingReducer.sh
     67~/hadoop-0.18.2$ bin/hadoop jar ./contrib/streaming/hadoop-0.18.2-streaming.jar -input conf -output out1 -mapper `pwd`/streamingMapper.sh -reducer `pwd`/streamingReducer.sh
    5868}}}
    5969 * [http://www.hadoop.tw/2008/09/php-hadoop.html 用 "單機" 跟 "PHP" 開發 Hadoop 程式]