= 2010-08-04 =

== Hadoop : Steaming ==

 * [範例] [http://www.mail-archive.com/common-user@hadoop.apache.org/msg00422.html 使用 gzip 當作輸入格式]
   * 目前只支援三種壓縮格式，詳[http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/io/compress/CompressionCodec.html org.apache.hadoop.io.compress.CompressionCodec]
{{{
#!sh
hadoop dfs -rmr $4
hadoop jar /usr/local/share/hadoop/contrib/streaming/hadoop-*-streaming.jar
-mapper $1 -reducer $2 -input $3/* -output
 $4 -file $1 -file $2 -jobconf mapred.job.name="$5"   -jobconf
stream.recordreader.compression=gzip \
-jobconf mapred.output.compress=true \
-jobconf
mapred.output.compression.codec=org.apache.hadoop.io.compress.GzipCodec
}}}