Changes between Version 22 and Version 23 of jazz/08-11-05


Ignore:
Timestamp:
Nov 5, 2008, 12:14:24 PM (16 years ago)
Author:
jazz
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • jazz/08-11-05

    v22 v23  
    88== Hadoop Hands-on Labs (2) ==
    99
     10 * 執行 Wordcount 範例
     11{{{
     12~/hadoop-0.18.2$ bin/hadoop fs -put conf conf
     13~/hadoop-0.18.2$ bin/hadoop fs -ls
     14Found 1 items
     15drwxr-xr-x   - jazz supergroup          0 2008-11-05 19:34 /user/jazz/conf
     16~/hadoop-0.18.2$ bin/hadoop jar /home/jazz/hadoop-0.18.2/hadoop-0.18.2-examples.jar wordcount
     17ERROR: Wrong number of parameters: 0 instead of 2.
     18wordcount [-m <maps>] [-r <reduces>] <input> <output>
     19Generic options supported are
     20-conf <configuration file>     specify an application configuration file
     21-D <property=value>            use value for given property
     22-fs <local|namenode:port>      specify a namenode
     23-jt <local|jobtracker:port>    specify a job tracker
     24-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
     25-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
     26-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.
     27
     28The general command line syntax is
     29bin/hadoop command [genericOptions] [commandOptions]
     30~/hadoop-0.18.2$ bin/hadoop jar /home/jazz/hadoop-0.18.2/hadoop-0.18.2-examples.jar wordcount conf output
     31}}}
     32 * Wordcount 的原始碼
     33{{{
     34jazz@drbl:~/hadoop-0.18.2/$ vi src/examples/org/apache/hadoop/examples/WordCount.java
     35}}}
    1036 * 示範 Wordcount.java 如何除錯: 故意加一段 IOException 讓 mapper 產生錯誤
    1137{{{
     
    1844}}}
    1945 * 詳細說明在官方文件: http://hadoop.apache.org/core/docs/r0.18.2/mapred_tutorial.html
     46 * Input and Output Formats
     47   * 通常輸入跟輸出都是純文字格式,因此預設是 !TextInputFormat 跟 !TextOutputFormat
     48   * 但如果輸入跟輸出是二進位格式,那就必須使用 !SequenceFileInputFormat 跟 !SequenceFileOutputFormat 當作 Map/Reduce 的 !KeyClass
    2049 * [http://www.hadoop.tw/2008/09/php-hadoop.html 用 "單機" 跟 "PHP" 開發 Hadoop 程式]
    2150