Changes between Version 22 and Version 23 of jazz/08-11-05

Timestamp:

Nov 5, 2008, 12:14:24 PM (17 years ago)

Author:

jazz

Comment:

--

jazz/08-11-05

@@ -8 +8 @@
 == Hadoop Hands-on Labs (2) ==
 
+ * 執行 Wordcount 範例
+{{{
+~/hadoop-0.18.2$ bin/hadoop fs -put conf conf
+~/hadoop-0.18.2$ bin/hadoop fs -ls
+Found 1 items
+drwxr-xr-x   - jazz supergroup          0 2008-11-05 19:34 /user/jazz/conf
+~/hadoop-0.18.2$ bin/hadoop jar /home/jazz/hadoop-0.18.2/hadoop-0.18.2-examples.jar wordcount
+ERROR: Wrong number of parameters: 0 instead of 2.
+wordcount [-m <maps>] [-r <reduces>] <input> <output>
+Generic options supported are
+-conf <configuration file>     specify an application configuration file
+-D <property=value>            use value for given property
+-fs <local|namenode:port>      specify a namenode
+-jt <local|jobtracker:port>    specify a job tracker
+-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
+-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
+-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.
+
+The general command line syntax is
+bin/hadoop command [genericOptions] [commandOptions]
+~/hadoop-0.18.2$ bin/hadoop jar /home/jazz/hadoop-0.18.2/hadoop-0.18.2-examples.jar wordcount conf output
+}}}
+ * Wordcount 的原始碼
+{{{
+jazz@drbl:~/hadoop-0.18.2/$ vi src/examples/org/apache/hadoop/examples/WordCount.java
+}}}
  * 示範 Wordcount.java 如何除錯: 故意加一段 IOException 讓 mapper 產生錯誤
 {{{
@@ -18 +44 @@
 }}}
  * 詳細說明在官方文件: http://hadoop.apache.org/core/docs/r0.18.2/mapred_tutorial.html
+ * Input and Output Formats
+   * 通常輸入跟輸出都是純文字格式，因此預設是 !TextInputFormat 跟 !TextOutputFormat
+   * 但如果輸入跟輸出是二進位格式，那就必須使用 !SequenceFileInputFormat 跟 !SequenceFileOutputFormat 當作 Map/Reduce 的 !KeyClass
  * [http://www.hadoop.tw/2008/09/php-hadoop.html 用 "單機" 跟 "PHP" 開發 Hadoop 程式]