◢ <實作二十> | <回課程大綱> ▲ | <實作二十二 > ◣

實作二十一 Lab21

修改檔案輸入格式
KeyValueTextInputFormat

請先連線至 nodeN.3du.me , N 為您的報名編號

cd ~/hadoop_labs/lab012
ant
mkdir -p kv_input
printf "A\t1\n" >  kv_input/input1
printf "B\t2\n" >> kv_input/input1
printf "C\t3\n" >> kv_input/input1
printf "A\t1\n" >  kv_input/input2
printf "C\t2\n" >> kv_input/input2
printf "B\t1\n" >> kv_input/input2
hadoop fs -put kv_input kv_input
hadoop jar WordCount.jar kv_input kv_output
hadoop fs -ls kv_output
hadoop fs -cat kv_output/part-*

export HADOOP_CONF_DIR=~/hadoop/conf.local/
hadoop jar WordCount.jar kv_input kv_output
ls -al kv_output
cat kv_output/part-*
unset HADOOP_CONF_DIR

• Reference:
  • ​http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.html
  • ​http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.html

    public class TextInputFormat
    extends FileInputFormat<LongWritable,Text>
    

  • ​http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/KeyValueTextInputFormat.html

    public class KeyValueTextInputFormat
    extends FileInputFormat<Text,Text>
    

  • ​http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.html

    public class NLineInputFormat
    extends FileInputFormat<LongWritable,Text>
    
    NLineInputFormat which splits N lines of input as one split.
    

實作習題

<問題 1>