實作二： HDFS 指令操作練習

前言

• 此部份接續實做一

Content 1. 基本操作

1.1 瀏覽你HDFS目錄

1.2 上傳資料到HDFS目錄

1.3 下載HDFS的資料到本地目錄

1.4 更多指令操作

Content 2. Hadoop 運算命令

2.1 Hadoop運算命令 grep

2.2 Hadoop運算命令 WordCount

2.3 更多運算命令

可執行的指令一覽表：

aggregatewordcount	An Aggregate based map/reduce program that counts the words in the input files.
aggregatewordhist	An Aggregate based map/reduce program that computes the histogram of the words in the input files.
grep	A map/reduce program that counts the matches of a regex in the input.
join	A job that effects a join over sorted, equally partitioned datasets
multifilewc	A job that counts words from several files.
pentomino	A map/reduce tile laying program to find solutions to pentomino problems.
pi	A map/reduce program that estimates Pi using monte-carlo method.
randomtextwriter	A map/reduce program that writes 10GB of random textual data per node.
randomwriter	A map/reduce program that writes 10GB of random data per node.
sleep	A job that sleeps at each map and reduce task.
sort	A map/reduce program that sorts the data written by the random writer.
sudoku	A sudoku solver.
wordcount	A map/reduce program that counts the words in the input files.

請參考 ​org.apache.hadoop.examples

Class Summary
AggregateWordCount	This is an example Aggregated Hadoop Map/Reduce application. It reads the text input files, breaks each line into words and counts them. The output is a locally sorted list of words and the count of how often they occurred. To run: bin/hadoop jar hadoop-*-examples.jar aggregatewordcount in-dir out-dir numOfReducers textinputformat
AggregateWordHistogram	This is an example Aggregated Hadoop Map/Reduce application. Computes the histogram of the words in the input texts. To run: bin/hadoop jar hadoop-*-examples.jar aggregatewordhist in-dir out-dir numOfReducers textinputformat
ExampleDriver	A description of an example program based on its class and a human-readable description.
Grep
Join	This is the trivial map/reduce program that does absolutely nothing other than use the framework to fragment and sort the input values. To run: bin/hadoop jar build/hadoop-examples.jar join [-m maps] [-r reduces] [-inFormat input format class] [-outFormat output format class] [-outKey output key class] [-outValue output value class] [-joinOp
RandomTextWriter	This program uses map/reduce to just run a distributed job where there is no interaction between the tasks and each task writes a large unsorted random sequence of words.To run: bin/hadoop jar hadoop-${version}-examples.jar randomtextwriter [-outFormat output format class] output
RandomWriter	This program uses map/reduce to just run a distributed job where there is no interaction between the tasks and each task write a large unsorted random binary sequence file of BytesWritable.To run: bin/hadoop jar hadoop-${version}-examples.jar randomwriter [-outFormat output format class] output
Sort<K,V>	This is the trivial map/reduce program that does absolutely nothing other than use the framework to fragment and sort the input values.To run: bin/hadoop jar build/hadoop-examples.jar sort [-m maps] [-r reduces] [-inFormat input format class] [-outFormat output format class] [-outKey output key class] [-outValue output value class] [-totalOrder pcnt num samples max splits] in-dir out-dir
WordCount	This is an example Hadoop Map/Reduce application.

 

Content 3. 使用網頁Gui瀏覽訊息

練習