Changes between Version 3 and Version 4 of jazz/Hadoop_Lab2


Ignore:
Timestamp:
Mar 24, 2009, 11:15:52 AM (15 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • jazz/Hadoop_Lab2

    v3 v4  
    1414 === 1.3 下載HDFS的資料到本地目錄 ===
    1515 
    16  === 1.4 更多指令操作
     16 === 1.4 更多指令操作 ===
    1717 
    1818 == Content 2. Hadoop 運算命令 ==
     
    2525 
    2626 請參考 [http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/package-summary.html org.apache.hadoop.examples]
     27 
     28 
    2729 
    2830{{{
     
    3739</tr>
    3840<tr class="TableRowColor" bgcolor="white">
    39 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/AggregateWordCount.html" title="class in org.apache.hadoop.examples">AggregateWordCount</a></b></td>
     41<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/AggregateWordCount.html" title="class in org.apache.hadoop.examples">AggregateWordCount</a></b></td>
    4042
    4143<td>This is an example Aggregated Hadoop Map/Reduce application.</td>
     
    4345
    4446<tr class="TableRowColor" bgcolor="white">
    45 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/AggregateWordHistogram.html" title="class in org.apache.hadoop.examples">AggregateWordHistogram</a></b></td>
     47<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/AggregateWordHistogram.html" title="class in org.apache.hadoop.examples">AggregateWordHistogram</a></b></td>
    4648<td>This is an example Aggregated Hadoop Map/Reduce application.</td>
    4749
    4850</tr>
    4951<tr class="TableRowColor" bgcolor="white">
    50 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/DBCountPageView.html" title="class in org.apache.hadoop.examples">DBCountPageView</a></b></td>
     52<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/DBCountPageView.html" title="class in org.apache.hadoop.examples">DBCountPageView</a></b></td>
    5153<td>This is a demonstrative program, which uses DBInputFormat for reading
    5254 the input data from a database, and DBOutputFormat for writing the data
     
    5456</tr>
    5557<tr class="TableRowColor" bgcolor="white">
    56 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/ExampleDriver.html" title="class in org.apache.hadoop.examples">ExampleDriver</a></b></td>
     58<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/ExampleDriver.html" title="class in org.apache.hadoop.examples">ExampleDriver</a></b></td>
    5759<td>A description of an example program based on its class and a
    5860 human-readable description.</td>
    5961</tr>
    6062<tr class="TableRowColor" bgcolor="white">
    61 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/Grep.html" title="class in org.apache.hadoop.examples">Grep</a></b></td>
     63<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/Grep.html" title="class in org.apache.hadoop.examples">Grep</a></b></td>
    6264
    6365<td>&nbsp;</td>
    6466</tr>
    6567<tr class="TableRowColor" bgcolor="white">
    66 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/Join.html" title="class in org.apache.hadoop.examples">Join</a></b></td>
     68<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/Join.html" title="class in org.apache.hadoop.examples">Join</a></b></td>
    6769<td>This is the trivial map/reduce program that does absolutely nothing
    6870 other than use the framework to fragment and sort the input values.</td>
    6971</tr>
    7072<tr class="TableRowColor" bgcolor="white">
    71 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/MultiFileWordCount.html" title="class in org.apache.hadoop.examples">MultiFileWordCount</a></b></td>
     73<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/MultiFileWordCount.html" title="class in org.apache.hadoop.examples">MultiFileWordCount</a></b></td>
    7274<td>MultiFileWordCount is an example to demonstrate the usage of
    7375 MultiFileInputFormat.</td>
    7476</tr>
    7577<tr class="TableRowColor" bgcolor="white">
    76 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/MultiFileWordCount.MapClass.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.MapClass</a></b></td>
     78<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/MultiFileWordCount.MapClass.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.MapClass</a></b></td>
    7779
    78 <td>This Mapper is similar to the one in <a href="../../../../org/apache/hadoop/examples/WordCount.MapClass.html" title="class in org.apache.hadoop.examples"><code>WordCount.MapClass</code></a>.</td>
     80<td>This Mapper is similar to the one in <a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/WordCount.MapClass.html" title="class in org.apache.hadoop.examples"><code>WordCount.MapClass</code></a>.</td>
    7981</tr>
    8082<tr class="TableRowColor" bgcolor="white">
    81 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/MultiFileWordCount.MultiFileLineRecordReader.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.MultiFileLineRecordReader</a></b></td>
     83<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/MultiFileWordCount.MultiFileLineRecordReader.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.MultiFileLineRecordReader</a></b></td>
    8284<td>RecordReader is responsible from extracting records from the InputSplit.</td>
    8385</tr>
    8486<tr class="TableRowColor" bgcolor="white">
    85 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/MultiFileWordCount.MyInputFormat.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.MyInputFormat</a></b></td>
    86 <td>To use <a href="../../../../org/apache/hadoop/mapred/MultiFileInputFormat.html" title="class in org.apache.hadoop.mapred"><code>MultiFileInputFormat</code></a>, one should extend it, to return a
    87  (custom) <a href="../../../../org/apache/hadoop/mapred/RecordReader.html" title="interface in org.apache.hadoop.mapred"><code>RecordReader</code></a>.</td>
     87<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/MultiFileWordCount.MyInputFormat.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.MyInputFormat</a></b></td>
     88<td>To use <a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/MultiFileInputFormat.html" title="class in org.apache.hadoop.mapred"><code>MultiFileInputFormat</code></a>, one should extend it, to return a
     89 (custom) <a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/mapred/RecordReader.html" title="interface in org.apache.hadoop.mapred"><code>RecordReader</code></a>.</td>
    8890
    8991</tr>
    9092<tr class="TableRowColor" bgcolor="white">
    91 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/MultiFileWordCount.WordOffset.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.WordOffset</a></b></td>
     93<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/MultiFileWordCount.WordOffset.html" title="class in org.apache.hadoop.examples">MultiFileWordCount.WordOffset</a></b></td>
    9294<td>This record keeps &lt;filename,offset&gt; pairs.</td>
    9395</tr>
    9496<tr class="TableRowColor" bgcolor="white">
    95 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/PiEstimator.html" title="class in org.apache.hadoop.examples">PiEstimator</a></b></td>
     97<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/PiEstimator.html" title="class in org.apache.hadoop.examples">PiEstimator</a></b></td>
    9698<td>A Map-reduce program to estimaate the valu eof Pi using monte-carlo
    9799 method.</td>
     
    99101<tr class="TableRowColor" bgcolor="white">
    100102
    101 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/PiEstimator.PiMapper.html" title="class in org.apache.hadoop.examples">PiEstimator.PiMapper</a></b></td>
     103<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/PiEstimator.PiMapper.html" title="class in org.apache.hadoop.examples">PiEstimator.PiMapper</a></b></td>
    102104<td>Mappper class for Pi estimation.</td>
    103105</tr>SleepJob.SleepInputFormat
    104106<tr class="TableRowColor" bgcolor="white">
    105 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/RandomTextWriter.html" title="class in org.apache.hadoop.examples">RandomTextWriter</a></b></td>
     107<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/RandomTextWriter.html" title="class in org.apache.hadoop.examples">RandomTextWriter</a></b></td>
    106108<td>This program uses map/reduce to just run a distributed job where there is
    107109 no interaction between the tasks and each task writes a large unsorted
     
    110112<tr class="TableRowColor" bgcolor="white">
    111113
    112 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/RandomWriter.html" title="class in org.apache.hadoop.examples">RandomWriter</a></b></td>
     114<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/RandomWriter.html" title="class in org.apache.hadoop.examples">RandomWriter</a></b></td>
    113115<td>This program uses map/reduce to just run a distributed job where there is
    114116 no interaction between the tasks and each task write a large unsorted
     
    116118</tr>
    117119<tr class="TableRowColor" bgcolor="white">
    118 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/SleepJob.html" title="class in org.apache.hadoop.examples">SleepJob</a></b></td>
     120<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/SleepJob.html" title="class in org.apache.hadoop.examples">SleepJob</a></b></td>
    119121<td>Dummy class for testing MR framefork.</td>
    120122</tr>
    121123
    122124<tr class="TableRowColor" bgcolor="white">
    123 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/Sort.html" title="class in org.apache.hadoop.examples">Sort&lt;K,V&gt;</a></b></td>
     125<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/Sort.html" title="class in org.apache.hadoop.examples">Sort&lt;K,V&gt;</a></b></td>
    124126<td>This is the trivial map/reduce program that does absolutely nothing
    125127 other than use the framework to fragment and sort the input values.</td>
    126128</tr>
    127129<tr class="TableRowColor" bgcolor="white">
    128 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/WordCount.html" title="class in org.apache.hadoop.examples">WordCount</a></b></td>
     130<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/WordCount.html" title="class in org.apache.hadoop.examples">WordCount</a></b></td>
    129131<td>This is an example Hadoop Map/Reduce application.</td>
    130132</tr>
    131133
    132134<tr class="TableRowColor" bgcolor="white">
    133 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/WordCount.MapClass.html" title="class in org.apache.hadoop.examples">WordCount.MapClass</a></b></td>
     135<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/WordCount.MapClass.html" title="class in org.apache.hadoop.examples">WordCount.MapClass</a></b></td>
    134136<td>Counts the words in each line.</td>
    135137</tr>
    136138<tr class="TableRowColor" bgcolor="white">
    137 <td width="15%"><b><a href="../../../../org/apache/hadoop/examples/WordCount.Reduce.html" title="class in org.apache.hadoop.examples">WordCount.Reduce</a></b></td>
     139<td width="15%"><b><a href="http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/WordCount.Reduce.html" title="class in org.apache.hadoop.examples">WordCount.Reduce</a></b></td>
    138140<td>A reducer class that just emits the sum of the input values.</td>
    139141</tr>