Changes between Version 18 and Version 19 of Hadoop_Lab5


Ignore:
Timestamp:
Apr 26, 2010, 6:28:55 PM (14 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Hadoop_Lab5

    v18 v19  
    3737 1 將javadoc (jdk-6u10-docs.zip) 下載下來
    3838 [https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jdk-6u10-docs-oth-JPR@CDS-CDS_Developer 下載點]
    39 [[Image(wiki:waue:2009:0617:1-1.png)]]
    40 [[Image(wiki:0428Hadoop_Lab1:hadoop_administration.png)]]
     39[[Image(wiki:Hadoop_Lab5:1-1.png)]]
     40
    4141 
    4242 2 下載完後將檔案放在 /tmp/ 下
     
    196196一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值
    197197
    198 [[Image(wiki:waue:2009:0617:2-1.png)]]
     198[[Image(wiki:Hadoop_Lab5:2-1.png)]]
    199199
    200200-------
     
    208208|| window -> || open pers.. -> || other.. -> || map/reduce||
    209209
    210 [[Image(wiki:waue:2009:0617:win-open-other.png)]]
     210[[Image(wiki:Hadoop_Lab5:win-open-other.png)]]
    211211
    212212-------
    213213
    214214設定要用 Map/Reduce 的視野
    215 [[Image(wiki:waue:2009:0617:2-2.png)]]
     215[[Image(wiki:Hadoop_Lab5:2-2.png)]]
    216216
    217217---------
    218218
    219219使用 Map/Reduce 的視野後的介面呈現
    220 [[Image(wiki:waue:2009:0617:2-3.png)]]
     220[[Image(wiki:Hadoop_Lab5:2-3.png)]]
    221221
    222222--------
     
    225225
    226226 || file ->  || new ->  || project ->  || Map/Reduce ->  || Map/Reduce Project -> ||  next ||
    227 [[Image(wiki:waue:2009:0617:file-new-project.png)]]
     227[[Image(wiki:Hadoop_Lab5:file-new-project.png)]]
    228228
    229229--------
     
    231231建立mapreduce專案(1)
    232232
    233 [[Image(wiki:waue:2009:0617:2-4.png)]]
     233[[Image(wiki:Hadoop_Lab5:2-4.png)]]
    234234
    235235-----------
     
    243243}}}
    244244
    245 [[Image(wiki:waue:2009:0617:2-4-2.png)]]
     245[[Image(wiki:Hadoop_Lab5:2-4-2.png)]]
    246246
    247247
     
    256256 Step1. 右鍵點選project的properties做細部設定
    257257
    258 [[Image(wiki:waue:2009:0617:2-5.png)]]
     258[[Image(wiki:Hadoop_Lab5:2-5.png)]]
    259259
    260260----------
     
    263263
    264264hadoop的javadoc的設定(1)
    265 [[Image(wiki:waue:2009:0617:Image(2-5-1.png)]]
     265[[Image(wiki:Hadoop_Lab5:Image(2-5-1.png)]]
    266266
    267267 * java Build Path -> Libraries -> hadoop-0.20.2-ant.jar
     
    278278------------
    279279 Step3. hadoop的javadoc的設定完後(2)
    280 [[Image(wiki:waue:2009:0617:Image(2-5-2.png)]]
     280[[Image(wiki:Hadoop_Lab5:Image(2-5-2.png)]]
    281281
    282282------------
     
    285285 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/
    286286 
    287 [[Image(wiki:waue:2009:0617:2-5-3.png)]]
     287[[Image(wiki:Hadoop_Lab5:2-5-3.png)]]
    288288
    289289-----
     
    295295--------
    296296 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示:
    297 [[Image(wiki:waue:2009:0617:2-6.png)]]
     297[[Image(wiki:Hadoop_Lab5:2-6.png)]]
    298298
    299299-------------
    300300 Step2. 進行eclipse 與 hadoop 間的設定(2)
    301 [[Image(wiki:waue:2009:0617:2-6-1.png)]]
     301[[Image(wiki:Hadoop_Lab5:2-6-1.png)]]
    302302
    303303{{{
     
    312312
    313313設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構
    314 [[Image(wiki:waue:2009:0617:2-6-2.png)]]
     314[[Image(wiki:Hadoop_Lab5:2-6-2.png)]]
    315315-------------
    316316
     
    330330 
    331331 || File ->  || new ->  || mapper ||
    332 [[Image(wiki:waue:2009:0617:file-new-mapper.png)]]
     332[[Image(wiki:Hadoop_Lab5:file-new-mapper.png)]]
    333333
    334334-----------
     
    336336 2. create
    337337 
    338 [[Image(wiki:waue:2009:0617:3-1.png)]]
     338[[Image(wiki:Hadoop_Lab5:3-1.png)]]
    339339{{{
    340340#!sh
     
    375375
    376376建立mapper.java後,貼入程式碼
    377 [[Image(wiki:waue:2009:0617:3-2.png)]]
     377[[Image(wiki:Hadoop_Lab5:3-2.png)]]
    378378
    379379------------
     
    384384
    385385 * File -> new -> reducer
    386 [[Image(wiki:waue:2009:0617:file-new-reducer.png)]]
     386[[Image(wiki:Hadoop_Lab5:file-new-reducer.png)]]
    387387
    388388-------
    389389 2. create
    390 [[Image(wiki:waue:2009:0617:3-3.png)]]
     390[[Image(wiki:Hadoop_Lab5:3-3.png)]]
    391391
    392392{{{
     
    427427
    428428 * File -> new -> Map/Reduce Driver
    429 [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]]
     429[[Image(wiki:Hadoop_Lab5:file-new-mr-driver.png)]]
    430430----------
    431431
     
    435435
    436436建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver
    437 [[Image(wiki:waue:2009:0617:3-4.png)]]
     437[[Image(wiki:Hadoop_Lab5:3-4.png)]]
    438438------------
    439439
     
    489489
    490490三個檔完成後並存檔後,整個程式建立完成
    491 [[Image(wiki:waue:2009:0617:3-5.png)]]
     491[[Image(wiki:Hadoop_Lab5:3-5.png)]]
    492492
    493493-------
     
    507507 * 右鍵點選WordCount.java -> run as -> run on Hadoop
    508508   
    509 [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]]
    510 
    511 
    512509
    513510= 五、結論 =