Changes between Version 43 and Version 44 of waue/2009/0617


Ignore:
Timestamp:
Sep 24, 2010, 5:14:17 PM (14 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • waue/2009/0617

    v43 v44  
    209209一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值
    210210
    211 [[Image(wiki:waue:2009:0617:2-1.png)]]
     211[[Image(wiki:waue/2009/0617:2-1.png)]]
    212212
    213213-------
     
    221221|| window -> || open pers.. -> || other.. -> || map/reduce||
    222222
    223 [[Image(win-open-other.png)]]
     223[[Image(wiki:waue/2009/0617:win-open-other.png)]]
    224224
    225225-------
    226226
    227227設定要用 Map/Reduce 的視野
    228 [[Image(wiki:waue:2009:0617:2-2.png)]]
     228[[Image(wiki:waue/2009/0617:2-2.png)]]
    229229
    230230---------
    231231
    232232使用 Map/Reduce 的視野後的介面呈現
    233 [[Image(wiki:waue:2009:0617:2-3.png)]]
     233[[Image(wiki:waue/2009/0617:2-3.png)]]
    234234
    235235--------
     
    238238
    239239 || file ->  || new ->  || project ->  || Map/Reduce ->  || Map/Reduce Project -> ||  next ||
    240 [[Image(wiki:waue:2009:0617:file-new-project.png)]]
     240[[Image(wiki:waue/2009/0617:file-new-project.png)]]
    241241
    242242--------
     
    244244建立mapreduce專案(1)
    245245
    246 [[Image(wiki:waue:2009:0617:2-4.png)]]
     246[[Image(wiki:waue/2009/0617:2-4.png)]]
    247247
    248248-----------
     
    256256}}}
    257257
    258 [[Image(wiki:waue:2009:0617:2-4-2.png)]]
     258[[Image(wiki:waue/2009/0617:2-4-2.png)]]
    259259
    260260
     
    269269 Step1. 右鍵點選project的properties做細部設定
    270270
    271 [[Image(wiki:waue:2009:0617:2-5.png)]]
     271[[Image(wiki:waue/2009/0617:2-5.png)]]
    272272
    273273----------
     
    276276
    277277hadoop的javadoc的設定(1)
    278 [[Image(wiki:waue:2009:0617:2-5-1.png)]]
     278[[Image(wiki:waue/2009/0617:2-5-1.png)]]
    279279
    280280 * java Build Path -> Libraries -> hadoop-0.20.0-ant.jar
     
    291291------------
    292292 Step3. hadoop的javadoc的設定完後(2)
    293 [[Image(wiki:waue:2009:0617:2-5-2.png)]]
     293[[Image(wiki:waue/2009/0617:2-5-2.png)]]
    294294
    295295------------
     
    298298 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/
    299299 
    300 [[Image(wiki:waue:2009:0617:2-5-3.png)]]
     300[[Image(wiki:waue/2009/0617:2-5-3.png)]]
    301301
    302302-----
     
    308308--------
    309309 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示:
    310 [[Image(wiki:waue:2009:0617:2-6.png)]]
     310[[Image(wiki:waue/2009/0617:2-6.png)]]
    311311
    312312-------------
    313313 Step2. 進行eclipse 與 hadoop 間的設定(2)
    314 [[Image(wiki:waue:2009:0617:2-6-1.png)]]
     314[[Image(wiki:waue/2009/0617:2-6-1.png)]]
    315315
    316316{{{
     
    325325
    326326設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構
    327 [[Image(wiki:waue:2009:0617:2-6-2.png)]]
     327[[Image(wiki:waue/2009/0617:2-6-2.png)]]
    328328-------------
    329329
     
    343343 
    344344 || File ->  || new ->  || mapper ||
    345 [[Image(wiki:waue:2009:0617:file-new-mapper.png)]]
     345[[Image(wiki:waue/2009/0617:file-new-mapper.png)]]
    346346
    347347-----------
     
    349349 2. create
    350350 
    351 [[Image(wiki:waue:2009:0617:3-1.png)]]
     351[[Image(wiki:waue/2009/0617:3-1.png)]]
    352352{{{
    353353#!sh
     
    388388
    389389建立mapper.java後,貼入程式碼
    390 [[Image(wiki:waue:2009:0617:3-2.png)]]
     390[[Image(wiki:waue/2009/0617:3-2.png)]]
    391391
    392392------------
     
    397397
    398398 * File -> new -> reducer
    399 [[Image(wiki:waue:2009:0617:file-new-reducer.png)]]
     399[[Image(wiki:waue/2009/0617:file-new-reducer.png)]]
    400400
    401401-------
    402402 2. create
    403 [[Image(wiki:waue:2009:0617:3-3.png)]]
     403[[Image(wiki:waue/2009/0617:3-3.png)]]
    404404
    405405{{{
     
    440440
    441441 * File -> new -> Map/Reduce Driver
    442 [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]]
     442[[Image(wiki:waue/2009/0617:file-new-mr-driver.png)]]
    443443----------
    444444
     
    448448
    449449建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver
    450 [[Image(wiki:waue:2009:0617:3-4.png)]]
     450[[Image(wiki:waue/2009/0617:3-4.png)]]
    451451------------
    452452
     
    502502
    503503三個檔完成後並存檔後,整個程式建立完成
    504 [[Image(wiki:waue:2009:0617:3-5.png)]]
     504[[Image(wiki:waue/2009/0617:3-5.png)]]
    505505
    506506-------
     
    521521   * 右鍵點選WordCount.java -> run as -> run on Hadoop :沒有效果
    522522   
    523 [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]]
     523[[Image(wiki:waue/2009/0617:run-on-hadoop.png)]]
    524524
    525525 * 因此,4.1 提供一個eclipse 上解除 run-on-hadoop 封印的方法。而4.2 則是避開run-on-hadoop 這個功能,用command mode端指令的方法執行。
     
    619619 * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔
    620620 
    621 [[Image(wiki:waue:2009:0617:4-1.png)]]
     621[[Image(wiki:waue/2009/0617:4-1.png)]]
    622622------
    623623 * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助
    624 [[Image(wiki:waue:2009:0617:4-2.png)]]
     624[[Image(wiki:waue/2009/0617:4-2.png)]]
    625625
    626626 === make output ===