Changes between Version 38 and Version 39 of waue/2009/0617


Ignore:
Timestamp:
Apr 26, 2010, 4:31:08 PM (14 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • waue/2009/0617

    v38 v39  
    207207
    208208一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值
    209 [[Image(2-1.png)]]
     209
     210[[Image(wiki:waue:2009:0617:2-1.png)]]
     211
    210212-------
    211213
     
    218220|| window -> || open pers.. -> || other.. -> || map/reduce||
    219221
    220 [[Image(win-open-other.png)]]
     222[[Image(wiki:waue:2009:0617:win-open-other.png)]]
    221223
    222224-------
    223225
    224226設定要用 Map/Reduce 的視野
    225 [[Image(2-2.png)]]
     227[[Image(wiki:waue:2009:0617:2-2.png)]]
    226228
    227229---------
    228230
    229231使用 Map/Reduce 的視野後的介面呈現
    230 [[Image(2-3.png)]]
     232[[Image(wiki:waue:2009:0617:2-3.png)]]
    231233
    232234--------
     
    235237
    236238 || file ->  || new ->  || project ->  || Map/Reduce ->  || Map/Reduce Project -> ||  next ||
    237 [[Image(file-new-project.png)]]
     239[[Image(wiki:waue:2009:0617:file-new-project.png)]]
    238240
    239241--------
     
    241243建立mapreduce專案(1)
    242244
    243 [[Image(2-4.png)]]
     245[[Image(wiki:waue:2009:0617:2-4.png)]]
    244246
    245247-----------
     
    253255}}}
    254256
    255 [[Image(2-4-2.png)]]
     257[[Image(wiki:waue:2009:0617:2-4-2.png)]]
    256258
    257259
     
    266268 Step1. 右鍵點選project的properties做細部設定
    267269
    268 [[Image(2-5.png)]]
     270[[Image(wiki:waue:2009:0617:2-5.png)]]
    269271
    270272----------
     
    273275
    274276hadoop的javadoc的設定(1)
    275 [[Image(2-5-1.png)]]
     277[[Image(wiki:waue:2009:0617:Image(2-5-1.png)]]
    276278
    277279 * java Build Path -> Libraries -> hadoop-0.20.0-ant.jar
     
    288290------------
    289291 Step3. hadoop的javadoc的設定完後(2)
    290 [[Image(2-5-2.png)]]
     292[[Image(wiki:waue:2009:0617:Image(2-5-2.png)]]
    291293
    292294------------
     
    295297 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/
    296298 
    297 [[Image(2-5-3.png)]]
     299[[Image(wiki:waue:2009:0617:2-5-3.png)]]
    298300
    299301-----
     
    305307--------
    306308 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示:
    307 [[Image(2-6.png)]]
     309[[Image(wiki:waue:2009:0617:2-6.png)]]
    308310
    309311-------------
    310312 Step2. 進行eclipse 與 hadoop 間的設定(2)
    311 [[Image(2-6-1.png)]]
     313[[Image(wiki:waue:2009:0617:2-6-1.png)]]
    312314
    313315{{{
     
    322324
    323325設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構
    324 [[Image(2-6-2.png)]]
     326[[Image(wiki:waue:2009:0617:2-6-2.png)]]
    325327-------------
    326328
     
    340342 
    341343 || File ->  || new ->  || mapper ||
    342 [[Image(file-new-mapper.png)]]
     344[[Image(wiki:waue:2009:0617:file-new-mapper.png)]]
    343345
    344346-----------
     
    346348 2. create
    347349 
    348 [[Image(3-1.png)]]
     350[[Image(wiki:waue:2009:0617:3-1.png)]]
    349351{{{
    350352#!sh
     
    385387
    386388建立mapper.java後,貼入程式碼
    387 [[Image(3-2.png)]]
     389[[Image(wiki:waue:2009:0617:3-2.png)]]
    388390
    389391------------
     
    394396
    395397 * File -> new -> reducer
    396 [[Image(file-new-reducer.png)]]
     398[[Image(wiki:waue:2009:0617:file-new-reducer.png)]]
    397399
    398400-------
    399401 2. create
    400 [[Image(3-3.png)]]
     402[[Image(wiki:waue:2009:0617:3-3.png)]]
    401403
    402404{{{
     
    437439
    438440 * File -> new -> Map/Reduce Driver
    439 [[Image(file-new-mr-driver.png)]]
     441[[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]]
    440442----------
    441443
     
    445447
    446448建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver
    447 [[Image(3-4.png)]]
     449[[Image(wiki:waue:2009:0617:3-4.png)]]
    448450------------
    449451
     
    499501
    500502三個檔完成後並存檔後,整個程式建立完成
    501 [[Image(3-5.png)]]
     503[[Image(wiki:waue:2009:0617:3-5.png)]]
    502504
    503505-------
     
    518520   * 右鍵點選WordCount.java -> run as -> run on Hadoop :沒有效果
    519521   
    520 [[Image(run-on-hadoop.png)]]
     522[[Image(wiki:waue:2009:0617:run-on-hadoop.png)]]
    521523
    522524 * 因此,4.1 提供一個eclipse 上解除 run-on-hadoop 封印的方法。而4.2 則是避開run-on-hadoop 這個功能,用command mode端指令的方法執行。
     
    616618 * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔
    617619 
    618 [[Image(4-1.png)]]
     620[[Image(wiki:waue:2009:0617:4-1.png)]]
    619621------
    620622 * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助
    621 [[Image(4-2.png)]]
     623[[Image(wiki:waue:2009:0617:4-2.png)]]
    622624
    623625 === make output ===