Changes between Version 3 and Version 4 of waue/2009/0822


Ignore:
Timestamp:
Aug 19, 2009, 12:18:00 PM (15 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • waue/2009/0822

    v3 v4  
    4141 1 將javadoc (jdk-6u10-docs.zip) 下載下來
    4242 [https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jdk-6u10-docs-oth-JPR@CDS-CDS_Developer 下載點]
    43 [[Image(wiki:waue:2009:0617:1-1.png)]]
     43[[Image(wiki:waue/2009/0617:1-1.png)]]
    4444 
    4545 2 下載完後將檔案放在 /tmp/ 下
     
    112112
    113113一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值
    114 [[Image(wiki:waue:2009:0617:2-1.png)]]
     114[[Image(wiki:waue/2009/0617:2-1.png)]]
    115115-------
    116116
     
    123123|| window -> || open pers.. -> || other.. -> || map/reduce||
    124124
    125 [[Image(wiki:waue:2009:0617:win-open-other.png)]]
     125[[Image(wiki:waue/2009/0617:win-open-other.png)]]
    126126
    127127-------
    128128
    129129設定要用 Map/Reduce 的視野
    130 [[Image(wiki:waue:2009:0617:2-2.png)]]
     130[[Image(wiki:waue/2009/0617:2-2.png)]]
    131131
    132132---------
    133133
    134134使用 Map/Reduce 的視野後的介面呈現
    135 [[Image(wiki:waue:2009:0617:2-3.png)]]
     135[[Image(wiki:waue/2009/0617:2-3.png)]]
    136136
    137137--------
     
    140140
    141141 || file ->  || new ->  || project ->  || Map/Reduce ->  || Map/Reduce Project -> ||  next ||
    142 [[Image(wiki:waue:2009:0617:file-new-project.png)]]
     142[[Image(wiki:waue/2009/0617:file-new-project.png)]]
    143143
    144144--------
     
    146146建立mapreduce專案(1)
    147147
    148 [[Image(wiki:waue:2009:0617:2-4.png)]]
     148[[Image(wiki:waue/2009/0617:2-4.png)]]
    149149
    150150-----------
     
    158158}}}
    159159
    160 [[Image(wiki:waue:2009:0617:2-4-2.png)]]
     160[[Image(wiki:waue/2009/0617:2-4-2.png)]]
    161161
    162162
     
    171171 Step1. 右鍵點選project的properties做細部設定
    172172
    173 [[Image(wiki:waue:2009:0617:2-5.png)]]
     173[[Image(wiki:waue/2009/0617:2-5.png)]]
    174174
    175175----------
     
    178178
    179179hadoop的javadoc的設定(1)
    180 [[Image(wiki:waue:2009:0617:2-5-1.png)]]
     180[[Image(wiki:waue/2009/0617:2-5-1.png)]]
    181181
    182182 * java Build Path -> Libraries -> hadoop0.18.3-ant.jar
     
    193193------------
    194194 Step3. hadoop的javadoc的設定完後(2)
    195 [[Image(wiki:waue:2009:0617:2-5-2.png)]]
     195[[Image(wiki:waue/2009/0617:2-5-2.png)]]
    196196
    197197------------
     
    200200 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/
    201201 
    202 [[Image(wiki:waue:2009:0617:2-5-3.png)]]
     202[[Image(wiki:waue/2009/0617:2-5-3.png)]]
    203203
    204204-----
     
    210210--------
    211211 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示:
    212 [[Image(wiki:waue:2009:0617:2-6.png)]]
     212[[Image(wiki:waue/2009/0617:2-6.png)]]
    213213
    214214-------------
    215215 Step2. 進行eclipse 與 hadoop 間的設定(2)
    216 [[Image(wiki:waue:2009:0617:2-6-1.png)]]
     216[[Image(wiki:waue/2009/0617:2-6-1.png)]]
    217217
    218218{{{
     
    227227
    228228設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構
    229 [[Image(wiki:waue:2009:0617:2-6-2.png)]]
     229[[Image(wiki:waue/2009/0617:2-6-2.png)]]
    230230-------------
    231231
     
    245245 
    246246 || File ->  || new ->  || mapper ||
    247 [[Image(wiki:waue:2009:0617:file-new-mapper.png)]]
     247[[Image(wiki:waue/2009/0617:file-new-mapper.png)]]
    248248
    249249-----------
     
    251251 2. create
    252252 
    253 [[Image(wiki:waue:2009:0617:3-1.png)]]
     253[[Image(wiki:waue/2009/0617:3-1.png)]]
    254254{{{
    255255#!sh
     
    290290
    291291建立mapper.java後,貼入程式碼
    292 [[Image(wiki:waue:2009:0617:3-2.png)]]
     292[[Image(wiki:waue/2009/0617:3-2.png)]]
    293293
    294294------------
     
    299299
    300300 * File -> new -> reducer
    301 [[Image(wiki:waue:2009:0617:file-new-reducer.png)]]
     301[[Image(wiki:waue/2009/0617:file-new-reducer.png)]]
    302302
    303303-------
    304304 2. create
    305 [[Image(wiki:waue:2009:0617:3-3.png)]]
     305[[Image(wiki:waue/2009/0617:3-3.png)]]
    306306
    307307{{{
     
    342342
    343343 * File -> new -> Map/Reduce Driver
    344 [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]]
     344[[Image(wiki:waue/2009/0617:file-new-mr-driver.png)]]
    345345----------
    346346
     
    350350
    351351建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver
    352 [[Image(wiki:waue:2009:0617:3-4.png)]]
     352[[Image(wiki:waue/2009/0617:3-4.png)]]
    353353------------
    354354
     
    404404
    405405三個檔完成後並存檔後,整個程式建立完成
    406 [[Image(wiki:waue:2009:0617:3-5.png)]]
     406[[Image(wiki:waue/2009/0617:3-5.png)]]
    407407
    408408-------
     
    425425   * 右鍵點選WordCount.java -> run as -> run on Hadoop
    426426   
    427 [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]]
     427[[Image(wiki:waue/2009/0617:run-on-hadoop.png)]]
    428428
    429429
     
    432432 * eclipse 可以產生出jar檔 :
    433433 
    434 File -> Export -> java -> JAR file
     434File -> Export -> java -> JAR file [[br]]
    435435-> next ->
    436436--------
    437437選擇要匯出的專案 ->
    438 jarfile: /home/hadooper/mytest.jar ->
     438jarfile: /home/hadooper/mytest.jar -> [[br]]
    439439next ->
    440440--------
    441441next ->
    442442--------
    443 main class: 選擇有Main的class   ->
     443main class: 選擇有Main的class   -> [[br]]
    444444Finish
    445445--------
     
    530530 * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔
    531531 
    532 [[Image(wiki:waue:2009:0617:4-1.png)]]
     532[[Image(wiki:waue/2009/0617:4-1.png)]]
    533533------
    534534 * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助
    535 [[Image(wiki:waue:2009:0617:4-2.png)]]
     535[[Image(wiki:waue/2009/0617:4-2.png)]]
    536536
    537537 === make output ===