close
Warning:
Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_fs.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.
- Timestamp:
-
Apr 26, 2010, 4:31:08 PM (14 years ago)
- Author:
-
waue
- Comment:
-
--
Legend:
- Unmodified
- Added
- Removed
- Modified
-
v38
|
v39
|
|
207 | 207 | |
208 | 208 | 一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值 |
209 | | [[Image(2-1.png)]] |
| 209 | |
| 210 | [[Image(wiki:waue:2009:0617:2-1.png)]] |
| 211 | |
210 | 212 | ------- |
211 | 213 | |
… |
… |
|
218 | 220 | || window -> || open pers.. -> || other.. -> || map/reduce|| |
219 | 221 | |
220 | | [[Image(win-open-other.png)]] |
| 222 | [[Image(wiki:waue:2009:0617:win-open-other.png)]] |
221 | 223 | |
222 | 224 | ------- |
223 | 225 | |
224 | 226 | 設定要用 Map/Reduce 的視野 |
225 | | [[Image(2-2.png)]] |
| 227 | [[Image(wiki:waue:2009:0617:2-2.png)]] |
226 | 228 | |
227 | 229 | --------- |
228 | 230 | |
229 | 231 | 使用 Map/Reduce 的視野後的介面呈現 |
230 | | [[Image(2-3.png)]] |
| 232 | [[Image(wiki:waue:2009:0617:2-3.png)]] |
231 | 233 | |
232 | 234 | -------- |
… |
… |
|
235 | 237 | |
236 | 238 | || file -> || new -> || project -> || Map/Reduce -> || Map/Reduce Project -> || next || |
237 | | [[Image(file-new-project.png)]] |
| 239 | [[Image(wiki:waue:2009:0617:file-new-project.png)]] |
238 | 240 | |
239 | 241 | -------- |
… |
… |
|
241 | 243 | 建立mapreduce專案(1) |
242 | 244 | |
243 | | [[Image(2-4.png)]] |
| 245 | [[Image(wiki:waue:2009:0617:2-4.png)]] |
244 | 246 | |
245 | 247 | ----------- |
… |
… |
|
253 | 255 | }}} |
254 | 256 | |
255 | | [[Image(2-4-2.png)]] |
| 257 | [[Image(wiki:waue:2009:0617:2-4-2.png)]] |
256 | 258 | |
257 | 259 | |
… |
… |
|
266 | 268 | Step1. 右鍵點選project的properties做細部設定 |
267 | 269 | |
268 | | [[Image(2-5.png)]] |
| 270 | [[Image(wiki:waue:2009:0617:2-5.png)]] |
269 | 271 | |
270 | 272 | ---------- |
… |
… |
|
273 | 275 | |
274 | 276 | hadoop的javadoc的設定(1) |
275 | | [[Image(2-5-1.png)]] |
| 277 | [[Image(wiki:waue:2009:0617:Image(2-5-1.png)]] |
276 | 278 | |
277 | 279 | * java Build Path -> Libraries -> hadoop-0.20.0-ant.jar |
… |
… |
|
288 | 290 | ------------ |
289 | 291 | Step3. hadoop的javadoc的設定完後(2) |
290 | | [[Image(2-5-2.png)]] |
| 292 | [[Image(wiki:waue:2009:0617:Image(2-5-2.png)]] |
291 | 293 | |
292 | 294 | ------------ |
… |
… |
|
295 | 297 | * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/ |
296 | 298 | |
297 | | [[Image(2-5-3.png)]] |
| 299 | [[Image(wiki:waue:2009:0617:2-5-3.png)]] |
298 | 300 | |
299 | 301 | ----- |
… |
… |
|
305 | 307 | -------- |
306 | 308 | Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示: |
307 | | [[Image(2-6.png)]] |
| 309 | [[Image(wiki:waue:2009:0617:2-6.png)]] |
308 | 310 | |
309 | 311 | ------------- |
310 | 312 | Step2. 進行eclipse 與 hadoop 間的設定(2) |
311 | | [[Image(2-6-1.png)]] |
| 313 | [[Image(wiki:waue:2009:0617:2-6-1.png)]] |
312 | 314 | |
313 | 315 | {{{ |
… |
… |
|
322 | 324 | |
323 | 325 | 設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構 |
324 | | [[Image(2-6-2.png)]] |
| 326 | [[Image(wiki:waue:2009:0617:2-6-2.png)]] |
325 | 327 | ------------- |
326 | 328 | |
… |
… |
|
340 | 342 | |
341 | 343 | || File -> || new -> || mapper || |
342 | | [[Image(file-new-mapper.png)]] |
| 344 | [[Image(wiki:waue:2009:0617:file-new-mapper.png)]] |
343 | 345 | |
344 | 346 | ----------- |
… |
… |
|
346 | 348 | 2. create |
347 | 349 | |
348 | | [[Image(3-1.png)]] |
| 350 | [[Image(wiki:waue:2009:0617:3-1.png)]] |
349 | 351 | {{{ |
350 | 352 | #!sh |
… |
… |
|
385 | 387 | |
386 | 388 | 建立mapper.java後,貼入程式碼 |
387 | | [[Image(3-2.png)]] |
| 389 | [[Image(wiki:waue:2009:0617:3-2.png)]] |
388 | 390 | |
389 | 391 | ------------ |
… |
… |
|
394 | 396 | |
395 | 397 | * File -> new -> reducer |
396 | | [[Image(file-new-reducer.png)]] |
| 398 | [[Image(wiki:waue:2009:0617:file-new-reducer.png)]] |
397 | 399 | |
398 | 400 | ------- |
399 | 401 | 2. create |
400 | | [[Image(3-3.png)]] |
| 402 | [[Image(wiki:waue:2009:0617:3-3.png)]] |
401 | 403 | |
402 | 404 | {{{ |
… |
… |
|
437 | 439 | |
438 | 440 | * File -> new -> Map/Reduce Driver |
439 | | [[Image(file-new-mr-driver.png)]] |
| 441 | [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]] |
440 | 442 | ---------- |
441 | 443 | |
… |
… |
|
445 | 447 | |
446 | 448 | 建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver |
447 | | [[Image(3-4.png)]] |
| 449 | [[Image(wiki:waue:2009:0617:3-4.png)]] |
448 | 450 | ------------ |
449 | 451 | |
… |
… |
|
499 | 501 | |
500 | 502 | 三個檔完成後並存檔後,整個程式建立完成 |
501 | | [[Image(3-5.png)]] |
| 503 | [[Image(wiki:waue:2009:0617:3-5.png)]] |
502 | 504 | |
503 | 505 | ------- |
… |
… |
|
518 | 520 | * 右鍵點選WordCount.java -> run as -> run on Hadoop :沒有效果 |
519 | 521 | |
520 | | [[Image(run-on-hadoop.png)]] |
| 522 | [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]] |
521 | 523 | |
522 | 524 | * 因此,4.1 提供一個eclipse 上解除 run-on-hadoop 封印的方法。而4.2 則是避開run-on-hadoop 這個功能,用command mode端指令的方法執行。 |
… |
… |
|
616 | 618 | * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔 |
617 | 619 | |
618 | | [[Image(4-1.png)]] |
| 620 | [[Image(wiki:waue:2009:0617:4-1.png)]] |
619 | 621 | ------ |
620 | 622 | * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助 |
621 | | [[Image(4-2.png)]] |
| 623 | [[Image(wiki:waue:2009:0617:4-2.png)]] |
622 | 624 | |
623 | 625 | === make output === |