close
Warning:
Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_fs.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.
- Timestamp:
-
Apr 26, 2010, 6:28:55 PM (16 years ago)
- Author:
-
waue
- Comment:
-
--
Legend:
- Unmodified
- Added
- Removed
- Modified
-
|
v18
|
v19
|
|
| 37 | 37 | 1 將javadoc (jdk-6u10-docs.zip) 下載下來 |
| 38 | 38 | [https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jdk-6u10-docs-oth-JPR@CDS-CDS_Developer 下載點] |
| 39 | | [[Image(wiki:waue:2009:0617:1-1.png)]] |
| 40 | | [[Image(wiki:0428Hadoop_Lab1:hadoop_administration.png)]] |
| | 39 | [[Image(wiki:Hadoop_Lab5:1-1.png)]] |
| | 40 | |
| 41 | 41 | |
| 42 | 42 | 2 下載完後將檔案放在 /tmp/ 下 |
| … |
… |
|
| 196 | 196 | 一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值 |
| 197 | 197 | |
| 198 | | [[Image(wiki:waue:2009:0617:2-1.png)]] |
| | 198 | [[Image(wiki:Hadoop_Lab5:2-1.png)]] |
| 199 | 199 | |
| 200 | 200 | ------- |
| … |
… |
|
| 208 | 208 | || window -> || open pers.. -> || other.. -> || map/reduce|| |
| 209 | 209 | |
| 210 | | [[Image(wiki:waue:2009:0617:win-open-other.png)]] |
| | 210 | [[Image(wiki:Hadoop_Lab5:win-open-other.png)]] |
| 211 | 211 | |
| 212 | 212 | ------- |
| 213 | 213 | |
| 214 | 214 | 設定要用 Map/Reduce 的視野 |
| 215 | | [[Image(wiki:waue:2009:0617:2-2.png)]] |
| | 215 | [[Image(wiki:Hadoop_Lab5:2-2.png)]] |
| 216 | 216 | |
| 217 | 217 | --------- |
| 218 | 218 | |
| 219 | 219 | 使用 Map/Reduce 的視野後的介面呈現 |
| 220 | | [[Image(wiki:waue:2009:0617:2-3.png)]] |
| | 220 | [[Image(wiki:Hadoop_Lab5:2-3.png)]] |
| 221 | 221 | |
| 222 | 222 | -------- |
| … |
… |
|
| 225 | 225 | |
| 226 | 226 | || file -> || new -> || project -> || Map/Reduce -> || Map/Reduce Project -> || next || |
| 227 | | [[Image(wiki:waue:2009:0617:file-new-project.png)]] |
| | 227 | [[Image(wiki:Hadoop_Lab5:file-new-project.png)]] |
| 228 | 228 | |
| 229 | 229 | -------- |
| … |
… |
|
| 231 | 231 | 建立mapreduce專案(1) |
| 232 | 232 | |
| 233 | | [[Image(wiki:waue:2009:0617:2-4.png)]] |
| | 233 | [[Image(wiki:Hadoop_Lab5:2-4.png)]] |
| 234 | 234 | |
| 235 | 235 | ----------- |
| … |
… |
|
| 243 | 243 | }}} |
| 244 | 244 | |
| 245 | | [[Image(wiki:waue:2009:0617:2-4-2.png)]] |
| | 245 | [[Image(wiki:Hadoop_Lab5:2-4-2.png)]] |
| 246 | 246 | |
| 247 | 247 | |
| … |
… |
|
| 256 | 256 | Step1. 右鍵點選project的properties做細部設定 |
| 257 | 257 | |
| 258 | | [[Image(wiki:waue:2009:0617:2-5.png)]] |
| | 258 | [[Image(wiki:Hadoop_Lab5:2-5.png)]] |
| 259 | 259 | |
| 260 | 260 | ---------- |
| … |
… |
|
| 263 | 263 | |
| 264 | 264 | hadoop的javadoc的設定(1) |
| 265 | | [[Image(wiki:waue:2009:0617:Image(2-5-1.png)]] |
| | 265 | [[Image(wiki:Hadoop_Lab5:Image(2-5-1.png)]] |
| 266 | 266 | |
| 267 | 267 | * java Build Path -> Libraries -> hadoop-0.20.2-ant.jar |
| … |
… |
|
| 278 | 278 | ------------ |
| 279 | 279 | Step3. hadoop的javadoc的設定完後(2) |
| 280 | | [[Image(wiki:waue:2009:0617:Image(2-5-2.png)]] |
| | 280 | [[Image(wiki:Hadoop_Lab5:Image(2-5-2.png)]] |
| 281 | 281 | |
| 282 | 282 | ------------ |
| … |
… |
|
| 285 | 285 | * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/ |
| 286 | 286 | |
| 287 | | [[Image(wiki:waue:2009:0617:2-5-3.png)]] |
| | 287 | [[Image(wiki:Hadoop_Lab5:2-5-3.png)]] |
| 288 | 288 | |
| 289 | 289 | ----- |
| … |
… |
|
| 295 | 295 | -------- |
| 296 | 296 | Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示: |
| 297 | | [[Image(wiki:waue:2009:0617:2-6.png)]] |
| | 297 | [[Image(wiki:Hadoop_Lab5:2-6.png)]] |
| 298 | 298 | |
| 299 | 299 | ------------- |
| 300 | 300 | Step2. 進行eclipse 與 hadoop 間的設定(2) |
| 301 | | [[Image(wiki:waue:2009:0617:2-6-1.png)]] |
| | 301 | [[Image(wiki:Hadoop_Lab5:2-6-1.png)]] |
| 302 | 302 | |
| 303 | 303 | {{{ |
| … |
… |
|
| 312 | 312 | |
| 313 | 313 | 設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構 |
| 314 | | [[Image(wiki:waue:2009:0617:2-6-2.png)]] |
| | 314 | [[Image(wiki:Hadoop_Lab5:2-6-2.png)]] |
| 315 | 315 | ------------- |
| 316 | 316 | |
| … |
… |
|
| 330 | 330 | |
| 331 | 331 | || File -> || new -> || mapper || |
| 332 | | [[Image(wiki:waue:2009:0617:file-new-mapper.png)]] |
| | 332 | [[Image(wiki:Hadoop_Lab5:file-new-mapper.png)]] |
| 333 | 333 | |
| 334 | 334 | ----------- |
| … |
… |
|
| 336 | 336 | 2. create |
| 337 | 337 | |
| 338 | | [[Image(wiki:waue:2009:0617:3-1.png)]] |
| | 338 | [[Image(wiki:Hadoop_Lab5:3-1.png)]] |
| 339 | 339 | {{{ |
| 340 | 340 | #!sh |
| … |
… |
|
| 375 | 375 | |
| 376 | 376 | 建立mapper.java後,貼入程式碼 |
| 377 | | [[Image(wiki:waue:2009:0617:3-2.png)]] |
| | 377 | [[Image(wiki:Hadoop_Lab5:3-2.png)]] |
| 378 | 378 | |
| 379 | 379 | ------------ |
| … |
… |
|
| 384 | 384 | |
| 385 | 385 | * File -> new -> reducer |
| 386 | | [[Image(wiki:waue:2009:0617:file-new-reducer.png)]] |
| | 386 | [[Image(wiki:Hadoop_Lab5:file-new-reducer.png)]] |
| 387 | 387 | |
| 388 | 388 | ------- |
| 389 | 389 | 2. create |
| 390 | | [[Image(wiki:waue:2009:0617:3-3.png)]] |
| | 390 | [[Image(wiki:Hadoop_Lab5:3-3.png)]] |
| 391 | 391 | |
| 392 | 392 | {{{ |
| … |
… |
|
| 427 | 427 | |
| 428 | 428 | * File -> new -> Map/Reduce Driver |
| 429 | | [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]] |
| | 429 | [[Image(wiki:Hadoop_Lab5:file-new-mr-driver.png)]] |
| 430 | 430 | ---------- |
| 431 | 431 | |
| … |
… |
|
| 435 | 435 | |
| 436 | 436 | 建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver |
| 437 | | [[Image(wiki:waue:2009:0617:3-4.png)]] |
| | 437 | [[Image(wiki:Hadoop_Lab5:3-4.png)]] |
| 438 | 438 | ------------ |
| 439 | 439 | |
| … |
… |
|
| 489 | 489 | |
| 490 | 490 | 三個檔完成後並存檔後,整個程式建立完成 |
| 491 | | [[Image(wiki:waue:2009:0617:3-5.png)]] |
| | 491 | [[Image(wiki:Hadoop_Lab5:3-5.png)]] |
| 492 | 492 | |
| 493 | 493 | ------- |
| … |
… |
|
| 507 | 507 | * 右鍵點選WordCount.java -> run as -> run on Hadoop |
| 508 | 508 | |
| 509 | | [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]] |
| 510 | | |
| 511 | | |
| 512 | 509 | |
| 513 | 510 | = 五、結論 = |