Changes between Version 26 and Version 27 of MR_manual
- Timestamp:
- Sep 2, 2008, 3:57:30 PM (16 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
MR_manual
v26 v27 8 8 9 9 == 1.1 簡介 == 10 * 近來雲端技術這個名詞由Google、Yahoo這兩個網路龍頭的推波助瀾而 刮起旋風,連微軟也宣稱下一代的作業系統為"雲端系統",因此雲端技術勢必為新的趨勢。國網中心為台灣重要的電腦技術研究中心,因此成立團隊開啟雲端技術的研究。[[BR]]10 * 近來雲端技術這個名詞由Google、Yahoo這兩個網路龍頭的推波助瀾而颳起旋風,連微軟也宣稱下一代的作業系統為"雲端系統",因此雲端技術勢必為新的趨勢。國網中心為台灣重要的電腦技術研究中心,因此成立團隊開啟雲端技術的研究。[[BR]] 11 11 * 本篇文章探討開源碼專案Hadoop以及HBase,第一章從概念開始介紹起,並在第二、三章詳細的說明如何安裝、設定及運行;第四章探討此系統的效能測試;前一部分介紹如何使用,後一部分開始偏重如何編寫Map Reduce的程式,因此在第五章介紹如何透過Eclipse建立起編程環境;第六章探討Map-Reduce的範例程式碼,最後附上相關連結。 12 12 == 1.2 雲端運算 == … … 56 56 $ apt-get install sun-java6-bin sun-java6-jdk sun-java6-jre sun-java6-plugin 57 57 }}} 58 suggest to remove the default java compiler “ gcj ”58 suggest to remove the default java compiler 「 gcj 」 59 59 {{{ 60 60 $ apt-get purge java-gcj-compat … … 170 170 $ vi conf/hbase-env.sh 171 171 export JAVA_HOME=/usr/lib/jvm/java-6-sun 172 CLASSPATH=/home/waue/workspace/hadoop/conf 173 }}} 174 172 175 }}} 173 176 * 用hbase連接hadoop DFS … … 253 256 * Then make sure http://localhost:50030/ by your explorer is on going. [[br]] 254 257 255 * Ps : if your system had error after restart, you could do there for resolving and renewing one. And repeat to “4. start up Hadoop”258 * Ps : if your system had error after restart, you could do there for resolving and renewing one. And repeat to 「4. start up Hadoop」 256 259 {{{ 257 260 $ cd $HADOOP_HOME … … 337 340 set compiler compliance level to 5.0 338 341 }}} 339 * Some eclipse-plugin may exhaust much resource, you may happen to “out of memory error”. We suggest to execute eclipse with some parameters as that :342 * Some eclipse-plugin may exhaust much resource, you may happen to 「out of memory error」. We suggest to execute eclipse with some parameters as that : 340 343 {{{ 341 344 $ eclipse -vmargs -Xmx 512m … … 352 355 > Finish 353 356 }}} 354 * at “Project explorer”, you will see “sample”tree. Now, you should create a sample code.357 * at 「Project explorer」, you will see 「sample」 tree. Now, you should create a sample code. 355 358 {{{ 356 359 Eclipse … … 362 365 [http://trac.nchc.org.tw/cloud/attachment/wiki/hadoop-sample-code/WordCount.java] 363 366 364 * paste the contents to your new adding file “!WordCount.java”367 * paste the contents to your new adding file 「!WordCount.java」 365 368 === 5.3.2. Connect to Hadoop File System === 366 369 … … 386 389 * It should show up under a little elephant icon in the Project Explorer (on the left side of Eclipse). 387 390 388 * ps : Pleast make sure your Hadoop is working on local system. If not, please refer “session 2 Hadoop Setup”for debuging, or you can not pass through.391 * ps : Pleast make sure your Hadoop is working on local system. If not, please refer 「session 2 Hadoop Setup」 for debuging, or you can not pass through. 389 392 390 393 {{{ … … 407 410 an existing server from the list below > finish 408 411 }}} 409 * A “console” tag will show beside “!MapReduce Server”tag.412 * A 「console」 tag will show beside 「!MapReduce Server」 tag. 410 413 411 414 * While Map Reduce is running, you can visit http://localhost:50030/ to view that Hadoop is dispatching jobs by Map Reduce.