Changes between Version 26 and Version 27 of MR_manual


Ignore:
Timestamp:
Sep 2, 2008, 3:57:30 PM (16 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • MR_manual

    v26 v27  
    88
    99 == 1.1 簡介 ==
    10  * 近來雲端技術這個名詞由Google、Yahoo這兩個網路龍頭的推波助瀾而起旋風,連微軟也宣稱下一代的作業系統為"雲端系統",因此雲端技術勢必為新的趨勢。國網中心為台灣重要的電腦技術研究中心,因此成立團隊開啟雲端技術的研究。[[BR]]
     10 * 近來雲端技術這個名詞由Google、Yahoo這兩個網路龍頭的推波助瀾而起旋風,連微軟也宣稱下一代的作業系統為"雲端系統",因此雲端技術勢必為新的趨勢。國網中心為台灣重要的電腦技術研究中心,因此成立團隊開啟雲端技術的研究。[[BR]]
    1111 * 本篇文章探討開源碼專案Hadoop以及HBase,第一章從概念開始介紹起,並在第二、三章詳細的說明如何安裝、設定及運行;第四章探討此系統的效能測試;前一部分介紹如何使用,後一部分開始偏重如何編寫Map Reduce的程式,因此在第五章介紹如何透過Eclipse建立起編程環境;第六章探討Map-Reduce的範例程式碼,最後附上相關連結。
    1212 == 1.2 雲端運算 ==
     
    5656$ apt-get install sun-java6-bin sun-java6-jdk sun-java6-jre sun-java6-plugin
    5757}}}
    58 suggest to remove the default java compiler “ gcj ”
     58suggest to remove the default java compiler 「 gcj 」
    5959{{{
    6060$ apt-get purge java-gcj-compat
     
    170170$ vi conf/hbase-env.sh
    171171  export JAVA_HOME=/usr/lib/jvm/java-6-sun
     172  CLASSPATH=/home/waue/workspace/hadoop/conf
     173}}}
     174
    172175}}}
    173176    * 用hbase連接hadoop DFS
     
    253256 * Then make sure http://localhost:50030/ by your explorer is on going.  [[br]]
    254257
    255  * Ps : if your system had error after restart, you could do there for resolving and renewing one. And repeat to “4. start up Hadoop”
     258 * Ps : if your system had error after restart, you could do there for resolving and renewing one. And repeat to 「4. start up Hadoop」
    256259{{{
    257260 $ cd $HADOOP_HOME
     
    337340set compiler compliance level to 5.0
    338341}}}
    339  * Some eclipse-plugin may exhaust much resource, you may happen to “out of memory error”. We suggest to execute eclipse with some parameters as that :
     342 * Some eclipse-plugin may exhaust much resource, you may happen to 「out of memory error」. We suggest to execute eclipse with some parameters as that :
    340343{{{
    341344 $  eclipse -vmargs -Xmx 512m
     
    352355 > Finish
    353356}}}
    354  * at “Project explorer”, you will see “sample” tree. Now, you should create a sample code.
     357 * at 「Project explorer」, you will see 「sample」 tree. Now, you should create a sample code.
    355358{{{
    356359Eclipse
     
    362365[http://trac.nchc.org.tw/cloud/attachment/wiki/hadoop-sample-code/WordCount.java]
    363366
    364  * paste the contents to your new adding file “!WordCount.java”
     367 * paste the contents to your new adding file 「!WordCount.java」
    365368 === 5.3.2. Connect to Hadoop File System ===
    366369
     
    386389 * It should show up under a little elephant icon in the Project Explorer  (on the left side of Eclipse).
    387390
    388  * ps  :  Pleast make sure your Hadoop is working on local system. If not, please refer “session 2 Hadoop Setup” for debuging, or you can not pass through.
     391 * ps  :  Pleast make sure your Hadoop is working on local system. If not, please refer 「session 2 Hadoop Setup」 for debuging, or you can not pass through.
    389392
    390393{{{
     
    407410an existing server from the list below > finish
    408411}}}
    409  * A “console” tag will show beside “!MapReduce Server” tag.
     412 * A 「console」 tag will show beside 「!MapReduce Server」 tag.
    410413
    411414 * While Map Reduce is running, you can visit http://localhost:50030/ to view that Hadoop is dispatching jobs by Map Reduce.