[[PageOutline]]
= Hadoop、Hbase、Zookeeper安裝 =
== 前言 ==
由於Nutch2.0開始,已經可以將索引結果存入Hbase,但由於Hbase內建的Zookeeper似乎沒有太穩定,因此自己手動整合才是王道...
== 安裝環境 ==
* OS: Ubuntu12.04
* Java: OpenJDK 6
== 套件版本 ==
* Hadoop 1.0.3
* Hbase 0.90.6(沒有選用最新的原因是因為Nutch官網上有提到0.90.x為他們的測試環境)
* Zookeper 3.3.6
== 安裝Hadoop ==
=== 下載&解壓縮Hadoop1.0.3 ===
=== 修改$HAHOOP_HOME/conf/hadoop-env.sh ===
* 設定JAVA_HOME
{{{
#!text
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
}}}
=== 修改$HAHOOP_HOME/conf/core-site.xml ===
{{{
#!text
fs.default.name
hdfs://localhost:9000
}}}
=== 修改$HAHOOP_HOME/conf/mapred-site.xml ===
{{{
#!text
mapred.job.tracker
localhost:9001
}}}
=== 修改$HAHOOP_HOME/conf/hdfs-site.xml ===
{{{
#!text
dfs.replication
1
dfs.support.append
true
}}}
=== 格式化HDFS ===
{{{
shunfa@shunfa-vmmVBox:/opt/hadoop-1.0.3$ bin/hadoop namenode -format
}}}
=== 啟動Hadoop ===
{{{
shunfa@shunfa-vmmVBox:/opt/hadoop-1.0.3$ bin/start-all.sh
}}}
== 安裝Zookeeper ==
=== 下載並解壓縮zookeeper到/opt/ ===
=== 建立zookeeper運作時需要的目錄並複製預設設定檔 ===
{{{
~$ cd /opt/zookeeper-3.3.6/
~$ sudo mkdir /var/zookeeper
~$ sudo chown -R shunfa:shunfa /var/zookeeper
~$ sudo chown -R shunfa:shunfa /opt/zookeeper-3.3.6/
~$ cp conf/zoo_sample.cfg conf/zoo.cfg
}}}
=== 修改zoo.cfg ===
{{{
#!text
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/zookeeper
clientPort=2181
}}}
=== 啟動 ===
{{{
~$ /opt/zookeeper-3.3.6/bin/zkServer.sh start
}}}
== 安裝Hbase ==
=== 下載並解壓縮Hbase到/opt/ ===
=== 修改 /opt/hbase-0.90.6/conf/hbase-env.sh ===
{{{
#!text
export HBASE_MANAGES_ZK=false
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
export HBASE_HEAPSIZE=256
export HBASE_CLASSPATH=/opt/hadoop-1.0.3/conf
}}}
=== 修改 /opt/hbase-0.90.6/conf/hbase-site.xml ===
{{{
#!text
hbase.rootdir
hdfs://localhost:54310/hbase
hbase.zookeeper.property.dataDir
/var/zookeeper
dfs.support.append
true
hbase.cluster.distributed
true
hbase.zookeeper.property.clientPort
2181
}}}
=== 複製相關lib ===
{{{
~$ cp $HADOOP_HOME/lib/commons-configuration-1.6.jar $HBASE_HOME/lib/
~$ cp $HADOOP_HOME/hadoop-core-1.0.3.jar $HBASE_HOME/lib/
}}}
=== 啟動 ===
{{{
~$ /opt/hbase-0.90.6/bin/start-hbase.sh
}}}
== 完成 ==
* 執行結果如下:
{{{
shunfa@shunfa-vmmVBox:~$ jps
23659 HRegionServer
21788 NameNode
22230 JobTracker
22148 SecondaryNameNode
5585 Jps
21972 DataNode
22410 TaskTracker
23069 QuorumPeerMain
}}}