wiki:Hbase_Lab1

Hadoop、Hbase、Zookeeper安裝

前言

由於Nutch2.0開始,已經可以將索引結果存入Hbase,但由於Hbase內建的Zookeeper似乎沒有太穩定,因此自己手動整合才是王道...

安裝環境

  • OS: Ubuntu12.04
  • Java: OpenJDK 6

套件版本

  • Hadoop 1.0.3
  • Hbase 0.90.6(沒有選用最新的原因是因為Nutch官網上有提到0.90.x為他們的測試環境)
  • Zookeper 3.3.6

安裝Hadoop

下載&解壓縮Hadoop1.0.3

修改$HAHOOP_HOME/conf/hadoop-env.sh

  • 設定JAVA_HOME
    export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
    

修改$HAHOOP_HOME/conf/core-site.xml

<configuration>
     <property>
         <name>fs.default.name</name>
         <value>hdfs://localhost:9000</value>
     </property>
</configuration>

修改$HAHOOP_HOME/conf/mapred-site.xml

<configuration>
 <property>
         <name>mapred.job.tracker</name>
         <value>localhost:9001</value>
     </property>
</configuration>

修改$HAHOOP_HOME/conf/hdfs-site.xml

<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>dfs.support.append</name>
    <value>true</value>
  </property>
</configuration>

格式化HDFS

shunfa@shunfa-vmmVBox:/opt/hadoop-1.0.3$ bin/hadoop namenode -format

啟動Hadoop

shunfa@shunfa-vmmVBox:/opt/hadoop-1.0.3$ bin/start-all.sh

安裝Zookeeper

下載並解壓縮zookeeper到/opt/

建立zookeeper運作時需要的目錄並複製預設設定檔

~$ cd /opt/zookeeper-3.3.6/
~$ sudo mkdir /var/zookeeper
~$ sudo chown -R shunfa:shunfa /var/zookeeper
~$ sudo chown -R shunfa:shunfa /opt/zookeeper-3.3.6/
~$ cp conf/zoo_sample.cfg conf/zoo.cfg

修改zoo.cfg

tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/zookeeper
clientPort=2181

啟動

~$ /opt/zookeeper-3.3.6/bin/zkServer.sh start

安裝Hbase

下載並解壓縮Hbase到/opt/

修改 /opt/hbase-0.90.6/conf/hbase-env.sh

export HBASE_MANAGES_ZK=false
export JAVA_HOME=/usr/lib/jvm/java-6-openjdk
export HBASE_HEAPSIZE=256
export HBASE_CLASSPATH=/opt/hadoop-1.0.3/conf

修改 /opt/hbase-0.90.6/conf/hbase-site.xml

<configuration>
  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://localhost:54310/hbase</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/var/zookeeper</value>
  </property>
  <property>
    <name>dfs.support.append</name>
    <value>true</value>
  </property>
 <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
 <property>
    <name>hbase.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>
</configuration>

複製相關lib

~$ cp $HADOOP_HOME/lib/commons-configuration-1.6.jar $HBASE_HOME/lib/
~$ cp $HADOOP_HOME/hadoop-core-1.0.3.jar $HBASE_HOME/lib/

啟動

~$ /opt/hbase-0.90.6/bin/start-hbase.sh

完成

  • 執行結果如下:
    shunfa@shunfa-vmmVBox:~$ jps
    23659 HRegionServer
    21788 NameNode
    22230 JobTracker
    22148 SecondaryNameNode
    5585 Jps
    21972 DataNode
    22410 TaskTracker
    23069 QuorumPeerMain
    

Last modified 12 years ago Last modified on Dec 10, 2012, 5:01:41 PM