wiki:HBase

Version 4 (modified by sunny, 16 years ago) (diff)

--

HBase

【簡易安裝手冊】

  • How to download and run HBase? in about 10 Minutes.
    • 以下有幾個簡單的步驟:(包含了下載.安裝及配置HBase)
      • 首先需要先下載HBase最新且穩定的版本Download
      • 接著將下載的檔案移到你想要跑HBase的目錄下,如/home/user/hbase
      • 移動到/home/user/hbase此目錄下
        $ cd /home/user/hbase
        
      • 編輯conf/hbase-env.sh檔案,加入以下這行 (需視使用者的JAVA路徑而定)
        $ vi conf/hbase-env.sh
          export JAVA_HOME=/usr/lib/jvm/java-6-sun-1.6.0.03
        
      • 啟動hbase
        $ bin/start-hbase.sh
        
      • 可以在預設的logs/目錄下看到不少logs檔案
      • 可以執行以下指令進到HQL指令模式,可以試試看
        $ bin/hbase shell
        Hbase Shell, 0.0.2 version.
        Copyright (c) 2007 by udanax, licensed to Apache Software Foundation.
        Type 'help;' for usage.
        
        hql > help;
        Type 'help COMMAND;' to see command-specific usage.
        +-------------------------+-------------------------+-------------------------+
        | Command                 | Description             | Example                 |
        +-------------------------+-------------------------+-------------------------+
        | EXIT                    | Exit shell              | EXIT;                   |
        +-------------------------+-------------------------+-------------------------+
        | ALTER                   | Alter structure of table| ALTER TABLE table_name A|
        |                         |                         | DD column_spec | ADD (co|
        |                         |                         | lumn_spec, column_spec, |
        |                         |                         | ...) | CHANGE column_fam|
        |                         |                         | ily column_spec | DROP c|
        |                         |                         | olumn_family_name | CHAN|
        |                         |                         | GE column_spec;         |
        +-------------------------+-------------------------+-------------------------+
        | CREATE                  | Create tables           | CREATE TABLE table_name |
        |                         |                         | (column_family_name [MAX|
        |                         |                         | _VERSIONS=n] [MAX_LENGTH|
        |                         |                         | =n] [COMPRESSION=NONE|RE|
        |                         |                         | CORD|BLOCK] [IN_MEMORY] |
        |                         |                         | [BLOOMFILTER=NONE|BLOOMF|
        |                         |                         | ILTER|COUNTING_BLOOMFILT|
        |                         |                         | ER|RETOUCHED_BLOOMFILTER|
        |                         |                         |  VECTOR_SIZE=n NUM_HASH=|
        |                         |                         | n], ...)                |
        +-------------------------+-------------------------+-------------------------+
        | INSERT                  | Insert values into table| INSERT INTO table_name (|
        |                         |                         | column_name, ...) VALUES|
        |                         |                         |  ('value', ...) WHERE ro|
        |                         |                         | w='row_key' [TIMESTAMP '|
        |                         |                         | timestamp'];            |
        +-------------------------+-------------------------+-------------------------+
        | DROP                    | Drop tables             | DROP TABLE table_name [,|
        |                         |                         |  table_name] ...;       |
        +-------------------------+-------------------------+-------------------------+
        | TRUNCATE                | Truncate table is used t| TRUNCATE TABLE table_nam|
        |                         | o clean all data from a | e;                      |
        |                         | table                   |                         |
        +-------------------------+-------------------------+-------------------------+
        | DESCRIBE                | Print table information | [DESCRIBE|DESC] table_na|
        |                         |                         | me;                     |
        +-------------------------+-------------------------+-------------------------+
        | JAR                     | Hadoop RunJar util      | JAR jarFile [mainClass] |
        |                         |                         | arguments...;           |
        +-------------------------+-------------------------+-------------------------+
        | FS                      | Hadoop FsShell; entering| FS [-option] arguments..|
        |                         |  a lone 'FS;' will emit | ;                       |
        |                         | usage                   |                         |
        +-------------------------+-------------------------+-------------------------+
        | DELETE                  | Delete table data       | DELETE {column_name, [, |
        |                         |                         | column_name] ... | *} FR|
        |                         |                         | OM table_name WHERE row=|
        |                         |                         | 'row-key';              |
        +-------------------------+-------------------------+-------------------------+
        | SELECT                  | Select values from table| SELECT {column_name, [, |
        |                         |                         | column_name] ... | expr[|
        |                         |                         | alias] | * } FROM table_|
        |                         |                         | name [WHERE row='row_key|
        |                         |                         | ' | STARTING FROM 'row-k|
        |                         |                         | ey' [UNTIL 'stop-key']] |
        |                         |                         | [NUM_VERSIONS = version_|
        |                         |                         | count] [TIMESTAMP 'times|
        |                         |                         | tamp'] [LIMIT = row_coun|
        |                         |                         | t] [INTO FILE 'file_name|
        |                         |                         | '];                     |
        +-------------------------+-------------------------+-------------------------+
        | SHOW                    | Show information about s| SHOW TABLES [or substitu|
        |                         | elected title           | tion variable name];    |
        +-------------------------+-------------------------+-------------------------+
        | CLEAR                   | Clear the screen        | CLEAR;                  |
        +-------------------------+-------------------------+-------------------------+
        hql > help;
        Type 'help COMMAND;' to see command-specific usage.
        +-------------------------+-------------------------+-------------------------+
        | Command                 | Description             | Example                 |
        +-------------------------+-------------------------+-------------------------+
        | EXIT                    | Exit shell              | EXIT;                   |
        +-------------------------+-------------------------+-------------------------+
        | ALTER                   | Alter structure of table| ALTER TABLE table_name A|
        |                         |                         | DD column_spec | ADD (co|
        |                         |                         | lumn_spec, column_spec, |
        |                         |                         | ...) | CHANGE column_fam|
        |                         |                         | ily column_spec | DROP c|
        |                         |                         | olumn_family_name | CHAN|
        |                         |                         | GE column_spec;         |
        +-------------------------+-------------------------+-------------------------+
        | CREATE                  | Create tables           | CREATE TABLE table_name |
        |                         |                         | (column_family_name [MAX|
        |                         |                         | _VERSIONS=n] [MAX_LENGTH|
        |                         |                         | =n] [COMPRESSION=NONE|RE|
        |                         |                         | CORD|BLOCK] [IN_MEMORY] |
        |                         |                         | [BLOOMFILTER=NONE|BLOOMF|
        |                         |                         | ILTER|COUNTING_BLOOMFILT|
        |                         |                         | ER|RETOUCHED_BLOOMFILTER|
        |                         |                         |  VECTOR_SIZE=n NUM_HASH=|
        |                         |                         | n], ...)                |
        +-------------------------+-------------------------+-------------------------+
        | INSERT                  | Insert values into table| INSERT INTO table_name (|
        |                         |                         | column_name, ...) VALUES|
        |                         |                         |  ('value', ...) WHERE ro|
        |                         |                         | w='row_key' [TIMESTAMP '|
        |                         |                         | timestamp'];            |
        +-------------------------+-------------------------+-------------------------+
        | DROP                    | Drop tables             | DROP TABLE table_name [,|
        |                         |                         |  table_name] ...;       |
        +-------------------------+-------------------------+-------------------------+
        | TRUNCATE                | Truncate table is used t| TRUNCATE TABLE table_nam|
        |                         | o clean all data from a | e;                      |
        |                         | table                   |                         |
        +-------------------------+-------------------------+-------------------------+
        | DESCRIBE                | Print table information | [DESCRIBE|DESC] table_na|
        |                         |                         | me;                     |
        +-------------------------+-------------------------+-------------------------+
        | JAR                     | Hadoop RunJar util      | JAR jarFile [mainClass] |
        |                         |                         | arguments...;           |
        +-------------------------+-------------------------+-------------------------+
        | FS                      | Hadoop FsShell; entering| FS [-option] arguments..|
        |                         |  a lone 'FS;' will emit | ;                       |
        |                         | usage                   |                         |
        +-------------------------+-------------------------+-------------------------+
        | DELETE                  | Delete table data       | DELETE {column_name, [, |
        |                         |                         | column_name] ... | *} FR|
        |                         |                         | OM table_name WHERE row=|
        |                         |                         | 'row-key';              |
        +-------------------------+-------------------------+-------------------------+
        | SELECT                  | Select values from table| SELECT {column_name, [, |
        |                         |                         | column_name] ... | expr[|
        |                         |                         | alias] | * } FROM table_|
        |                         |                         | name [WHERE row='row_key|
        |                         |                         | ' | STARTING FROM 'row-k|
        |                         |                         | ey' [UNTIL 'stop-key']] |
        |                         |                         | [NUM_VERSIONS = version_|
        |                         |                         | count] [TIMESTAMP 'times|
        |                         |                         | tamp'] [LIMIT = row_coun|
        |                         |                         | t] [INTO FILE 'file_name|
        |                         |                         | '];                     |
        +-------------------------+-------------------------+-------------------------+
        | SHOW                    | Show information about s| SHOW TABLES [or substitu|
        |                         | elected title           | tion variable name];    |
        +-------------------------+-------------------------+-------------------------+
        | CLEAR                   | Clear the screen        | CLEAR;                  |
        +-------------------------+-------------------------+-------------------------+
        hql > 
        
      • 要結束hbase server 可用以下指令,而避免使用kill pid指令來殺掉pid,尤其是恐怖的指令kill -9 pid
        $ bin/stop-hbase.sh
        
        • 解釋: kill -9 是當 kill -15 砍不掉的時候 才用,kill -15 會去幫你注意檔案或裝置讀寫有沒有關起來,kill -9 就不管,直接停掉process (reference from sunny)
      • 注意:如果有要跑一個遠端的HDFS,而希望可以用hbase連接,此時需要編輯conf/hbase-site.xml檔案,如下
        $ vi conf/hbase-site.xml 
        <configuration>
          <property>
             <name>hbase.rootdir</name>
             <value>file:///tmp/hbase-${user.home}/hbase</value>
             <value>hdfs://localhost:9000/hbase</value>
             <description>
                 The directory shared by region servers.
                 Should be fully-qualified to include the filesystem to use.
             </description>
          </property>
        </configuration>
        

【常見問題】

  1. HBase資料庫系統在執行關閉伺服器指令後,再重新啟動時裡面的資料表都會遺失?
    Ans: 在hbase/conf目錄裡有一個預設的組態檔hbase-default.xml,裡面的hbase.rootdir屬性預設值是"file:///tmp/hbase-${user.home}/hbase", 這表示hbase的檔案系統預設是掛載在使用本機的/tmp目錄底下,如果是沒有特別去備份/tmp目錄,或是把/tmp掛載到ram disc裡的情況下,裡面的資料庫資料會在重新啟動作業系統後被作業系統自動清空,因此,只要把hbase.rootdir指定到其他可以永續存放的磁碟空間,就可以解決資料表存放的問題。hbase.rootdir可以直接設定在hbase-default.xml裡或是hbase-site.xml,hbase系統讀取設定檔的優先權順序是hbase-site.xml高於hbase-default.xml,如果想保留原本的預設值,一般是建議直接放到hbase-site.xml就可以了。屬性內容如下:
    value可以自由設定路徑,description非必填,name則是固定的屬性名稱
    <property>
        <name>hbase.rootdir</name>
        <value>file:///${user.home}/hbase/dbstored</value>
         <description>The directory shared by region servers.
          Should be fully-qualified to include the filesystem to use.
         </description>
      </property>
    

【研究團隊】

  • 此網頁內容為國網中心雲端技術研究小組之雲端資料庫(Cloud DB)研究人員之共同研究心得分享,內容如有疏誤請多包含。
  • 技術顧問: 王耀聰
  • 研究人員: 陳惠珊,陳威宇,王心怡
  • 編輯人員: 陳惠珊,王心怡