[[PageOutline]] = Hypertable Knowledge Database = == 【Hypertable 源起】 == 搜尋引擎公司Zvents以Google的9位研究人員在2006年發表的Bigtable(簡稱BT，Google內部使用的文件儲存系統)設計規格為基礎[FYI]《[http://labs.google.com/papers/bigtable-osdi06.pdf Bigtable：結構化數據的分佈存儲系統]》，推出了一款開放源碼的分散式數據儲存系統Hypertable專案。以 C++ 撰寫，可架在 HDFS 和 KFS 上，按照1000節點比例設計，0.9 alpha測試版已經在10個節點上測試過，儘管還在初期階段，但已有不錯的效能：寫入 28M 列的資料，各節點寫入速率可達 7MB/s，讀取速率可達 1M cells/s。 == 【Hypertable 簡介】 == 總結來說Hypertable是一個高效能，分散式，開放源碼，與欄位導向的資料庫，可以儲存,處理叢集電腦上大量的節構化與非結構化的資料，Hypertable提供C++的API及HQL(Hypertable Query Language)給用戶端來存取資料庫內容。Hypertable的用途並不是為了取代傳統的資料庫管理系統(像是MySQL與Oracle DB)，而是為了可以儲存與管理大量的資料集。傳統的關連式資料庫RDBMs(Relational Database Management)是交易導向式，提供許多進階功能給使用者查詢結構化的資料庫內容。Hypertable為了逹到規模可彈性調整以及高效能的輸出，捨棄像是RDBMs常在使用的join或其他query的功能特色，MySQL之類的RDBMs系統屬於row-oriented較適合用於寫入動作較頻繁的工作負載情形，Hypertable屬於column-oriented較適合讀取動作較頻繁的工作負載情形。Hypertable建構於分散式檔案系統(DFS)之上，目前有許多DFS系統，任一DFS均可架設Hypertable，一套DFS可以讓許多機器看起來就好像單一虛擬磁碟，而且都均有容錯與備份機制，因此DFSs可以結合許多叢集電腦的儲存資源，提供高速與大資料量的存取效能，故Hypertable架構在DFS之上也就可以提供高速與高容量的資料庫儲存空間。 == 【How it works】 == == 【Performance】 == * Performance Test: [http://code.google.com/p/hypertable/wiki/PerformanceTestAOLQueryLog AOL Query Log](from Google) * Machine Profile * 8 data nodes * each node * 1 x 1.8GHz Dual-core Opteron Processor 2210 * 4 GB RAM * 4 x 7200 RPM SATA drives (mounted JBOD) * The AOL query logs were inserted into an 8-node Hypertable cluster. The average size of each row key was ~7 bytes and each value was ~15 bytes. The insert rate (with 4 simultaneous insert processes) was approximately 410K inserts/s. The table was scanned at a rate of approximately 671K cells/s. == 【How we use it】 == == 【Similar Project】 == * [http://hadoop.apache.org/hbase/ HBase]: Bigtable-like structured storage for Hadoop HDFS * Hypertable is based very closely on the design of Bigtable, with a few modifications. Hypertable is designed for speed and is written in C++, while Hbase is in Java. * [http://thrudb.org/ Thrudb] * Thrudb is a set of simple services built on top of Facebook’s Thrift framework that provides indexing and document storage services for building and scaling websites. Its purpose is to offer web developers flexible, fast and easy-to-use services that can enhance or replace traditional data storage and access layers. * [http://diststore.com/ DistStore] * DistStore is a family of lightweight [http://developers.facebook.com/thrift Thrift] based web services which are extremely scalable in both throughput and dataset size. Its purpose is to offer a simple and flexible data storage solution that can grow with your project from inception to millions of users. == 【Reference】 == * [http://www.hypertable.org/ HyperTable 官方網站] * [http://my.donews.com/eraera/2006/09/26/swogzstwtqdnwlfrzgsljctkjsbrtuiumxzj/ 翻譯:Google大表(BigTable)] - [http://labs.google.com/papers/bigtable-osdi06.pdf 原文(PDF)] * [http://blogs.the451group.com/opensource/2008/02/19/introducing-hypertable-a-new-open-source-database-project/ Introducing Hypertable - a new open source database project] * [http://funp.com/push/?tag=Hypertable&hot&stars=1 開放原始碼專案 Hypertable（BigTable clone）的效率] * [http://www.linuxworld.com/podcasts/linux/2008/013008-linuxcast.html LinuxWorld Hypertable 專訪 Doug Judd (PodCast) - 包含如何使用 HyperTable 來匯入大量 Apache Log 的範例] * [http://code.google.com/p/hypertable/ hypertable 在 Google Code 的討論群] * [http://code.google.com/p/hypertable/wiki/HowHypertableWorks How Hypertable Works - Google Code 上的說明文件] * [http://code.google.com/p/hypertable/wiki/ApacheLogLoad Google Code 上的範例 Apache log load example] * [http://mmdays.com/2008/02/14/cloud-computing/ Cloud Computing 雲端運算] * [http://ieeexplore.ieee.org/iel3/4957/13637/00626582.pdf A Hypertabular Visualizer of Query Results] == 【Related News】 == * [http://news.sina.com.tw/tech/sinacn/cn/2008-02-19/154038229686.shtml 微軟收購雅虎存在諸多障礙] * [http://www.linuxworld.com/news/2008/020608-hypertable.html Zvents releases open-source cluster database] * [http://reddit.com/r/programming/info/67zfd/comments/ 關於Hypertable資料寫入檔案系統的問題] == 【Technology】 == * Hypertable Installation Guide: [http://trac.nchc.org.tw/cloud/wiki/HyperTable/HowToInstall How to install] * Hypertable Documentation Tree: [http://140.110.138.176/hypertable/html/inherits.html Source Code Documentation Tree](Doxygen) * Hypertable Query Language(HQL) Guide: [http://trac.nchc.org.tw/cloud/wiki/HyperTable/HQLTutorial HQL Hwo To]