NoSQL

• ​Using MySQL as a NoSQL - A story for exceeding 750,000 qps on a commodity server (2010-10-20)
• ​維基百科：NoSQL 分類
• ​Enterprise NoSQL: Silver Bullet or Poison Pill?
• ​Which open source NoSQL databases are you using?

Histroy

• 2009-07-01: ​No to SQL? Anti-database movement gains steam
  • NoSQL 活動 - 漸漸有一股勢力在推動不要用 SQL Database .... 資料庫真的退流行了嗎??!! (2009-07-16)
  • 我個人的觀點是任何典範的轉移並不會一蹴可及，馬上轉換，往往是潛移默化，慢慢地改變。 (2009-07-27)
• NoSQL 會不會歷史重演 OODBMS ?! - ​Why Aren't You Using An OODMS? (2010-07-25)
  • ​I can't wait for NoSql to die - ​NoSQL to die (中文評論)

    There was no great story on schema migration either. 
    

Definition

• ​http://en.wikipedia.org/wiki/NoSQL - 維基百科
• ​The NoSQL Alternative

• ​Amazon's Dynamo

• ​Scaling Rails Site：Reading Material # 1~ # 5 (2010-07-26)
• 2009-07-25 : ​Distributed Key-Value Database - gslin 介紹分散式 Key-Value Datastore 的一些背景觀念 (2010-07-26)
• 2009-11-06 : ​Key-Value 系統 分類整理 (NoSQL)

Primer

• NoSQL 的 A (ACID), B (BASE), C (CAP Theorem)
  • ​ACID
  • ​BASE: An Acid Alternative
  • ​CAP Theorem - ​Errors in Database Systems, Eventual Consistency, and the CAP Theorem
    • ​CAP theorem - Consistency (一致性), Availibility (可用性), Partition Tolerance (容錯性)三個勢必要犧牲其中一個！！

• ​Is NoSQL only for BigData? (2010-11-01) - 這篇文章提到 NoSQL 使用的三個適當時機，至少要滿足其中兩個。

      * Velocity - The data needs to be processed in a timely manner
      * Volume - The amount of data is large (this fits the traditional Big Data use case)
      * Variety - The data varies and is not a good fit for a fixed schema
  

Papers and Websites

• ​http://nosqlsummer.org/ - 一個關於 NoSQL 與分散式系統的論文閱讀俱樂部!!!!
• ​http://nosql-database.org/ - 介紹 NoSQL 分類的重要入口網站
• ​http://www.nosqldatabases.com/
• ​http://nosql.mypopescu.com/

• ​A Universal NoSQL Engine, Using a Tried and Tested Technology - 這篇文章蠻清楚地說明了不同的 NoSQL 資料庫型態在資料結構上的差異。
  • 
  • 
  • 
  • 
  • 
  • 
  • 
  • 

趨勢分析 Trend Observation

• ​從 Simply Hired 職缺趨勢來看 NoSQL 的需求量，可以看到 ​Amazon 的 SimpleDB 曾經一度很高。 2010 年 NoSQL 職缺排行榜：Cassandra > HBase > CouchDB =~ MongoDB > SimpleDB (2010-07-25)(2010-07-27)
• 
  

  HBase, Cassandra, MongoDB, CouchDB Job Trends

  Hbase jobs - Cassandra jobs - Mongodb jobs - Couchdb jobs

• ​比較 memcached, MongoDB, CouchDB, HBase, Apache Cassandra (2010-06-29)
• 

• 當然縱使 MongoDB 與 CouchDB 已經有很高的排行榜，跟 SQLite 比起來，還是差很多。 從 ​Google Trends 的搜尋趨勢可以看到 SQLite > Google Gears > CouchDB 的趨勢，象徵著 distributed database 的影響力尚未普及。 (2009-09-23)
  • 

• ​MongoDB, CouchDB, HBase, Cassandra, Hypertable 趨勢觀察 (2010-04-24)
  • 

• 從 ​2010 年 Google Trends 的搜尋趨勢可以看到 MongoDB 已經接近 Google Gears 了(當然跟 Google 宣布支援 HTML5 Local Database 有關). (2010-04-24)
  • 

Comparison

• ​NoSQL Solution: Evaluation Guide
  • 

• ​Structured storage (2010-07-26)

• ​NoSQL benchmarks and performance evaluations

• ​Which databases solve my problem? a survey of open source databases - ​<note> Survey of Open Source Database - ​http://ossdbsurvey.org/
• ​HBase vs. Cassandra: NoSQL Battle!

• 2010-07-04 : ​MongoDB, CouchDB, MySQL Compare Grid (2010-07-26)
• 2010-06-30 : ​A brief comparison of MongoDB and CouchDB (2010-07-26)
• 2010-06-12 : ​Nifty New Databases (2010-07-26)
• 2010-03-30 : ​CouchDB vs MongoDB Comparison (2010-07-26)
• 2010-02-25 : ​Datastore Comparison: High Performance Scalable Data Stores (PDF) - ​Paper: High Performance Scalable Data Stores (2010-07-26)

• 2010-01-05 : ​NoSql Databases – Part 1 - Landscape

Distributed	Not Distributed (responsibility on client)
* Amazon Dynamo * Amazon S3 * Scalaris * Voldemort * CouchDB (thru Lounge) * Riak * MongoDB (in alpha) * BigTable * Cassandra * HyperTable * HBase	* Redis * Tokyo Tyrant * MemcacheDB * Amazon SimpleDB

• 2009-11-25 : ​NoSQL数据库探讨之一 － 为什么要用非关系数据库？

• 2009-11-15 : ​NoSQL: scaling to size and scaling to complexity - 說明了 Key-Value Store, BigTable Clones (aka "Column Family") , Document Oriented, Graph Databases 在大小(Size)與複雜度上的關係。
  • 

• 2009-11-15 : ​MongoDB’s performance as compared to others
  • 

• 2009-11-09 : ​NoSQL Ecosystem (2010-07-25)
  • 
  • *

• 2009-10-21 : ​NoSQL Misconceptions - 關於 NoSQL 的錯誤觀念

• [影片] ​MapReduce vs MySQL (speaker Stu Hood) - Part 1 (2009-09-23)
• [影片] ​MapReduce vs MySQL (speaker Stu Hood) - Part 2 (2009-09-23)
• [影片] ​MapReduce vs MySQL (speaker Stu Hood) - Part 3 (2009-09-23)

Open Source Projects

• 2009-06-13: ​NOSQL debrief (2009-08-22)
• ​NOSQL - CouchDB (2009-08-11)
• ​NOSQL - Hypertable (2009-08-11)
• ​NOSQL - HBase (2009-08-11)

• 2009-01-19 : ​Anti-RDBMS: A list of distributed key-value stores - 列舉出一些分散式 key-value 資料庫 (2009-09-23)
  • 噗浪的​lightcloud。

• ​gears-dblib - A simple abstraction on top of the Database object in Gears (2009-09-23)
• ​orient - NoSQL document database light, portable and fast. Supports ACID Tx, Indexes, asynch queries, SQL layer, clustering, etc (2009-09-23)

• 2009-03-30 : ​non-RDBM distributed databases, map/reduce, key/value and cloud computing (2010-07-25)

NoSQL : HBase

• 11-04-08 : ​How to design a schema for Hbase

• 11:10–12:00 Upcoming improvements for HBase - Andrew Purtell (Trend Micro) (2010-04-24)
  • Big Data -> Medium Data 都需要
  • Cloud Computing - Scale Free
  • Disk Seek time remains nearly constant -> Index(B-Tree), Seek (RMDB) 慢！！
  • No distributed transactions, no complex locking, no waits or deadlocks
  • 不要用 Spreadsheet 的想法看待 HBase, 或許可以用 Tag 的想法去看待它。
  • HBase 跟 BigTable 都是 CP 架構(注重 Consistancy 與 Partition Tolerance，根據 CAP Theorem 因此無法確保 Avaibility，寧可服務中斷也要資料正確！！)
  • HDFS-200 (working append) 將在 HBase 0.20.5 加入支援資料持續遞增的功能。
  • ACID ? - ​atomicity, consistency, isolation, durability
  • 新功能：
    • 跨資料中心備份 - 透過 Log Ship ​HBASE-1295 : Multi data center replication
    • 安全性強化 - 支援 authentication, authorization，Yahoo! 寫了很多新的安全性支援，包括 Kerberos 認證、Data isolation at the HDFS layer、Secure RPC。因此必須新增角色來作存取控管(Access Control Role)
    • Coprocessor - 靈感來自於 BigTable 的新功能 Coprocessor，加入 RegionObservor (需要再花點時間看清楚用途！！)

• 最近 waue 在投影片中用到一張新的 MapReduce 圖，今天再次在 Andrew 的演講中看到，出處是 Lars George 的部落格『​HBase MapReduce 101 - Part I』
  • 
• ​FOSDEM 2010 NoSQL Talk
  • ​FOSDEM NoSQL session
• ​HBase vs. CouchDB in Berlin

• ​How glow.mozilla.org gets its data - Mozilla 用 HBase 來統計 Firefox 4 的即時下載數量

NoSQL : Cassandra

• ​Cassandra - a highly scalable, eventually consistent, distributed, structured key-value store. (2009-09-18)
  • ​Looking to the future with Cassandra
  • Cassandra was open sourced by Facebook in 2008
  • Cassandra brings together the distributed systems technologies from Dynamo and the data model from Google's BigTable.

• 09:00–09:50 nosql cassandra - Gasol (Pixnet) (2010-04-24)
  • ​http://cassandra.apache.org/
  • 具備副本機制，優先存在記憶體中，後續寫入 commit log 中。採取完全平等的分散式架構，沒有 Hadoop NameNode 單點失效問題(Single Point of Failure)

• 2010-03-02: ​4 Months with Cassandra, a love story - 用 Cassandra 的經驗分享 (2010-03-04)

• ​DevOpsDays LiveBlog – Cassandra and Puppet - 談用 Puppet 佈署 Cassandra 叢集

Use Case

• 2010-07-09 : ​Cassandra at Twitter Today (2010-07-30)
• 2010-03-07 : ​Saying Yes to NoSQL; Going Steady with Cassandra - Digg 決定繼續使用 Cassandra
• 2010-02-23 : ​Cassandra @ Twitter: An Interview with Ryan King - Twitter 很早就想用 Cassandra
• 2009-07-02 : ​up and running with cassandra (內容於 2010-05-11 更新)
• 2009-09-09 : ​Looking to the future with Cassandra - Digg 使用 Cassandra

  After considering HBase, Hypertable, Cassandra, Tokyo Cabinet/Tyrant, Voldemort, and Dynomite, we settled on Cassandra.
  

Installation

• ​Cassandra Debian 套件 (2010-07-25)
• ​Consistency (中文) (2010-07-25)

• 2010-05-30 : ​Cassandra Jump Start For The Windows Developer (2010-07-27)
• 2010-05-24 : ​How to install NOSQL cassandra DB in ubuntu and debian via ppa repository (2010-07-27)

NoSQL : MongoDB

• 2011-03-28 : ​The Little MongoDB Book
  • ​PDF 檔
• 2010-02-28 : ​Notes from a production MongoDB deployment - 大型 MongoDB 不數的建議
• 2009-07-25 : ​Choosing a non-relational database; why we migrated from MySQL to MongoDB

• 2009-11-19 : ​Getting Non-Relational with MongoDB

NoSQL : CouchDB

• ​CouchDB (2009-09-23)
  • ​Ubuntu couchdb 套件 (2009-09-23)
  • ​Debian couchdb 套件 (2009-09-23)
  • ​Modeling Entity Relationships in CouchDB (2009-09-23)
  • ​couchdb-fuse - CouchDB FUSE File System (2009-09-23)
  • [影片] ​Next Generation Data Storage with CouchDB (speaker: Jan Lehnardt) - Part 1 (2009-09-23)
  • [影片] ​Next Generation Data Storage with CouchDB (speaker: Jan Lehnardt) - Part 2 (2009-09-23)
  • 在看維基百科的時候，發現 CouchDB 同時被分屬在 ​Column-oriented DBMS 跟 ​Document-oriented database (2009-09-23)
  • ​Interactive CouchDB - 這裡用 Java Script 示範了 CouchDB 結合不同用途的 MapReduce 實作。
• ​dot.Cloud - an open-source cloud federation platform. - 從 Feature 看起來頗 Powerful (2009-09-23)
  • Keep your servers under revision control
  • Stop worrying about maintaining state: just create and kill instances
  • Use tools you know: rsync, mercurial/git, ssh
  • Push a small upgrade to your images without moving gigabytes around
  • Replicate multi-server setups in just one command
  • Cleanly separate data (DB, logs, content) and code (OS, libraries, binaries, configuration)
  • Map data volumes to any available storage (NAS, EBS, S3)
  • Save bandwidth by delivering your app closer to the consumer

• 2009-11-04 : ​Benchmarking CouchDB with Baracus

NoSQL : Voldemort

• ​Voldemort - a distributed key-value storage system (2009-09-18)
  • used at LinkedIn for certain high-scalability storage problems

NoSQL : Redis

• ​redis - A persistent key-value database with built-in net interface written in ANSI-C for Posix systems
  • ​Redis: Lightweight key/value Store That Goes the Extra Mile - 另一個輕量級的 Key/Value? 資料庫 (2009-09-02)

NoSQL : Neo4j

• Graph database - 用來儲存關係的 Graph
• ​http://neo4j.org/
• Neo4j has been in commercial development for 10 years and in production for over 7 years. - 哇！還開發蠻久了耶～
• 目前看到生物資訊的資料庫有用它 - ​http://bio4j.com/ - Bio4j is a bioinformatics graph based DB including most data available in UniProt? (SwissProt? + Trembl) plus Gene Ontology (GO) and UniRef? (50,90,100).

NoSQL : Trinity

• ​Trinity - a graph database and computation platform over distributed memory cloud. 由微軟開發的 Graph Database。
• 文中提到 Trinity 與 Neo4j, HyperGraphDB, InfiniteGraph, 以及 Google 的 ​Pregel(2010-10-20)

Use Case