= 2010-11-24 =

== Hadoop & GPFS ==

 * [http://www.enterprisestorageforum.com/technology/news/article.php/3913996/IBM-Builds-on-Hadoop-with-New-Storage-Architecture.htm IBM Builds on Hadoop with New Storage Architecture] - IBM 讓 GPFS 可以跟 HDFS 合作，提供更高可用度的儲存服務。稱之為『General Parallel File System-Shared Nothing Cluster (GPFS-SNC)』
 * [http://www.zdnet.com.tw/news/hardware/0,2000085676,20148269,00.htm IBM發表新儲存架構 GPFS-SNC]
   * [http://www.almaden.ibm.com/storagesystems/projects/hadoop/ IBM 研究中心的專案：Hadoop on GPFS]

== Public Large Data Set 公用大型資料集 ==

 * 維基百科 - http://en.wikipedia.org/wiki/Wikipedia:Database_download
 * 亞馬遜提供的公用資料 - http://aws.amazon.com/publicdatasets/
   * 包括基因資料(Genome, Ex. 1000 Genome Project)
 * http://www.statmt.org/europarl/
 * http://www.opendatacenteralliance.org/
 * [http://www.data.gov/raw/92 Data.gov] - 美國公部門的資料 - 台灣應該可以上 [http://www.archives.gov.tw/ 檔案管理局] ([wiki:jazz/10-11-08 2010-11-08])
 * http://ckan.org/
 * http://stat-computing.org/dataexpo/2009/ - 飛航紀錄 - 1987-2008 years of airlines performance data. 1GB, 12M records.
 * http://kdd.ics.uci.edu/ - UCI Knowledge Discovery in Databases Archive - The data sets are categorized neatly. These are very useful for many machine-learning exercises with the use of Hadoop / Mahout.
 * A list of about 70 data sets compiled for Open Data Day: http://www.opendataday.org/wiki/Data
 * A list of 70 datasets from technology review: https://www.technologyreview.com/blog/arxiv/26097/
 * Stanford large network database collection: http://snap.stanford.edu/data/index.html
 * Digging into data dataset list: http://www.diggingintodata.org/Repositories/tabid/167/Default.aspx
 * Many Eyes from IBM: http://www-958.ibm.com/software/data/cognos/manyeyes/
 * Data360: http://www.data360.org/index.aspx
 * UN Data Explorer: http://data.un.org/Explorer.aspx
 * OECD Data Sets: http://stats.oecd.org/index.aspx
 * Freebase data dumps: http://wiki.freebase.com/wiki/Data_dumps 
 * http://infochimps.org - They have over a billion tweets. 
 * http://snap.stanford.edu/data/ - 史丹佛大學的一些資料集
 * http://data.stackexchange.com/
 * http://ogdisdk.cloudapp.net/ - Open Government Data Initiative (OGDI)
 * http://data.worldbank.org/ - 世界銀行的一些資料集
 * [http://creativecommons.org.tw/blog/archives/000237.html 公部門資訊的自由取得與再次使用]
 * http://opengovernmentdata.org/
 * http://www.opengovdata.org/
 * http://data.gov.uk/
 * http://OpenFlights.org - a crowd-sourced database of airlines, flights, airports etc. 
 * ([wiki:jazz/11-09-13 2011-09-13])
  * [http://www.jtv.com.tw/AppStar/ App Star 高手爭霸戰]
  * [http://www.opendata.tw/open-data/tpe-od-platform/ 臺北市政府 vs Data.Taipei]
  * http://taipeiogdi.cloudapp.net/DataCatalog/DataSetList - 台北市公開政府資料平臺-開發人員專區資料服務(OGDI)
   * cloudapp.net 這個網域是 Microsoft Azure 平台在用的～看樣子台北市政府的開放平台是改自 http://ogdisdk.cloudapp.net/
 * (2012-01-17) [http://www.findbestopensource.com/article-detail/free-large-data-corpus 10 sites to get the large data set or data corpus for free]