close Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_fs.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.

Changes between Version 6 and Version 7 of waue/2009/1119


Ignore:
Timestamp:
Dec 3, 2009, 11:56:47 AM (16 years ago)
Author:
waue
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • waue/2009/1119

    v6 v7  
    7575 * Wireless Communications, Networking and Mobile Computing, 2009. WiCom '09. 5th International Conference on
    7676 
     77
     78= 11. 解決hdfs小檔的一些問題 =
     79 * hdfs 由於要支援海量的資料，因此每個block約64mb，然而小檔案會有一些效能瓶頸的問題
     80 * 對於如果有設定quota（ex: N=7,S=6G）的話，那如果超過個數或容量任一的話，就會造成錯誤，而mapreduce run到一半的話因為quota超過就會失敗
     81 * 本篇利用一個支援hadoop的壓縮方法"harballing"來使hdfs更好使用
     82 * 結果顯示，利用他們的方法模擬出可以減少 metadata 1/42的使用率
     83
     84