= Hadoop分佈式文件系統使用指南二 = = 升級 = * 由於換版本的話,資料夾內的conf設定檔也勢必被更改,因此目前作法為: 把conf 移至/opt/conf ,hadoop 0.16 與 hadoop 0.18用 ln 做捷徑代換。由於conf已不在hadoop_home內,因此記得匯入conf/hadoop-env.sh {{{ $ source /opt/conf/hadoop-env.sh }}} * 先看狀態 {{{ $ bin/hadoop dfsadmin -upgradeProgress status There are no upgrades in progress. }}} * 停止hdfs * 注意不可使用bin/stop-all.sh來停止 {{{ $ bin/stop-dfs.sh }}} * 部署新版本的Hadoop * 注意每個node的版本都要統一,否則會出現問題 * 啟動 {{{ $ bin/start-dfs.sh -upgrade }}} * namenode管理網頁會出現升級狀態 = 退回 = * 停止集群 {{{ $ bin/stop-dfs.sh }}} * 部署老版本的Hadoop * 退回之前版本 {{{ $ bin/start-dfs.sh -rollback }}} = 檢查hdfs磁碟狀況 = * fsck 可以 {{{ $ bin/hadoop fsck / . /user/waue/input/1.txt: Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). /user/waue/input/1.txt: Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). . /user/waue/input/2.txt: Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s). . /user/waue/input/3.txt: Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s). . /user/waue/input/4.txt: Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY Total size: 143451003 B Total dirs: 8 Total files: 4 Total blocks (validated): 5 (avg. block size 28690200 B) Minimally replicated blocks: 5 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 5 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 3 Average block replication: 2.0 Corrupt blocks: 0 Missing replicas: 5 (50.0 %) Number of data-nodes: 2 Number of racks: 1 The filesystem under path '/' is HEALTHY }}} * 加不同的參數有不同的用處,如 {{{ $ bin/hadoop fsck / -files /tmp /tmp/hadoop /tmp/hadoop/hadoop-waue /tmp/hadoop/hadoop-waue/mapred /tmp/hadoop/hadoop-waue/mapred/system /user /user/waue /user/waue/input /user/waue/input/1.txt 115045564 bytes, 2 block(s): Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). /user/waue/input/2.txt 987864 bytes, 1 block(s): Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s). /user/waue/input/3.txt 1573048 bytes, 1 block(s): Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s). /user/waue/input/4.txt 25844527 bytes, 1 block(s): Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s). Status: HEALTHY ....(同上) }}}