Hadoop分佈式文件系統使用指南二

升級

• 由於換版本的話，資料夾內的conf設定檔也勢必被更改，因此目前作法為： 把conf 移至/opt/conf ，hadoop 0.16 與 hadoop 0.18用 ln 做捷徑代換。由於conf已不在hadoop_home內，因此記得匯入conf/hadoop-env.sh

  $ source /opt/conf/hadoop-env.sh
  

• 先看狀態

  $ bin/hadoop dfsadmin -upgradeProgress status
  
  There are no upgrades in progress.
  

• 停止hdfs
  • 注意不可使用bin/stop-all.sh來停止

    $ bin/stop-dfs.sh
    

• 部署新版本的Hadoop
  • 注意每個node的版本都要統一，否則會出現問題

• 啟動

  $ bin/start-dfs.sh -upgrade
  

• namenode管理網頁會出現升級狀態

退回

• 停止集群

  $ bin/stop-dfs.sh
  

• 部署老版本的Hadoop

• 退回之前版本

  $ bin/start-dfs.sh -rollback
  

檢查hdfs磁碟狀況

• fsck 可以

  $ bin/hadoop fsck /
  
  .
  /user/waue/input/1.txt:  Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s).
  /user/waue/input/1.txt:  Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s).
  .
  /user/waue/input/2.txt:  Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s).
  .
  /user/waue/input/3.txt:  Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s).
  .
  /user/waue/input/4.txt:  Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s).
  Status: HEALTHY
   Total size:	143451003 B
   Total dirs:	8
   Total files:	4
   Total blocks (validated):	5 (avg. block size 28690200 B)
   Minimally replicated blocks:	5 (100.0 %)
   Over-replicated blocks:	0 (0.0 %)
   Under-replicated blocks:	5 (100.0 %)
   Mis-replicated blocks:		0 (0.0 %)
   Default replication factor:	3
   Average block replication:	2.0
   Corrupt blocks:		0
   Missing replicas:		5 (50.0 %)
   Number of data-nodes:		2
   Number of racks:		1
  The filesystem under path '/' is HEALTHY
  

• 加不同的參數有不同的用處，如

  $ bin/hadoop fsck / -files
  
  
  /tmp <dir>
  
  /tmp/hadoop <dir>
  
  /tmp/hadoop/hadoop-waue <dir>
  
  /tmp/hadoop/hadoop-waue/mapred <dir>
  
  /tmp/hadoop/hadoop-waue/mapred/system <dir>
  
  /user <dir>
  
  /user/waue <dir>
  
  /user/waue/input <dir>
  
  /user/waue/input/1.txt 115045564 bytes, 2 block(s):  Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s).
  
   Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s).
  
  /user/waue/input/2.txt 987864 bytes, 1 block(s):  Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s).
  
  /user/waue/input/3.txt 1573048 bytes, 1 block(s):  Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s).
  
  /user/waue/input/4.txt 25844527 bytes, 1 block(s):  Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s).
  
  Status: HEALTHY
  
  ....（同上）