46 | | = 檢查hdfs磁碟狀況 = |
47 | | * fsck 可以 |
48 | | {{{ |
49 | | $ bin/hadoop fsck / |
50 | | |
51 | | . |
52 | | /user/waue/input/1.txt: Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). |
53 | | /user/waue/input/1.txt: Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). |
54 | | . |
55 | | /user/waue/input/2.txt: Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s). |
56 | | . |
57 | | /user/waue/input/3.txt: Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s). |
58 | | . |
59 | | /user/waue/input/4.txt: Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s). |
60 | | Status: HEALTHY |
61 | | Total size: 143451003 B |
62 | | Total dirs: 8 |
63 | | Total files: 4 |
64 | | Total blocks (validated): 5 (avg. block size 28690200 B) |
65 | | Minimally replicated blocks: 5 (100.0 %) |
66 | | Over-replicated blocks: 0 (0.0 %) |
67 | | Under-replicated blocks: 5 (100.0 %) |
68 | | Mis-replicated blocks: 0 (0.0 %) |
69 | | Default replication factor: 3 |
70 | | Average block replication: 2.0 |
71 | | Corrupt blocks: 0 |
72 | | Missing replicas: 5 (50.0 %) |
73 | | Number of data-nodes: 2 |
74 | | Number of racks: 1 |
75 | | The filesystem under path '/' is HEALTHY |
76 | | }}} |
77 | | |
78 | | * 加不同的參數有不同的用處,如 |
79 | | {{{ |
80 | | $ bin/hadoop fsck / -files |
81 | | |
82 | | /tmp <dir> |
83 | | /tmp/hadoop <dir> |
84 | | /tmp/hadoop/hadoop-waue <dir> |
85 | | /tmp/hadoop/hadoop-waue/mapred <dir> |
86 | | /tmp/hadoop/hadoop-waue/mapred/system <dir> |
87 | | /user <dir> |
88 | | /user/waue <dir> |
89 | | /user/waue/input <dir> |
90 | | /user/waue/input/1.txt 115045564 bytes, 2 block(s): Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). |
91 | | Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). |
92 | | /user/waue/input/2.txt 987864 bytes, 1 block(s): Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s). |
93 | | /user/waue/input/3.txt 1573048 bytes, 1 block(s): Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s). |
94 | | /user/waue/input/4.txt 25844527 bytes, 1 block(s): Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s). |
95 | | Status: HEALTHY |
96 | | ....(同上) |
97 | | }}} |
| 100 | |
| 101 | == fsck == |
| 102 | * HDFS文件系統檢查工具 |
| 103 | |
| 104 | {{{ |
| 105 | $ bin/hadoop fsck / |
| 106 | |
| 107 | . |
| 108 | /user/waue/input/1.txt: Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). |
| 109 | /user/waue/input/1.txt: Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). |
| 110 | . |
| 111 | /user/waue/input/2.txt: Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s). |
| 112 | . |
| 113 | /user/waue/input/3.txt: Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s). |
| 114 | . |
| 115 | /user/waue/input/4.txt: Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s). |
| 116 | Status: HEALTHY |
| 117 | Total size: 143451003 B |
| 118 | Total dirs: 8 |
| 119 | Total files: 4 |
| 120 | Total blocks (validated): 5 (avg. block size 28690200 B) |
| 121 | Minimally replicated blocks: 5 (100.0 %) |
| 122 | Over-replicated blocks: 0 (0.0 %) |
| 123 | Under-replicated blocks: 5 (100.0 %) |
| 124 | Mis-replicated blocks: 0 (0.0 %) |
| 125 | Default replication factor: 3 |
| 126 | Average block replication: 2.0 |
| 127 | Corrupt blocks: 0 |
| 128 | Missing replicas: 5 (50.0 %) |
| 129 | Number of data-nodes: 2 |
| 130 | Number of racks: 1 |
| 131 | The filesystem under path '/' is HEALTHY |
| 132 | }}} |
| 133 | |
| 134 | * 加不同的參數有不同的用處,如 |
| 135 | {{{ |
| 136 | $ bin/hadoop fsck / -files |
| 137 | |
| 138 | /tmp <dir> |
| 139 | /tmp/hadoop <dir> |
| 140 | /tmp/hadoop/hadoop-waue <dir> |
| 141 | /tmp/hadoop/hadoop-waue/mapred <dir> |
| 142 | /tmp/hadoop/hadoop-waue/mapred/system <dir> |
| 143 | /user <dir> |
| 144 | /user/waue <dir> |
| 145 | /user/waue/input <dir> |
| 146 | /user/waue/input/1.txt 115045564 bytes, 2 block(s): Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). |
| 147 | Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). |
| 148 | /user/waue/input/2.txt 987864 bytes, 1 block(s): Under replicated blk_-2300843106107816641_1002. Target Replicas is 3 but found 2 replica(s). |
| 149 | /user/waue/input/3.txt 1573048 bytes, 1 block(s): Under replicated blk_-1561577350198661966_1003. Target Replicas is 3 but found 2 replica(s). |
| 150 | /user/waue/input/4.txt 25844527 bytes, 1 block(s): Under replicated blk_1316726598778579026_1004. Target Replicas is 3 but found 2 replica(s). |
| 151 | Status: HEALTHY |
| 152 | ....(同上) |
| 153 | }}} |
| 154 | |
| 155 | === -move === |
| 156 | * 移動受損文件到/lost+found |
| 157 | === -delete === |
| 158 | * 刪除受損文件。 |
| 159 | === -openforwrite === |
| 160 | * 印出寫打開的文件。 |
| 161 | |
| 162 | === -files === |
| 163 | * 印出正被檢查的文件 |
| 164 | === -blocks === |
| 165 | * 印出區塊的資訊 |
| 166 | === -locations === |
| 167 | * 印出每個區塊的位置 |
| 168 | === -racks === |
| 169 | * 印出data-node的網絡拓撲結構 |
| 170 | |
| 171 | 如: |
| 172 | |
| 173 | $ bin/hadoop fsck /user/waue/input/1.txt -files -blocks -locations |
| 174 | |
| 175 | /user/waue/input/1.txt 115045564 bytes, 2 block(s): Under replicated blk_-90085106852013388_1001. Target Replicas is 3 but found 2 replica(s). |
| 176 | |
| 177 | Under replicated blk_-4027196261436469955_1001. Target Replicas is 3 but found 2 replica(s). |
| 178 | |
| 179 | 0. blk_-90085106852013388_1001 len=67108864 repl=2 [140.110.138.191:50010, 140.110.141.129:50010] |
| 180 | |
| 181 | 1. blk_-4027196261436469955_1001 len=47936700 repl=2 [140.110.138.191:50010, 140.110.141.129:50010] |
| 182 | |
| 183 | |
| 184 | |
| 185 | Status: HEALTHY |
| 186 | |
| 187 | Total size: 115045564 B |
| 188 | |
| 189 | Total dirs: 0 |
| 190 | |
| 191 | Total files: 1 |
| 192 | |