◢ <實作三> | <回課程大綱> ▲ | <實作五> ◣

實作四 Lab 4

HDFS 叢集環境操作練習
HDFS full distributed mode in practice

以下練習，請連線至 hadoop.3du.me 操作。底下的 userXX 等於您的用戶名稱。

產生目錄

user0@hdp:~$ hadoop fs -ls                                                    
user0@hdp:~$ hadoop fs -mkdir tmp                                             
user0@hdp:~$ hadoop fs -ls                                                    
Found 1 items                                                                 
drwxr-xr-x   - user0 supergroup          0 2014-03-23 02:04 /user/user0/tmp   

產生一個 100 MB 的檔案

user0@hdp:~$ dd if=/dev/zero of=100mb.img bs=1M count=100                     
100+0 records in                                                              
100+0 records out                                                             
104857600 bytes (105 MB) copied, 0.0589096 s, 1.8 GB/s                        
user0@hdp:~$ hadoop fs -put 100mb.img test.img                                
user0@hdp:~$ hadoop fs -ls                                                    
Found 2 items                                                                 
-rw-r--r--   1 user0 supergroup  104857600 2014-03-23 02:08 /user/user0/test.img
drwxr-xr-x   - user0 supergroup          0 2014-03-23 02:04 /user/user0/tmp

觀察 block 分佈情形

user0@hdp:~$ hadoop fsck /user/$USER/test.img -files -blocks -locations       
FSCK started by user0 from /10.240.1.123 for path /user/$USER/test.img at Sun 
Mar 23 02:11:08 UTC 2014                                                      
/user/user0/test.img 104857600 bytes, 2 block(s):  OK                         
0. blk_-4422576463029178583_1044 len=67108864 repl=1 [10.240.1.123:50010]     
1. blk_7793285600029540175_1044 len=37748736 repl=1 [10.240.1.123:50010]      
                                                                              
Status: HEALTHY                                                               
 Total size:    104857600 B                                                   
 Total dirs:    0                                                             
 Total files:   1                                                             
 Total blocks (validated):      2 (avg. block size 52428800 B)                
 Minimally replicated blocks:   2 (100.0 %)                                   
 Over-replicated blocks:        0 (0.0 %)                                     
 Under-replicated blocks:       0 (0.0 %)                                     
 Mis-replicated blocks:         0 (0.0 %)                                     
 Default replication factor:    1                                             
 Average block replication:     1.0                                           
 Corrupt blocks:                0                                             
 Missing replicas:              0 (0.0 %)                                     
 Number of data-nodes:          9                                             
 Number of racks:               1                                             
FSCK ended at Sun Mar 23 02:11:08 UTC 2014 in 9 milliseconds                  
                                                                              
                                                                              
The filesystem under path '/user/$USER/test.img' is HEALTHY                   

修改副本個數

user0@hdp:~$ hadoop fs -setrep 2 /user/$USER/test.img                         
Replication 2 set: hdfs://hdp:9000/user/$USER/test.img
user0@hdp:~$ hadoop fsck /user/$USER/test.img -files -blocks -locations       
FSCK started by user0 from /10.240.1.123 for path /user/$USER/test.img at Sun 
Mar 23 02:45:11 UTC 2014                                                      
/user/$USER/test.img 104857600 bytes, 2 block(s):  OK                         
0. blk_-4422576463029178583_1044 len=67108864 repl=2 [10.240.1.123:50010, 10.2
40.174.73:50010]                                                              
1. blk_7793285600029540175_1044 len=37748736 repl=2 [10.240.1.123:50010, 10.24
0.116.151:50010]                                                              
                                                                              
Status: HEALTHY                                                               
 Total size:    104857600 B                                                   
 Total dirs:    0                                                             
 Total files:   1                                                             
 Total blocks (validated):      2 (avg. block size 52428800 B)                
 Minimally replicated blocks:   2 (100.0 %)                                   
 Over-replicated blocks:        0 (0.0 %)                                     
 Under-replicated blocks:       0 (0.0 %)                                     
 Mis-replicated blocks:         0 (0.0 %)                                     
 Default replication factor:    1                                             
 Average block replication:     2.0                                           
 Corrupt blocks:                0                                             
 Missing replicas:              0 (0.0 %)                                     
 Number of data-nodes:          9                                             
 Number of racks:               1                                             
FSCK ended at Sun Mar 23 02:45:11 UTC 2014 in 0 milliseconds                  
                                                                              
                                                                              
The filesystem under path '/user/$USER/test.img' is HEALTHY