◢ <實作三> | <回課程大綱> ▲ | <實作五> ◣

實作四 Lab 4

HDFS 叢集環境操作練習
HDFS full distributed mode in practice

以下練習，請連線至 https://lab.3du.me 操作。底下的 hXXXX 等於您的用戶名稱。

準備工作

• 由於 Google Cloud Engine 的 ssh 有些限制，目前先以 https 瀏覽器的連線方式進行實作
• 瀏覽器操作模式比較受限，目前已知 IE 會有問題，Firefox 會打不出 "-" 號，建議採用 Chrome 瀏覽器進行操作。
• 下載 ​Google Chrome
• 連線至 ​https://lab.3du.me (使用期限: 2014/03/23 00:00 ~ 2014/03/30 23:00)
• 備援入口: ​https://lab2.3du.me , ​https://lab3.3du.me
• 保障您自己使用的權利，請先修改密碼:

  user0@hdp:~$ passwd                                                           
  Changing password for user0.                                                  
  (current) UNIX password:                                                      
  Enter new UNIX password:                                                      
  Retype new UNIX password:
  

產生目錄

user0@hdp:~$ hadoop fs -ls                                                    
user0@hdp:~$ hadoop fs -mkdir tmp                                             
user0@hdp:~$ hadoop fs -ls                                                    
Found 1 items                                                                 
drwxr-xr-x   - user0 supergroup          0 2014-03-23 02:04 /user/user0/tmp   

產生一個 100 MB 的檔案

user0@hdp:~$ dd if=/dev/zero of=100mb.img bs=1M count=100                     
100+0 records in                                                              
100+0 records out                                                             
104857600 bytes (105 MB) copied, 0.0589096 s, 1.8 GB/s                        
user0@hdp:~$ hadoop fs -put 100mb.img test.img                                
user0@hdp:~$ hadoop fs -ls                                                    
Found 2 items                                                                 
-rw-r--r--   1 user0 supergroup  104857600 2014-03-23 02:08 /user/user0/test.img
drwxr-xr-x   - user0 supergroup          0 2014-03-23 02:04 /user/user0/tmp

觀察 block 分佈情形

user0@hdp:~$ hadoop fsck /user/user0/test.img -files -blocks -locations       
FSCK started by user0 from /10.240.1.123 for path /user/user0/test.img at Sun 
Mar 23 02:11:08 UTC 2014                                                      
/user/user0/test.img 104857600 bytes, 2 block(s):  OK                         
0. blk_-4422576463029178583_1044 len=67108864 repl=1 [10.240.1.123:50010]     
1. blk_7793285600029540175_1044 len=37748736 repl=1 [10.240.1.123:50010]      
                                                                              
Status: HEALTHY                                                               
 Total size:    104857600 B                                                   
 Total dirs:    0                                                             
 Total files:   1                                                             
 Total blocks (validated):      2 (avg. block size 52428800 B)                
 Minimally replicated blocks:   2 (100.0 %)                                   
 Over-replicated blocks:        0 (0.0 %)                                     
 Under-replicated blocks:       0 (0.0 %)                                     
 Mis-replicated blocks:         0 (0.0 %)                                     
 Default replication factor:    1                                             
 Average block replication:     1.0                                           
 Corrupt blocks:                0                                             
 Missing replicas:              0 (0.0 %)                                     
 Number of data-nodes:          9                                             
 Number of racks:               1                                             
FSCK ended at Sun Mar 23 02:11:08 UTC 2014 in 9 milliseconds                  
                                                                              
                                                                              
The filesystem under path '/user/user0/test.img' is HEALTHY                   

修改副本個數

user0@hdp:~$ hadoop fs -setrep 2 /user/user0/test.img                         
Replication 2 set: hdfs://hdp:9000/user/user0/test.img
user0@hdp:~$ hadoop fsck /user/user0/test.img -files -blocks -locations       
FSCK started by user0 from /10.240.1.123 for path /user/user0/test.img at Sun 
Mar 23 02:45:11 UTC 2014                                                      
/user/user0/test.img 104857600 bytes, 2 block(s):  OK                         
0. blk_-4422576463029178583_1044 len=67108864 repl=2 [10.240.1.123:50010, 10.2
40.174.73:50010]                                                              
1. blk_7793285600029540175_1044 len=37748736 repl=2 [10.240.1.123:50010, 10.24
0.116.151:50010]                                                              
                                                                              
Status: HEALTHY                                                               
 Total size:    104857600 B                                                   
 Total dirs:    0                                                             
 Total files:   1                                                             
 Total blocks (validated):      2 (avg. block size 52428800 B)                
 Minimally replicated blocks:   2 (100.0 %)                                   
 Over-replicated blocks:        0 (0.0 %)                                     
 Under-replicated blocks:       0 (0.0 %)                                     
 Mis-replicated blocks:         0 (0.0 %)                                     
 Default replication factor:    1                                             
 Average block replication:     2.0                                           
 Corrupt blocks:                0                                             
 Missing replicas:              0 (0.0 %)                                     
 Number of data-nodes:          9                                             
 Number of racks:               1                                             
FSCK ended at Sun Mar 23 02:45:11 UTC 2014 in 0 milliseconds                  
                                                                              
                                                                              
The filesystem under path '/user/user0/test.img' is HEALTHY