◢ <實作三> | <回課程大綱> ▲ | <實作五> ◣

實作四 Lab 4

HDFS 基本指令操作練習
HDFS in practice (full distributed mode)

Content 1: HDFS Shell 基本操作

Content 1: Basic HDFS Shell Commands

1.1 瀏覽你HDFS目錄

1.1 Browsing Your HDFS Folder

~$ hadoop fs -ls

1.2 上傳資料到 HDFS 目錄

1.2 Upload Files or Folder to HDFS

• 上傳 Upload

~$ hadoop fs -put ${HOME}/hadoop/conf toHDFS

• 檢查 Check

~$ hadoop fs -ls

1.3 下載 HDFS 的資料到本地目錄

1.3 Download HDFS Files or Folder to Local

• 下載 Download

~$ hadoop fs -get toHDFS fromHDFS

• 檢查 Check

  ~$ ls -al | grep fromHDFS
  ~$ diff ${HOME}/hadoop/conf fromHDFS/
  

1.4 刪除檔案

1.4 Remove Files or Folder

~$ hadoop fs -ls toHDFS/masters
~$ hadoop fs -rm toHDFS/masters

1.5 直接看檔案

1.5 Browse Files Directly

~$ hadoop fs -ls toHDFS/slaves
~$ hadoop fs -cat toHDFS/slaves

1.6 更多指令操作

1.6 More Commands -- Help message

~$ hadoop fs 

Usage: java FsShell
           [-ls <path>]
           [-lsr <path>]
           [-du <path>]
           [-dus <path>]
           [-count[-q] <path>]
           [-mv <src> <dst>]
           [-cp <src> <dst>]
           [-rm <path>]
           [-rmr <path>]
           [-expunge]
           [-put <localsrc> ... <dst>]
           [-copyFromLocal <localsrc> ... <dst>]
           [-moveFromLocal <localsrc> ... <dst>]
           [-get [-ignoreCrc] [-crc] <src> <localdst>]
           [-getmerge <src> <localdst> [addnl]]
           [-cat <src>]
           [-text <src>]
           [-copyToLocal [-ignoreCrc] [-crc] <src> <localdst>]
           [-moveToLocal [-crc] <src> <localdst>]
           [-mkdir <path>]
           [-setrep [-R] [-w] <rep> <path/file>]
           [-touchz <path>]
           [-test -[ezd] <path>]
           [-stat [format] <path>]
           [-tail [-f] <file>]
           [-chmod [-R] <MODE[,MODE]... | OCTALMODE> PATH...]
           [-chown [-R] [OWNER][:[GROUP]] PATH...]
           [-chgrp [-R] GROUP PATH...]
           [-help [cmd]]

Generic options supported are
-conf <configuration file>     specify an application configuration file
-D <property=value>            use value for given property
-fs <local|namenode:port>      specify a namenode
-jt <local|jobtracker:port>    specify a job tracker
-files <comma separated list of files>    specify comma separated files to be copied to the map reduce cluster
-libjars <comma separated list of jars>    specify comma separated jar files to include in the classpath.
-archives <comma separated list of archives>    specify comma separated archives to be unarchived on the compute machines.
The general command line syntax is
hadoop command [genericOptions] [commandOptions]

Content 2: 使用網頁 GUI 瀏覽資訊

Content 2: User Web GUI to browse HDFS

• 連上 ​http://nodeN.3du.me:50070 的 NameNode Web Interface ，請將 N 取代為您的報名序號

  w3m http://nodeN.3du.me:50070