Version 3 (modified by jazz, 12 years ago) (diff) |
---|
◢ <實作十> | <回課程大綱> ▲ | <實作十二> ◣
實作十一 Lab 11
練習豬的拉丁語
Pig Latin in Practice
Pig Latin in Practice
以下練習,請連線至 hadoop.nchc.org.tw 操作。底下的 hXXXX 等於您的用戶名稱。
Aggregation (Local Mode)
~$ wget http://hadoop.nchc.org.tw/excite-small.log ~$ pig -x local grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query); grunt> grpd = GROUP log BY user; grunt> cntd = FOREACH grpd GENERATE group, COUNT(log); grunt> STORE cntd INTO 'lab8_out1'; grunt> quit ~$ head lab8_out1
Filter (Local Mode)
~$ pig -x local grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query); grunt> grpd = GROUP log BY user; grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt; grunt> fltrd = FILTER cntd BY cnt > 50; grunt> STORE fltrd INTO 'lab8_out2'; grunt> quit ~$ head lab8_out2
Sorting (Local Mode)
~$ pig -x local grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query); grunt> grpd = GROUP log BY user; grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt; grunt> fltrd = FILTER cntd BY cnt > 50; grunt> srtd = ORDER fltrd BY cnt; grunt> STORE srtd INTO 'lab8_out3'; grunt> quit ~$ head lab8_out3
Connect Pig to Hadoop (Full Distributed Mode)
~$ hadoop fs -put excite-small.log . ~$ pig grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query); grunt> grpd = GROUP log BY user; grunt> cntd = FOREACH grpd GENERATE group, COUNT(log); grunt> STORE cntd INTO 'lab8_out1'; grunt> quit ~$ hadoop fs -cat lab8_out1/part-00000