◢ <實作十> | <回課程大綱> ▲ | <實作十二> ◣

實作十一 Lab 11

練習豬的拉丁語
Pig Latin in Practice

以下練習，請連線至 hadoop.3du.me 操作。底下的 userXX 等於您的用戶名稱。

Aggregation (Local Mode)

~$ wget http://www.hadoop.tw/excite-small.log
~$ pig -x local
grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
grunt> grpd = GROUP log BY user;
grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
grunt> STORE cntd INTO 'lab8_out1';
grunt> quit
~$ head lab8_out1/part-*

Filter (Local Mode)

~$ pig -x local
grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
grunt> grpd = GROUP log BY user;
grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
grunt> fltrd = FILTER cntd BY cnt > 50;
grunt> STORE fltrd INTO 'lab8_out2';
grunt> quit
~$ head lab8_out2/part-*

Sorting (Local Mode)

~$ pig -x local
grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
grunt> grpd = GROUP log BY user;
grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
grunt> fltrd = FILTER cntd BY cnt > 50;
grunt> srtd = ORDER fltrd BY cnt;
grunt> STORE srtd INTO 'lab8_out3';
grunt> quit
~$ head lab8_out3/part-*

Connect Pig to Hadoop (Full Distributed Mode)

~$ hadoop fs -put excite-small.log .
~$ pig
grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
grunt> grpd = GROUP log BY user;
grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
grunt> STORE cntd INTO 'lab8_out1';
grunt> quit
~$ hadoop fs -cat lab8_out1/part-00000