Changes between Initial Version and Version 1 of III140503/Lab11

Timestamp:

May 4, 2014, 3:24:34 PM (11 years ago)

Author:

jazz

Comment:

--

III140503/Lab11

@@ +1 @@
+[[PageOutline]]
+
+◢ <[wiki:III140503/Lab10 實作十]> | <[wiki:III140503 回課程大綱]> ▲ | <[wiki:III140503/Lab12 實作十二]> ◣
+
+= 實作十一 Lab 11 =
+
+{{{
+#!html
+<div style="text-align: center;"><big style="font-weight: bold;"><big>練習豬的拉丁語<br/>Pig Latin in Practice</big></big></div>
+}}}
+
+{{{
+#!text
+以下練習，請連線至 hadoop.3du.me 操作。底下的 userXX 等於您的用戶名稱。
+}}}
+
+== Aggregation (Local Mode) ==
+
+{{{
+~$ wget http://hadoop.nchc.org.tw/excite-small.log
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ head lab8_out1/part-*
+}}}
+
+== Filter (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> STORE fltrd INTO 'lab8_out2';
+grunt> quit
+~$ head lab8_out2/part-*
+}}}
+
+== Sorting (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> srtd = ORDER fltrd BY cnt;
+grunt> STORE srtd INTO 'lab8_out3';
+grunt> quit
+~$ head lab8_out3/part-*
+}}}
+
+== Connect Pig to Hadoop (Full Distributed Mode) ==
+
+{{{
+~$ hadoop fs -put excite-small.log .
+~$ pig
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ hadoop fs -cat lab8_out1/part-00000
+}}}