Changes between Initial Version and Version 1 of FDC110829/Lab7

Timestamp:

Aug 29, 2011, 11:22:50 PM (13 years ago)

Author:

jazz

Comment:

--

FDC110829/Lab7

@@ +1 @@
+[[PageOutline]]
+
+◢ <[wiki:FDC110829/Lab6 實作六]> | <[wiki:FDC110829 回課程大綱]> ▲ | <---> ◣
+
+= 實作七 Lab7 =
+
+{{{
+#!html
+<div style="text-align: center;"><big style="font-weight: bold;"><big>練習豬的拉丁語<br/>Pig Latin in Practice</big></big></div>
+}}}
+
+ * 首先使用您的帳號，連線至 hadoop.nchc.org.tw
+
+== Aggregation (Local Mode) ==
+
+{{{
+~$ wget http://hadoop.nchc.org.tw/excite-small.log
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ head lab8_out1
+}}}
+
+== Filter (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> STORE fltrd INTO 'lab8_out2';
+grunt> quit
+~$ head lab8_out2
+}}}
+
+== Sorting (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> srtd = ORDER fltrd BY cnt;
+grunt> STORE srtd INTO 'lab8_out3';
+grunt> quit
+~$ head lab8_out3
+}}}
+
+== Connect Pig to Hadoop (Full Distributed Mode) ==
+
+{{{
+~$ hadoop fs -put excite-small.log .
+~$ pig
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ hadoop fs -cat lab8_out1/part-00000
+}}}