Changes between Initial Version and Version 1 of YMU110509/Lab8

Timestamp:

May 9, 2011, 11:31:29 AM (14 years ago)

Author:

jazz

Comment:

--

YMU110509/Lab8

@@ +1 @@
+[[PageOutline]]
+
+◢ <[wiki:YMU110509/Lab7 實作七]> | <[wiki:YMU110509 回課程大綱]> ▲ | <---> ◣
+
+= 實作八 Lab8 =
+
+{{{
+#!html
+<div style="text-align: center;"><big style="font-weight: bold;"><big>練習豬的拉丁語<br/>Pig Latin in Practice</big></big></div>
+}}}
+
+== Aggregation (Local Mode) ==
+
+{{{
+~$ wget http://hadoop.nchc.org.tw/excite-small.log
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ head lab8_out1
+}}}
+
+== Filter (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> STORE fltrd INTO 'lab8_out2';
+grunt> quit
+~$ head lab8_out2
+}}}
+
+== Sorting (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> srtd = ORDER fltrd BY cnt;
+grunt> STORE srtd INTO 'lab8_out3';
+grunt> quit
+~$ head lab8_out3
+}}}
+
+== Connect Pig to Hadoop (Full Distributed Mode) ==
+
+{{{
+~$ hadoop fs -put excite-small.log .
+~$ pig
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ hadoop fs -cat lab8_out1/part-00000
+}}}