* !FuseAlert.java 100% finished and testing finished = 實驗一: 一個檔案, 共7行 = {{{ 1;538;15;NETBIOS SMB IPC$ unicode share access ;Generic Protocol Command ecode;3;09;04;17;53;56;168.150.177.165:1051;168.150.177.166:139;TCP; 1;538;15;NETBIOS SMB IPC$ unicode share access ;Generic Protocol Command Decode;3;10;04;20;53;56;168.150.177.165:1051;168.150.177.166:139;TCP; 1;1917;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;43;56;168.150.177.164:1032;239.255.255.250:1900;UDP; 1;1927;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;43;56;168.150.177.164:1032;239.255.255.250:1900;UDP; 1;1927;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;43;56;140.110.100.100:1032;239.255.255.250:1900;UDP; 1;1917;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;53;56;168.150.177.164:1032;239.255.255.250:1900;UDP; 1;1917;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;14;20;00;00;168.150.177.164:1032;239.255.255.250:1900;UDP; }}} to {{{ 1917;SCAN UPnP service discover attempt ;Detection of a Network Scan;3@@168.150.177.164@@239.255.255.250 904174356-914200000 1927;SCAN UPnP service discover attempt ;Detection of a Network Scan;3@@140.110.100.100@@239.255.255.250 904174356-904174356 1927;SCAN UPnP service discover attempt ;Detection of a Network Scan;3@@168.150.177.164@@239.255.255.250 904174356-904174356 538;NETBIOS SMB IPC$ unicode share access ;Generic Protocol Command Decode;3@@168.150.177.165@@168.150.177.166 904175356-1004205356 }}} * 單一FuseAlert 需 23秒左右 = 實驗二、六個檔案,共620186行 共42M = * 從正歸化到FuseAlert,化簡成 15150 行, 共需46.407秒 = 注意要點: = 1. the output is difference between Map and Reduce class ! {{{ #!java // set Map output key and value class job.setMapOutputKeyClass(Text.class); job.setMapOutputValueClass(LongWritable.class); // set Reduce output key and value class job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); }}} 2. 由於時間的位元長度過長,故20090101595959 已經超過 intWritable 的定址範圍 改為 0101595959 用 longWritable 宣告 3. 在map 與 reduce class內宣告重要參數,需用全名,如 int 改為 Integer 較佳 4. 也許因為key 過長,因此似乎影響到效能,此job 需run完需20secs