wiki:waue/2009/1028
  • FuseAlert.java 100% finished and testing finished

實驗一: 一個檔案, 共7行

1;538;15;NETBIOS SMB IPC$ unicode share access ;Generic Protocol Command ecode;3;09;04;17;53;56;168.150.177.165:1051;168.150.177.166:139;TCP;
1;538;15;NETBIOS SMB IPC$ unicode share access ;Generic Protocol Command Decode;3;10;04;20;53;56;168.150.177.165:1051;168.150.177.166:139;TCP;
1;1917;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;43;56;168.150.177.164:1032;239.255.255.250:1900;UDP;
1;1927;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;43;56;168.150.177.164:1032;239.255.255.250:1900;UDP;
1;1927;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;43;56;140.110.100.100:1032;239.255.255.250:1900;UDP;
1;1917;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;04;17;53;56;168.150.177.164:1032;239.255.255.250:1900;UDP;
1;1917;6;SCAN UPnP service discover attempt ;Detection of a Network Scan;3;09;14;20;00;00;168.150.177.164:1032;239.255.255.250:1900;UDP;

to

1917;SCAN UPnP service discover attempt ;Detection of a Network Scan;3@@168.150.177.164@@239.255.255.250	904174356-914200000
1927;SCAN UPnP service discover attempt ;Detection of a Network Scan;3@@140.110.100.100@@239.255.255.250	904174356-904174356
1927;SCAN UPnP service discover attempt ;Detection of a Network Scan;3@@168.150.177.164@@239.255.255.250	904174356-904174356
538;NETBIOS SMB IPC$ unicode share access ;Generic Protocol Command Decode;3@@168.150.177.165@@168.150.177.166	904175356-1004205356
  • 單一FuseAlert 需 23秒左右

實驗二、六個檔案,共620186行 共42M

  • 從正歸化到FuseAlert,化簡成 15150 行, 共需46.407秒

注意要點:

  1. the output is difference between Map and Reduce class !
        
        // set Map output key and value class
        job.setMapOutputKeyClass(Text.class);
        job.setMapOutputValueClass(LongWritable.class);
        // set Reduce output key and value class
        job.setOutputKeyClass(Text.class);
        job.setOutputValueClass(Text.class);
    
  1. 由於時間的位元長度過長,故20090101595959 已經超過 intWritable 的定址範圍

改為 0101595959 用 longWritable 宣告

  1. 在map 與 reduce class內宣告重要參數,需用全名,如 int 改為 Integer 較佳
  1. 也許因為key 過長,因此似乎影響到效能,此job 需run完需20secs
Last modified 11 years ago Last modified on Oct 29, 2009, 9:49:40 AM