wiki:icasIII

Version 48 (modified by waue, 13 years ago) (diff)

--

ICAS III
安裝說明

Change Log

  • 0627
    • crawlweb2
    • 測試 install.sh 無誤
  • 0701
    • upload.php 即時運算
    • classify bug fix
    • blacklist 製作
  • 0704
    • priority 修正
    • log 紀錄檔
  • 0705
    • 整理圖功能
    • FTP 設定
    • 備份
    • Install 更新
  • 0714
    • 修改 bug
  • 0811
    • 支援 xml格式的設定檔
    • mail 通知管理者

必要目錄

  • bin
  • hadoop
  • www
  • report
    • black
    • dot
    • svg
  • log_input
    • idp8200
    • nk7admin
    • snort
  • workspace [可為空]

資料流程
GraphViz image

讀取設定檔

/opt/icas/icas_setup.xml

<?xml version="1.0" encoding="UTF-8"?>
<icas>
  <mail>
    <sender>waue@nchc.org.tw</sender>
    <receiver>waue0920@gmail.com, waue@nchc.narl.org.tw</receiver>
    <subject>ICAS Alarm</subject>
    <host>localhost</host>
    <protocal>smtp</protocal>
  </mail>
  <logpath>/opt/icas/icas.log</logpath>
</icas>

一、DataCheck

  • 檢查各資料夾內是否有資料
  • 有資料則依各入侵偵測格式進行正規化
  • 輸出

1 snort

[**] [1:2189:3] BAD-TRAFFIC IP Proto 103 PIM [**]
[Classification: Detection of a non-standard protocol or event] [Priority: 2] 
05/17-08:30:14.750704 140.110.138.253 -> 224.0.0.13
PIM TTL:1 TOS:0xC0 ID:4076 IpLen:20 DgmLen:58
[Xref => http://cve.mitre.org/cgi-bin/cvename.cgi?name=2003-0567][Xref => http://www.securityfocus.com/bid/8211]
  • 呼叫 Classify 作分類簡化
	String[][] class_str = { { // 1 Detection
			"Detection of a Network Scan", "Decode of an RPC Query",
					"A client was using an unusual port",
					"Detection of a non-standard protocol or event" },
			// 2 attempt info
			{
					"Attempted Information Leak",
					"Information Leak",
					"Large Scale Information Leak",
					"Attempted Denial of Service",
					"Attempted User Privilege Gain",
					"Attempted Administrator Privilege Gain",
					"An attempted login using a suspicious username was detected",
					"Attempt to login by a default username and password",
					"Unsuccessful User Privilege Gain" },

			// 3 user gain
			{ "Successful User Privilege Gain" },
			// 4 admin gain
			{ "Successful Administrator Privilege Gain" },

			// 5 attack
			{ "Misc Attack", "A suspicious filename was detected",
					"A system call was detected",
					"Executable code was detected", "SCORE! Get the lotion!",
					"access to a potentially vulnerable web application",
					"Web Application Attack",
					"Potential Corporate Privacy Violation" },
			// 6 dos
			{ "Denial of Service", "Detection of a Denial of Service Attack" },
			// 7 Trojan
			{ "A Network Trojan was detected" },
			// 8 Info
			{ "Not Suspicious Traffic", "Unknown Traffic",
					"Potentially Bad Traffic",
					"A suspicious string was detected",
					"Generic Protocol Command Decode", "Generic ICMP event",
					"Misc activity", "A TCP connection was detected" } };

先轉化成單筆警訊一行 =>

  • 格式
    ids編號;sid;說明 ;分類名稱;嚴重性;月;日;時;分;秒;  來源:埠  ; 目的:埠  ; 協定;
    
  • 範例
    1;2189;BAD-TRAFFIC IP Proto 103 PIM ;1;2;20110728;160514;140.110.134.253;224.0.0.13;0;
    

2 Idp8200

  • 設定編號為
  • 原始格式
    Time Received ## Src Addr ## Dst Addr ## Action ## Protocol ## Dst Port ## interface ## Description ## Severity
    
  • 範例
    2003/8/11 13:05,140.113.130.221,0.0.0.0,Accepted,TCP,65432,'interface=eth2',FTP: Format String in Command,Major
    2003/8/11 13:05,140.113.130.221,phe96.sro.nchc.org.tw,Accepted,TCP,65432,'interface=eth2',FTP: Format String in Command,Major
    

3 NK7Admin

  • 原始格式
    編號 ## 名稱 ## 來源位址 ## 目的位址 ##  開始時間 ##  總數 ## 來源埠 ##  目的埠
    
  • 範例
    1,TCP SYN,60.173.26.116,140.110.127.253,2011/3/1 14:41,1,6000,9415
    2,UDP PORT SCAN,168.95.1.1,140.110.104.84,2011/3/1 14:41,1,53,34953
    

輸出格式

  • icas III 正規化後格式
0 ids來源; 1 警訊識別id ; 2 攻擊說明 ; 3 分類 ; 4 嚴重性(1最嚴重~3普通) ; 5 年月日 ; 6 時分秒 ; 7 來源ip ; 8 目標ip ; 9 目標port
  • class (分類資訊) 與 sig_id (特徵碼id 編號) 是兩台硬體 ids 沒有的資訊,會補 0 顯示
  • 嚴重性 = 1~3, 1最嚴重
  • 偵測裝置編號 : 1=snort, 2=idp8200, 3=nk7admin

二、 IntegrateAlert

警訊整合

  • 於 hadoop 上運作
  • 將正規化的資料當輸入
  • 運算結果格式
0 攻擊者ip -> 目標ip 1 嚴重性(1~3, 1最嚴重) 2 開始日期_時間點~結束日期_時間點 3 [分類資訊,...] 4 [sig_id,...] 5 [攻擊說明1,攻擊說明2,...] 6 [目標port1,目標port2, ...] 7 [偵測裝置編號,...] 8 "整合總筆數"-"整合分類筆數"-"整合sig_id編號筆數"

map output

key : src_ip - dst_ip
val : date @@ time @@ class_id @@ ids @@ s-id @@ priority @@ port @@ description

reduce output

key: src_ip -> des_ip
values: priority @@ t1~tn @@ [class,...] @@ [sig_id,...] @@ [attact,...] @@
        [port,...] @@ [ids,...] @@
        "tatal_count"-"class_count"-"sig_id_count"

sample

  • input
1;0;FTP: Format String in Command;no;1;2003811;130500;140.113.130.221;0.0.0.0;65432;
2;0;FTP: Format String in Command;no;1;2003811;130500;140.113.130.221;0.0.0.0;65432;
3;1;FTP: Format String in Command;no;1;2003811;130500;140.113.130.222;10.10.0.2;65432;
1;2;FTP: Format String in ;no;1;2003811;150500;140.113.130.221;phe96.sro.nchc.org.tw;65432;
2;2;FTP: Format String ;no;1;2003811;160500;140.113.130.221;phe96.sro.nchc.org.tw;65432;
3;1;FTP: Format ;no;1;2003811;130500;140.113.130.221;phe96.sro.nchc.org.tw;65432;
  • result
140.113.130.221->0.0.0.0  @@1@@2003811_130500~2003811_130500@@[0]@@[0]@@[FTP: Format String in Command]@@[65432]@@[1, 2]@@2-1-1
140.113.130.221->phe96.sro.nchc.org.tw  @@1@@2003811_130500~2003811_160500@@[0]@@[2, 1]@@[FTP: Format String in , FTP: Format String , FTP: Format ]@@[65432]@@[1, 2, 3]@@3-1-2
140.113.130.222->10.10.0.2  @@1@@2003811_130500~2003811_130500@@[0]@@[1]@@[FTP: Format String in Command]@@[65432]@@[3]@@1-1-1

三、 Aggreggate

  • 將運算結果從 hdfs 下載到 local
  • dot -> 繪圖-> svg 攻擊圖
  • 黑名單
  • 警訊列表

DotGraph

繪圖

digraph G { size ="8,0"; node[style=filled,peripheries=2,color="lightskyblue"]; 
{"140.113.130.221"}->{"0.0.0.0"}[color=red, label="NIDS \n \n[FTP: Format String in Command]"];
{"140.113.130.221"}->{"phe96.sro.nchc.org.tw"}[color=red, label="NIDS \n \n[FTP: Format String in Command]"];
{"168.150.177.164"}->{"239.255.255.250"}[color=red, label="NIDS \n \n[SCAN UPnP service discover attempt ]"];
{"168.150.177.165"}->{"168.150.177.166"}[color=red, label="NIDS \n \n[NETBIOS SMB IPC$ unicode share access ]"];
{"168.95.1.1"}->{"140.110.104.84"}[color=red, label="NIDS \n \n[UDP PORT SCAN]"];
{"60.173.26.116"}->{"140.110.127.253"}[color=red, label="NIDS \n \n[TCP SYN]"];
}
GraphViz image

demo.svg

list

  • 列表格式
[ "來源IP " , "目標IP " , "起始時間" , "結束時間" , "嚴重性" , "IDS", "總數" , "說明"]

blacklist

輸入:

0 @@ 1 @@ 2 @@ 3 @@ 4 @@ 5 @@ 6 @@ 7 @@ 8 @@
77.68.104.162->140.110.134.198 priority time~time [class] [sid] [detail ] [ port ] [ids] count

輸出結果: sip @@ dip @@ port @@ time @@ detail @@ ids @@ count

mail 通知

Attack List src ip dst ip prio time range detail ids count dst port list 140.110.114.141 74.125.71.103 1 20110812_085900~20110812_085900 WORM Conficker on HTTP Search nk7admin 1 80 140.110.114.141 74.125.71.104 1 20110812_085900~20110812_085900 WORM Conficker on HTTP Search nk7admin 1 80 140.110.116.253 74.125.153.100 1 20110812_085700~20110812_085800 WORM Conficker on HTTP Search nk7admin 9 80 140.110.134.253 224.0.0.13 2 20110812_081507~20110812_091438 BAD-TRAFFIC IP Proto 103 PIM snort 122 0 199.93.56.125 140.110.112.4 1 20110812_080600~20110812_080600 EXPLOIT RealNetworks RealPlayer FLV Parsing Two integer overflow vulnerabilities nk7admin 1 60215 199.93.56.125 140.110.114.136 1 20110812_084500~20110812_084500 EXPLOIT RealNetworks RealPlayer FLV Parsing Two integer overflow vulnerabilities nk7admin 1 49221 199.93.56.125 140.110.116.253 1 20110812_083200~20110812_083200 EXPLOIT RealNetworks RealPlayer FLV Parsing Two integer overflow vulnerabilities nk7admin 1 59570 218.10.246.123 140.110.127.250 1 20110812_081800~20110812_081800 TCP SYN nk7admin 1 1433 61.147.112.50 140.110.112.78 1 20110812_080300~20110812_080300 TCP SYN nk7admin 1 1433 61.147.112.50 140.110.127.255 1 20110812_080300~20110812_080300 TCP SYN nk7admin 1 1433 67.195.19.74 140.110.117.190 1 20110812_085500~20110812_085500 EXPLOIT Microsoft Color Management Module Buffer Overflow nk7admin 1 2549 Graph Show

範例

111.165.17.16->phe96.sro.nchc.org.tw  @@1@@20030811_121000~20030811_121000@@[0]@@[0]@@[FTP: Format String in Command]@@[65432,555]@@[1,2,3]@@222-1-1
112.78.196.214->140.110.110.4 @@1@@20030811_123000~20030811_123000@@[0]@@[0]@@[SSH: Pragma Fortress Key OverFlow]@@[22,999]@@[1]@@44-1-1
112.78.196.214->140.110.111.11  @@2@@20030811_121800~20030811_121800@@[0]@@[0]@@[SSH: Pragma Fortress Key OverFlow]@@[22]@@[2]@@1-1-1
112.78.196.214->140.110.113.131 @@3@@20030811_115400~20030811_115400@@[0]@@[0]@@[SSH: Pragma Fortress Key OverFlow]@@[22]@@[2]@@2-1-1
111.165.17.16@@phe96.sro.nchc.org.tw@@65432,555@@20030811_121000~20030811_121000@@FTP: Format String in Command@@snort,idp8200,nk7admin@@222
112.78.196.214@@140.110.110.4@@22,999@@20030811_123000~20030811_123000@@SSH: Pragma Fortress Key OverFlow@@snort@@44

試算結果

06/09

  • snort 警訊 1081 筆,idp8200 警訊 1000 筆, nk7admin 警訊 1000 筆,共 3081 筆資訊
  • 整合後得 654 筆輸出結果,以及一張攻擊圖,
  • 運算時間為 34 秒
  • 之後會將輸出結果導入資料庫,並且最佳化攻擊圖。

07/07

  • snort 警訊 1081 筆,idp8200 警訊 1000 筆, nk7admin 警訊 1000 筆,共 3081 筆資訊
  • 整合後得 654 筆警訊列表,黑名單 654 筆資訊, 圖 654 筆事件
  • 運算時間 4.5 秒

Attachments (2)

Download all attachments as: .zip