{{{ #!html
crawlzilla 新版
v 1.0
}}} [[PageOutline]] = 目標 = * 多人共用版本 * 網頁介面更新 * 加入排程等新功能 * 更新 nutch 版本至 1.2 * svn 庫上的安裝測試模式 * slave安裝可搭配網頁引導 = 系統分析 = == 目錄結構 == * /home/crawler/crawlzilla || 目錄1 || 目錄2 || 說明 || || ./workspace/ || || hadoop 的運算資料夾 || || ./meta/ || || dialog 產生的中間檔 || || ./meta/tmp/ || || 暫存檔 || || ./user/ || || 於後說明 || * /home/crawler/crawlzilla/user 下的目錄格式說明 || 目錄1 || 目錄2 || 說明 || || [admin,_username_]/ || || admin 為必有資料夾,_username_ 為之後新增的使用者 || || || ./webs/ || 內放搜尋網頁的資料夾 (註1)|| || || ./webs/_DBName_/ || 名稱為_DBName_的搜尋網頁 || || || ./IDB/ || 內放該使用者已完成的 indexDB 資料夾 || || || ./IDB/_DBName_/ || _DBName_ 為索引庫名稱 || || || ./IDB/_DBName_/meta/ || meta 放每個索引庫的相關檔案 || || || ./IDB/_DBName_/index~segments/ || index~segments 為 lucene db 的必要五個資料夾|| || || ./tmp/ || 內放該使用者未完成的 indexDB 資料夾 || || || ./tmp/_DBName_/ || _DBName_ 為索引庫名稱 || || || ./tmp/_DBName_/meta/ || meta 放每個索引庫的相關檔案 || || || ./meta/ || 該使用者的個人資訊,如pwd,email 等 || * /opt/crawlzilla/ || 目錄1 || 目錄2 || 說明 || || ./tomcat/ || || tomcat || || ./tomcat/ || ./webapps/_username_/_DBName_ || 對應到 _username_ 的 _DBName_ 索引庫 (註1)|| || ./nutch/ || || nutch 的目錄 || || ./slave/ || || 給 slave 安裝需要的檔案 || || ./main/ || || 放 crawlzilla 的執行檔|| 註: /home/crawler/crawlzilla/user/_username_/webs/_DBName_ ==鍊結到==> /opt/crawlzilla/tomcat/webapps/_username_/_DBName_ 如:ln -sf /home/crawler/crawlzilla/user/admin/webs/test_3 /opt/crawlzilla/tomcat/webapps/admin/test_3 * /var/log/crawlzilla/ || 目錄1 || 目錄2 || 說明 || || ./hadoop-logs/ || || || || ./hadoop-pids/ || || || || ./shell-logs/ || || || || ./tomcat-logs/ || || || == 新舊 檔案\目錄 對照 == || 舊 || ==> || 新 || 說明 || || /home/crawler/crawlzilla/logs || ==> || || 刪除此鍊結 || || /home/crawler/crawlzilla/nutch || ==> || || 刪除此鍊結 || || /home/crawler/crawlzilla/tmp || ==> || /home/crawler/crawlzilla/tmp || 不變 || || /home/crawler/crawlzilla/source || ==> || /opt/crawlzilla/slave || || || /home/crawler/crawlzilla/archieve/_DBName_ || ==> || /home/crawler/crawlzilla/user/admin/IDB/_DBName_ || || || /home/crawler/crawlzilla/urls || ==> || /home/crawler/crawlzilla/meta/urls || || || /home/crawler/crawlzilla/.metadata/_DBName_ || ==> || /home/crawler/crawlzilla/user/admin/IDB/_DBName_/meta (註2) || || || /home/crawler/crawlzilla/.menu_tmp || ==> || /home/crawler/crawlzilla/meta/menu_tmp || || || /home/crawler/crawlzilla/system/ || ==> || 於下說明 || || * 註2: 0.3 版以前,無論完成與否的IDB中間資料都放在 /home/crawler/crawlzilla/.metadata/。但 1.0 版以後,未完成的 /home/crawler/crawlzilla/user/admin/tmp/_DBName_/meta ,完成之後搬移到 /home/crawler/crawlzilla/user/admin/IDB/_DBName_/meta * /home/crawler/crawlzilla/system: || 舊 || ==> || 新 || 說明 || || 執行檔 || ==> || /opt/crawlzilla/main/執行檔 || 如 crawlzilla, install, go.sh ... || || lang/ || ==> || /opt/crawlzilla/main/lang/ || 語言檔資料夾 || || hosts || ==> || /home/crawler/crawlzilla/meta/ || || || hosts.old || ==> || /home/crawler/crawlzilla/meta/ || || || hosts.bak || ==> || /home/crawler/crawlzilla/meta/ || || || version || ==> || /opt/crawlzilla/version || || || crawl_nodes || ==> || /home/crawler/crawlzilla/meta/ || || || crawl_nodes.bak || ==> || /home/crawler/crawlzilla/meta/ || || || crawl_nodes.old || ==> || /home/crawler/crawlzilla/meta/ || || == 環境參數 == (以下為舊的) * Crawlzilla_Install_PATH="/opt/crawlzilla" * Tomcat_HOME="/opt/crawlzilla/tomcat" * Crawlzilla_HOME="/home/crawler/crawlzilla" * Work_Path=$Crawlzilla_HOME/system * Manu_Tmp_Path="/home/crawler/crawlzilla/meta" * Hadoop_Daemon="/opt/crawlzilla/nutch/bin/hadoop-daemon.sh" * PID_Dir="/var/log/crawlzilla/hadoop-pids" * Crawl_Nodes=$Crawlzilla_HOME/meta/crawl_nodes = 功能 = == shell == * 狀態 * 運算設定 * 快速設定 * 網頁伺服器設定 * 多人版帳號管理 * 語言切換 * slave安裝提示