= 2009-09-11 = * [陽明] 長久以來，/etc/hosts 對於 MPICH2 就容易造成一些困擾，今天又發現對 torque 也有影響。 * [狀況] pbs_mom 連不上 pbs_server - 根據 /var/spool/torque/mom_logs/ {{{ 09/11/2009 00:00:49;0002; pbs_mom;n/a;mom_server_check_connection;connection to server bio037 timeout 09/11/2009 00:00:49;0002; pbs_mom;n/a;mom_server_check_connection;sending hello to server bio037 09/11/2009 02:28:36;0001; pbs_mom;Svr;pbs_mom;mom_server_valid_message_source, bad connect from 192.168.128.254:1023 - unauthorized server }}} * [觀察] pbs_server 跟 pbs_sched 開的 port {{{ jazz@bio037:~$ sudo netstat -nap | grep pbs tcp 0 0 0.0.0.0:15001 0.0.0.0:* LISTEN 18343/pbs_server tcp 0 0 140.129.162.183:15004 0.0.0.0:* LISTEN 18345/pbs_sched udp 0 0 0.0.0.0:15001 0.0.0.0:* 18343/pbs_server udp 0 0 0.0.0.0:1022 0.0.0.0:* 18345/pbs_sched udp 0 0 0.0.0.0:1023 0.0.0.0:* 18343/pbs_server }}} * [解法] 把 hostname 的對應 IP 設到對內的網卡上去。 {{{ #!diff --- /etc/hosts.org 2009-09-11 02:41:13.000000000 +0800 +++ /etc/hosts 2009-09-11 02:34:26.000000000 +0800 @@ -1,6 +1,6 @@ 127.0.0.1 localhost -140.X.X.X bio2.classcloud.org bio037 -192.168.128.254 bio-eth0 +140.X.X.X bio2.classcloud.org +192.168.128.254 bio-eth0 bio037 192.168.128.13 bio013 192.168.128.14 bio014 192.168.128.15 bio015 }}} * [解法] 一勞永逸法 - 因為每次跑 drblpush 或跑 re-deploy 的時候都會改掉 /etc/hosts，比較好的方法還是 patch drbl 的 script {{{ #!diff --- drbl-etc-hosts.org 2010-08-03 18:03:13.000000000 +0800 +++ drbl-etc-hosts 2010-08-03 18:03:31.000000000 +0800 @@ -48,7 +48,7 @@ srv_ip="$(drbl-get-ipadd $interface)" if grep -q -E "^$srv_ip[[:space:]]+" $HOSTS_OUT; then # found the old one, replace that - perl -pi -e "s|^$srv_ip.*|$srv_ip ${hostname_prefix}-$interface|g" $HOSTS_OUT + perl -pi -e "s|^$srv_ip.*|$srv_ip ${hostname_prefix}-$interface $(hostname)|g" $HOSTS_OUT else # old one does not exist, create one cat <<-EOF >> $HOSTS_OUT }}} * [解法] 如果 pbs_mom 跑不起來，要注意 "/var/spool/torque/spool/" 跟 "/var/spool/torque/undelivered/" 兩個目錄有無寫入權限 {{{ bio011:~# /opt/torque/sbin/pbs_mom pbs_mom: Permission denied (13) in chk_file_sec, Security violation with "/var/spool/torque/spool/" - /var/spool/torque/spool/ cannot be accessed pbs_mom: Permission denied (13) in chk_file_sec, Security violation with "/var/spool/torque/undelivered/" - /var/spool/torque/undelivered/ cannot be accessed }}} {{{ ~# dsh -a killall -9 pbs_mom ~# rm -rf `for i in /tftpboot/nodes/192.168.129.*; do find $i -type d -name "*torque"; done` ~# for i in /tftpboot/nodes/192.168.129.*; do cp -r /var/spool/torque $i/var/spool/; done ~# dsh -a chmod a+w /var/spool/torque/spool ~# dsh -a chmod a+w /var/spool/torque/undelivered ~# dsh -a /opt/torque/sbin/pbs_mom }}} * [課程] 還原電腦教室 A 的 Hadoop Image {{{ #!sh #!/bin/bash /opt/drbl/sbin/drbl-ocs -b -g auto -e1 auto -e2 -x -j2 -o1 -p poweroff --clients-to-wait 31 --max-time-to-wait 300 --broadcast -h "192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4 192.168.100.5 192.168.100.6 192.168.100.7 192.168.100.8 192.168.100.9 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14 192.168.100.15 192.168.100.16 192.168.100.17 192.168.100.18 192.168.100.19 192.168.100.20 192.168.100.21 192.168.100.22 192.168.100.23 192.168.100.24 192.168.100.25 192.168.100.26 192.168.100.27 192.168.100.28 192.168.100.29 192.168.100.30 192.168.100.31" -l zh_TW.UTF-8 startdisk multicast_restore 2009-08-24-Hadoop_Class sda /opt/drbl/sbin/dcs -h "192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4 192.168.100.5 192.168.100.6 192.168.100.7 192.168.100.8 192.168.100.9 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14 192.168.100.15 192.168.100.16 192.168.100.17 192.168.100.18 192.168.100.19 192.168.100.20 192.168.100.21 192.168.100.22 192.168.100.23 192.168.100.24 192.168.100.25 192.168.100.26 192.168.100.27 192.168.100.28 192.168.100.29 192.168.100.30 192.168.100.31" Wake-on-LAN }}} {{{ ### 刪掉上次留著的視窗分頁紀錄 ### drbl:~# dsh -a rm /home/hadooper/.mozilla/firefox/yq376prp.default/sessionstore.js ### 替換瀏覽器預設首頁為新的 URL ### drbl:~# dsh -a "cat /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js | sed 's#NCHCCloudCourse090824#NCHCCloudCourse090914#' > /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js.new" drbl:~# dsh -a "mv /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js.new /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js" drbl:~# dsh -a grep trac.nchc.org.tw /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js }}} == Open Source : Perl == * [http://search.cpan.org/~hayashi/Term-ReadLine-Gnu-1.19/eg/perlsh perlsh] * [http://debian.pkgs.cpan.org/ CPAN 額外的 Debian 套件庫] == Open Source : Python == * [討論] [http://www.mail-archive.com/debian-python@lists.debian.org/msg05583.html from python egg to debian package : a good example ?] * [http://ghantoos.org/2008/10/19/creating-a-deb-package-from-a-python-setuppy/ Creating a .deb package from a python setup.py] * [http://packages.qa.debian.org/p/python-debian.html python-debian] * [http://packages.qa.debian.org/p/python-apt.html python-apt] * [http://juliank.wordpress.com/2009/04/15/python-apt-to-become-first-debian-package-to-support-python-3/ python-apt to become first Debian package to support Python 3] * [http://www.debian.org/doc/packaging-manuals/python-policy/ch-module_packages.html Debian Python Policy] * [http://blog.bytecraft.com.my/blog/kamal/2009/08/14/build-debian-package-python-modules Build Debian package for Python modules] * [http://ascendwiki.cheme.cmu.edu/Building_a_Debian_package_from_a_source_tarball Building a Debian package from a source tarball] - dtar 是用 python-apt 和 python-debian 寫的打包工具。從文章裡面可以知道如果要打包rpm的話，只要有 .spec 檔。而要打包deb的話，必須寫好 .debian 目錄的 control, rule 等。 {{{ The only requirement for the 'rpmbuild -ta' command is that the tarball contain a 'myprog.spec' file inside. Any source-code tarball that includes the 'debian/*' files should be able to be built directly with it. }}} * 中文叢書 * [http://tlsj.tenlong.com.tw/WebModule/BookSearch/bookSearchViewAction.do?isbn=9789866840302&sid=48113&aid=69e42ffb Python 學習手冊 (Learning Python, 3/e)] * [http://www.books.com.tw/exep/prod/booksfile.php?item=0010434463& Python 3 技術手冊] * IPython - 功能強大，可以按 tab 補齊指令的互動式 Python Shell * http://ipython.scipy.org/ * https://www.osc.edu/cms/sip/ - 一些 IPython 以及 Python 平行化的影片 * [http://showmedo.com/videos/series?name=PythonIPythonSeries iPython 的 screencast 影片] * 入門書《Python Essential Reference》 - [http://fcamel.twbbs.org/archives/2009/07/22/859/ 建議出處] * [http://www.oreilly.com.tw/product2_c.php?id=a247_toc Python的Unix∕Linux 系統管理應用] - [http://shopping.pchome.com.tw/?mod=item&func=exhibit&IT_NO=DJAA05-A40941341&c=A10 這個賣書的]還真有趣 {{{ 把雲端計算（Cloud Computing）整合進你的基礎架構，以及撰寫 Google App Engine 應用程式。 }}} * [http://www.ptt.cc/man/Python/D1F0/M.1220960760.A.8F3.html <推荐> Python 的 IDE] - 看起來 [http://packages.debian.org/eclipse-pydev PyDev] 這個 Eclipse 擴充套件是蠻多人喜歡啦...可是我就是喜歡 command line 的做法，最容易取得。至於怎麼用 iPython 進行 debug 也許就是要等慢慢跟它培養感情了.... * [http://support.oss.org.tw/wiki/index.php/Ipython Ipython - 技術支援資料庫(TechWiki)] * [http://pida.co.uk/ pida] - 有[http://packages.debian.org/pida Debian 套件 pida] - Python Integrated Development Application, a Python IDE - 用 GTK 寫的整合開發工具 == Message Queue == * 目前在評估兩種實作 drbl-agent 的 Message Queue，其一是 POE::Component::!MessageQueue 另一個則是先前測試過的 Orbited * 關於 POE::Component::!MessageQueue * [http://poe.perl.org/ POE: Perl Object Environment] * [http://search.cpan.org/~rcaputo/POE-1.267/lib/POE.pm POE] * [http://packages.debian.org/search?suite=all§ion=all&arch=any&searchon=names&keywords=libpoe Perl Object Environment 相關 Debian 套件] * [http://search.cpan.org/~dsnopek/POE-Component-MessageQueue-0.2.6/lib/POE/Component/MessageQueue.pm POE::Component::!MessageQueue] * [http://search.cpan.org/~aimass/POE-Component-Server-AsyncEndpoint-0.10_a/lib/POE/Component/Server/AsyncEndpoint/ChannelAdapter/Stomp.pm POE::Component::Server::AsyncEndpoint::ChannelAdapter::Stomp] * 關於 Orbited * 2009-09-10 - [http://groups.google.com/group/orbited-users/browse_thread/thread/f42e3c96cdd81b5e?hl=en Orbited debian packages available] {{{ dget -u http://mentors.debian.net/debian/pool/main/o/orbited/orbited_0.7.10-1.dsc dget -u http://mentors.debian.net/debian/pool/main/r/rabbitmq-stomp/rabbitmq-stomp_1.6.0-1.dsc }}} * [註] dget 是 devscripts 套件裡的指令 {{{ devscripts: /usr/bin/dget }}} == SQLite == * [http://www.linuxjournal.com/article/6650/ Embedding an SQL Database with SQLite] - 有講到 SQLite 的架構 * [http://mailliststock.wordpress.com/2007/03/01/sqlite-examples-with-bash-perl-and-python/ SQLite examples with Bash, Perl and Python] - 用很簡潔的方式介紹 Bash, Perl 跟 Python 如何存取 SQLite {{{ #!perl #!/usr/bin/perl -w use DBI; use strict; my $db = DBI->connect("dbi:SQLite:test.db", "", "", {RaiseError => 1, AutoCommit => 1}); $db->do("CREATE TABLE n (id INTEGER PRIMARY KEY, f TEXT, l TEXT)"); $db->do("INSERT INTO n VALUES (NULL, 'john', 'smith')"); my $all = $db->selectall_arrayref("SELECT * FROM n"); foreach my $row (@$all) { my ($id, $first, $last) = @$row; print "$id|$first|$lastn"; } }}} {{{ #!python #!/usr/bin/python from pysqlite2 import dbapi2 as sqlite db = sqlite.connect('test.db') cur = db.cursor() cur.execute('CREATE TABLE n (id INTEGER PRIMARY KEY, f TEXT, l TEXT)') db.commit() cur.execute('INSERT INTO n (id, f, l) VALUES(NULL, "john", "smith")') db.commit() cur.execute('SELECT * FROM n') print cur.fetchall() }}} {{{ #!sh #!/bin/bash sqlite3 test.db "create table n (id INTEGER PRIMARY KEY,f TEXT,l TEXT);" sqlite3 test.db "insert into n (f,l) values ('john','smith');" sqlite3 test.db "select * from n"; }}} * [http://tw.php.net/sqlite PHP: SQLite - Manual] - PHP 存取 SQLite 的原文手冊 * [http://wiki.oss.org.tw/index.php/Php_sqlite PHP-SQLite - 資策會技術支援資料庫] {{{ }}} {{{ arrayQuery('SELECT * FROM tbl', SQLITE_ASSOC); foreach ($query as $entry) { echo Title: ' . $entry['title'] . ' Length: ' . $entry['length']; echo "
"; } ?> }}} == OpenPBS / Torque == * [http://debianclusters.cs.uni.edu/index.php/Resource_Manager:_Torque Resource Manager: Torque] * [http://debianclusters.cs.uni.edu/index.php/Torque_Queue_Configuration Torque Queue Configuration] * [http://debianclusters.cs.uni.edu/index.php/Torque_Queues_Example Torque Queues Example] * [http://drbl.nchc.org.tw/debian/pccluster.html DRBL for Debian 格網(Grid)計算與個人電腦叢集(PC Cluster)] * [http://www.globusconsortium.org/tutorial/ch2/ Globus Toolkit Tutorial Chapter 2: Deploying Torque (Open PBS)]