wiki:jazz/09-09-11

Version 21 (modified by jazz, 14 years ago) (diff)

--

2009-09-11

  • [陽明] 長久以來，/etc/hosts 對於 MPICH2 就容易造成一些困擾，今天又發現對 torque 也有影響。
    • [狀況] pbs_mom 連不上 pbs_server - 根據 /var/spool/torque/mom_logs/ 
      09/11/2009 00:00:49;0002;   pbs_mom;n/a;mom_server_check_connection;connection to server bio037 timeout
09/11/2009 00:00:49;0002;   pbs_mom;n/a;mom_server_check_connection;sending hello to server bio037

09/11/2009 02:28:36;0001;   pbs_mom;Svr;pbs_mom;mom_server_valid_message_source, bad connect from 192.168.128.254:1023 - unauthorized server
    • [觀察] pbs_server 跟 pbs_sched 開的 port 
      jazz@bio037:~$ sudo netstat -nap | grep pbs
tcp        0      0 0.0.0.0:15001       0.0.0.0:*               LISTEN      18343/pbs_server
tcp        0      0 140.129.162.183:15004   0.0.0.0:*               LISTEN      18345/pbs_sched
udp        0      0 0.0.0.0:15001       0.0.0.0:*                        18343/pbs_server
udp        0      0 0.0.0.0:1022       0.0.0.0:*                        18345/pbs_sched
udp        0      0 0.0.0.0:1023       0.0.0.0:*                        18343/pbs_server
    • [解法] 把 hostname 的對應 IP 設到對內的網卡上去。

      • /etc/hosts

        old new  
        11127.0.0.1      localhost
        2 140.X.X.X bio2.classcloud.org bio037
        3 192.168.128.254 bio-eth0
         2140.X.X.X bio2.classcloud.org
         3192.168.128.254 bio-eth0 bio037
        44192.168.128.13 bio013
        55192.168.128.14 bio014
        66192.168.128.15 bio015
    • [解法] 一勞永逸法 - 因為每次跑 drblpush 或跑 re-deploy 的時候都會改掉 /etc/hosts，比較好的方法還是 patch drbl 的 script

      • drbl-etc-hosts

        old new  
        4848    srv_ip="$(drbl-get-ipadd $interface)"
        4949    if grep -q -E "^$srv_ip[[:space:]]+" $HOSTS_OUT; then
        5050      # found the old one, replace that
        51       perl -pi -e "s|^$srv_ip.*|$srv_ip ${hostname_prefix}-$interface|g" $HOSTS_OUT
         51      perl -pi -e "s|^$srv_ip.*|$srv_ip ${hostname_prefix}-$interface $(hostname)|g" $HOSTS_OUT
        5252    else
        5353      # old one does not exist, create one
        5454      cat <<-EOF >> $HOSTS_OUT
    • [解法] 如果 pbs_mom 跑不起來，要注意 "/var/spool/torque/spool/" 跟 "/var/spool/torque/undelivered/" 兩個目錄有無寫入權限 
      bio011:~# /opt/torque/sbin/pbs_mom
pbs_mom: Permission denied (13) in chk_file_sec, Security violation with "/var/spool/torque/spool/" - /var/spool/torque/spool/ cannot be accessed
pbs_mom: Permission denied (13) in chk_file_sec, Security violation with "/var/spool/torque/undelivered/" - /var/spool/torque/undelivered/ cannot be accessed

      ~# dsh -a killall -9 pbs_mom
~# rm -rf `for i in /tftpboot/nodes/192.168.129.*; do find $i -type d -name "*torque"; done`
~# for i in /tftpboot/nodes/192.168.129.*; do cp -r /var/spool/torque $i/var/spool/; done
~# dsh -a chmod a+w /var/spool/torque/spool
~# dsh -a chmod a+w /var/spool/torque/undelivered
~# dsh -a /opt/torque/sbin/pbs_mom
  • [課程] 還原電腦教室 A 的 Hadoop Image 
    #!/bin/bash
/opt/drbl/sbin/drbl-ocs -b -g auto -e1 auto -e2 -x -j2 -o1 -p poweroff --clients-to-wait 31 --max-time-to-wait 300 --broadcast -h "192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4 192.168.100.5 192.168.100.6 192.168.100.7 192.168.100.8 192.168.100.9 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14 192.168.100.15 192.168.100.16 192.168.100.17 192.168.100.18 192.168.100.19 192.168.100.20 192.168.100.21 192.168.100.22 192.168.100.23 192.168.100.24 192.168.100.25 192.168.100.26 192.168.100.27 192.168.100.28 192.168.100.29 192.168.100.30 192.168.100.31" -l zh_TW.UTF-8 startdisk multicast_restore 2009-08-24-Hadoop_Class sda
/opt/drbl/sbin/dcs -h "192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4 192.168.100.5 192.168.100.6 192.168.100.7 192.168.100.8 192.168.100.9 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14 192.168.100.15 192.168.100.16 192.168.100.17 192.168.100.18 192.168.100.19 192.168.100.20 192.168.100.21 192.168.100.22 192.168.100.23 192.168.100.24 192.168.100.25 192.168.100.26 192.168.100.27 192.168.100.28 192.168.100.29 192.168.100.30 192.168.100.31" Wake-on-LAN

    ### 刪掉上次留著的視窗分頁紀錄 ###
drbl:~# dsh -a rm /home/hadooper/.mozilla/firefox/yq376prp.default/sessionstore.js        
### 替換瀏覽器預設首頁為新的 URL ###
drbl:~# dsh -a "cat /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js | sed 's#NCHCCloudCourse090824#NCHCCloudCourse090914#' > /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js.new"
drbl:~# dsh -a "mv /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js.new /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js"
drbl:~# dsh -a grep trac.nchc.org.tw /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js

Open Source : Perl

Open Source : Python

Message Queue

SQLite

  • ​Embedding an SQL Database with SQLite - 有講到 SQLite 的架構
  • ​SQLite examples with Bash, Perl and Python - 用很簡潔的方式介紹 Bash, Perl 跟 Python 如何存取 SQLite 
    #!/usr/bin/perl -w
use DBI;
use strict;
my $db = DBI->connect("dbi:SQLite:test.db", "", "",
{RaiseError => 1, AutoCommit => 1});

$db->do("CREATE TABLE n (id INTEGER PRIMARY KEY, f TEXT, l TEXT)");
$db->do("INSERT INTO n VALUES (NULL, 'john', 'smith')");
my $all = $db->selectall_arrayref("SELECT * FROM n");

foreach my $row (@$all) {
my ($id, $first, $last) = @$row;
print "$id|$first|$lastn";
}

    #!/usr/bin/python
from pysqlite2 import dbapi2 as sqlite

db = sqlite.connect('test.db')
cur = db.cursor()
cur.execute('CREATE TABLE n (id INTEGER PRIMARY KEY, f TEXT, l TEXT)')
db.commit()
cur.execute('INSERT INTO n (id, f, l) VALUES(NULL, "john", "smith")')
db.commit()
cur.execute('SELECT * FROM n')
print cur.fetchall()

    #!/bin/bash
sqlite3 test.db  "create table n (id INTEGER PRIMARY KEY,f TEXT,l TEXT);"
sqlite3 test.db  "insert into n (f,l) values ('john','smith');"
sqlite3 test.db  "select * from n";
  • ​PHP: SQLite - Manual - PHP 存取 SQLite 的原文手冊
  • ​PHP-SQLite - 資策會技術支援資料庫 
    <?PHP
 
$db = sqlite_open('ex3.db', 0666, $sqliteerror);
$query = sqlite_query($db, "SELECT * FROM tbl");
var_dump($query);
sqlite_close($db);
?>

    <?PHP
$db = sqlite_open('ex3.db', 0666, $sqliteerror);
$query = sqlite_array_query($db, "SELECT * FROM tbl", SQLITE_ASSOC);
//Object-oriented style
//$dbhandle = new SQLiteDatabase('ex3.db');
//$query = $dbhandle->arrayQuery('SELECT * FROM tbl', SQLITE_ASSOC);
 
foreach ($query as $entry) {
echo Title: ' . $entry['title'] . ' Length: ' . $entry['length'];
echo "<br>";
}
?>

OpenPBS / Torque