wiki:jazz/09-09-11

2009-09-11

  • [陽明] 長久以來,/etc/hosts 對於 MPICH2 就容易造成一些困擾,今天又發現對 torque 也有影響。
    • [狀況] pbs_mom 連不上 pbs_server - 根據 /var/spool/torque/mom_logs/
      09/11/2009 00:00:49;0002;   pbs_mom;n/a;mom_server_check_connection;connection to server bio037 timeout
      09/11/2009 00:00:49;0002;   pbs_mom;n/a;mom_server_check_connection;sending hello to server bio037
      
      09/11/2009 02:28:36;0001;   pbs_mom;Svr;pbs_mom;mom_server_valid_message_source, bad connect from 192.168.128.254:1023 - unauthorized server
      
    • [觀察] pbs_server 跟 pbs_sched 開的 port
      jazz@bio037:~$ sudo netstat -nap | grep pbs
      tcp        0      0 0.0.0.0:15001       0.0.0.0:*               LISTEN      18343/pbs_server
      tcp        0      0 140.129.162.183:15004   0.0.0.0:*               LISTEN      18345/pbs_sched
      udp        0      0 0.0.0.0:15001       0.0.0.0:*                        18343/pbs_server
      udp        0      0 0.0.0.0:1022       0.0.0.0:*                        18345/pbs_sched
      udp        0      0 0.0.0.0:1023       0.0.0.0:*                        18343/pbs_server
      
    • [解法] 把 hostname 的對應 IP 設到對內的網卡上去。
      • /etc/hosts

        old new  
        11127.0.0.1      localhost
        2 140.X.X.X bio2.classcloud.org bio037
        3 192.168.128.254 bio-eth0
         2140.X.X.X bio2.classcloud.org
         3192.168.128.254 bio-eth0 bio037
        44192.168.128.13 bio013
        55192.168.128.14 bio014
        66192.168.128.15 bio015
    • [解法] 一勞永逸法 - 因為每次跑 drblpush 或跑 re-deploy 的時候都會改掉 /etc/hosts,比較好的方法還是 patch drbl 的 script
      • drbl-etc-hosts

        old new  
        4848    srv_ip="$(drbl-get-ipadd $interface)"
        4949    if grep -q -E "^$srv_ip[[:space:]]+" $HOSTS_OUT; then
        5050      # found the old one, replace that
        51       perl -pi -e "s|^$srv_ip.*|$srv_ip ${hostname_prefix}-$interface|g" $HOSTS_OUT
         51      perl -pi -e "s|^$srv_ip.*|$srv_ip ${hostname_prefix}-$interface $(hostname)|g" $HOSTS_OUT
        5252    else
        5353      # old one does not exist, create one
        5454      cat <<-EOF >> $HOSTS_OUT
    • [解法] 如果 pbs_mom 跑不起來,要注意 "/var/spool/torque/spool/" 跟 "/var/spool/torque/undelivered/" 兩個目錄有無寫入權限
      bio011:~# /opt/torque/sbin/pbs_mom
      pbs_mom: Permission denied (13) in chk_file_sec, Security violation with "/var/spool/torque/spool/" - /var/spool/torque/spool/ cannot be accessed
      pbs_mom: Permission denied (13) in chk_file_sec, Security violation with "/var/spool/torque/undelivered/" - /var/spool/torque/undelivered/ cannot be accessed
      
      ~# dsh -a killall -9 pbs_mom
      ~# rm -rf `for i in /tftpboot/nodes/192.168.129.*; do find $i -type d -name "*torque"; done`
      ~# for i in /tftpboot/nodes/192.168.129.*; do cp -r /var/spool/torque $i/var/spool/; done
      ~# dsh -a chmod a+w /var/spool/torque/spool
      ~# dsh -a chmod a+w /var/spool/torque/undelivered
      ~# dsh -a /opt/torque/sbin/pbs_mom
      
  • [課程] 還原電腦教室 A 的 Hadoop Image
    #!/bin/bash
    /opt/drbl/sbin/drbl-ocs -b -g auto -e1 auto -e2 -x -j2 -o1 -p poweroff --clients-to-wait 31 --max-time-to-wait 300 --broadcast -h "192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4 192.168.100.5 192.168.100.6 192.168.100.7 192.168.100.8 192.168.100.9 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14 192.168.100.15 192.168.100.16 192.168.100.17 192.168.100.18 192.168.100.19 192.168.100.20 192.168.100.21 192.168.100.22 192.168.100.23 192.168.100.24 192.168.100.25 192.168.100.26 192.168.100.27 192.168.100.28 192.168.100.29 192.168.100.30 192.168.100.31" -l zh_TW.UTF-8 startdisk multicast_restore 2009-08-24-Hadoop_Class sda
    /opt/drbl/sbin/dcs -h "192.168.100.1 192.168.100.2 192.168.100.3 192.168.100.4 192.168.100.5 192.168.100.6 192.168.100.7 192.168.100.8 192.168.100.9 192.168.100.10 192.168.100.11 192.168.100.12 192.168.100.13 192.168.100.14 192.168.100.15 192.168.100.16 192.168.100.17 192.168.100.18 192.168.100.19 192.168.100.20 192.168.100.21 192.168.100.22 192.168.100.23 192.168.100.24 192.168.100.25 192.168.100.26 192.168.100.27 192.168.100.28 192.168.100.29 192.168.100.30 192.168.100.31" Wake-on-LAN
    
    ### 刪掉上次留著的視窗分頁紀錄 ###
    drbl:~# dsh -a rm /home/hadooper/.mozilla/firefox/yq376prp.default/sessionstore.js        
    ### 替換瀏覽器預設首頁為新的 URL ###
    drbl:~# dsh -a "cat /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js | sed 's#NCHCCloudCourse090824#NCHCCloudCourse090914#' > /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js.new"
    drbl:~# dsh -a "mv /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js.new /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js"
    drbl:~# dsh -a grep trac.nchc.org.tw /home/hadooper/.mozilla/firefox/yq376prp.default/prefs.js
    

Open Source : Perl

Open Source : Python

Message Queue

SQLite

  • Embedding an SQL Database with SQLite - 有講到 SQLite 的架構
  • SQLite examples with Bash, Perl and Python - 用很簡潔的方式介紹 Bash, Perl 跟 Python 如何存取 SQLite
    #!/usr/bin/perl -w
    use DBI;
    use strict;
    my $db = DBI->connect("dbi:SQLite:test.db", "", "",
    {RaiseError => 1, AutoCommit => 1});
    
    $db->do("CREATE TABLE n (id INTEGER PRIMARY KEY, f TEXT, l TEXT)");
    $db->do("INSERT INTO n VALUES (NULL, 'john', 'smith')");
    my $all = $db->selectall_arrayref("SELECT * FROM n");
    
    foreach my $row (@$all) {
    my ($id, $first, $last) = @$row;
    print "$id|$first|$lastn";
    }
    
    #!/usr/bin/python
    from pysqlite2 import dbapi2 as sqlite
    
    db = sqlite.connect('test.db')
    cur = db.cursor()
    cur.execute('CREATE TABLE n (id INTEGER PRIMARY KEY, f TEXT, l TEXT)')
    db.commit()
    cur.execute('INSERT INTO n (id, f, l) VALUES(NULL, "john", "smith")')
    db.commit()
    cur.execute('SELECT * FROM n')
    print cur.fetchall()
    
    #!/bin/bash
    sqlite3 test.db  "create table n (id INTEGER PRIMARY KEY,f TEXT,l TEXT);"
    sqlite3 test.db  "insert into n (f,l) values ('john','smith');"
    sqlite3 test.db  "select * from n";
    
  • PHP: SQLite - Manual - PHP 存取 SQLite 的原文手冊
  • PHP-SQLite - 資策會技術支援資料庫
    <?PHP
     
    $db = sqlite_open('ex3.db', 0666, $sqliteerror);
    $query = sqlite_query($db, "SELECT * FROM tbl");
    var_dump($query);
    sqlite_close($db);
    ?>
    
    <?PHP
    $db = sqlite_open('ex3.db', 0666, $sqliteerror);
    $query = sqlite_array_query($db, "SELECT * FROM tbl", SQLITE_ASSOC);
    //Object-oriented style
    //$dbhandle = new SQLiteDatabase('ex3.db');
    //$query = $dbhandle->arrayQuery('SELECT * FROM tbl', SQLITE_ASSOC);
     
    foreach ($query as $entry) {
    echo Title: ' . $entry['title'] . ' Length: ' . $entry['length'];
    echo "<br>";
    }
    ?>
    

OpenPBS / Torque

Last modified 12 years ago Last modified on Jan 23, 2013, 12:12:40 AM