Changes between Version 7 and Version 8 of krg_DRBL


Ignore:
Timestamp:
Feb 21, 2008, 1:41:23 PM (16 years ago)
Author:
rock
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • krg_DRBL

    v7 v8  
    111111
    112112 * install DRBL
    113 ||Before we install DRBL, we must clear plan our DRBL environment. The below layout is our environment, eth0 used to connect WAN, eth1 used for DRBL internal clients.||
    114 {{{
    115            NIC     NIC IP                              Clients
     113{{{
     114Before we install DRBL, we must clear plan our DRBL environment. The below layout is our environment, eth0 used to connect WAN, eth1 used for DRBL internal clients.
     115
     116             NIC     NIC IP                              Clients
    116117+-----------------------------+
    117 |     DRBL SERVER             |
    118 |                             |
     118|     DRBL SERVER         |
     119|                                         |
    119120|     +-- [eth0] 140.110.X.X  +- to WAN
    120 |                             |
    121 |     +-- [eth1] 192.168.0.1  +- to clients group 1 [ 7 clients, their IP from 192.168.0.2 - 192.168.0.8]
    122 |                             |
     121|                                         |
     122|     +-- [eth1] 192.168.0.1 +- to clients group 1 [ 7 clients, their IP from 192.168.0.2 - 192.168.0.8]
     123|                                         |
    123124+-----------------------------+
    124125}}}
     126
     127{{{
     128sudo aptitude install drbl
     129(DRBL will be installed in directory /opt/drbl )
     130
     131sudo /opt/drbl/sbin/drblsrv -i
     132sudo /opt/drbl/sbin/drblpush-offline -s `uname -r`
     133(The command used interactive mothod help user to install. It install related packages (nfs, dhcp, tftp......) and create /tftpboot directory. The /tftpboot include:
     134nbi_img: kenrel , initrd image and grub menu
     135node_root: server directories copy
     136nodes: each nodes' individual directories
     137)
     138
     139sudo /opt/drbl/sbin/drblpush -i
     140(the command will deploy client environment, like client name, DRBL mode, swap ...)
     141}}}
     142
     143 * setup each node's grub menu
     144{{{
     145cd /tftpboot/nbi_im/pxelinux.cfg
     146(named rule is IP's hexadecimal 
     147ex. 192.168.0.2  ->  C0A80002)
     148
     149cp default  C0A80002
     150vim  C0A80002
     151( add node_id in append line:
     152ex.
     153label drbl
     154  MENU DEFAULT
     155  # MENU HIDE
     156  MENU LABEL Ubuntu 7.04 Linux (DRBL mode, mostly local resources)
     157  # MENU PASSWD
     158  kernel vmlinuz-pxe
     159  append initrd=initrd-pxe.img devfs=nomount drblthincli=off selinux=0 node_id=0 session_id=9
     160)
     161}}}
     162
     163== test GPFS ==
     164 * running
     165{{{
     166If kerrighed module don't auto load when booting:
     167sudo /etc/init.d/kerrighed start
     168(all node must load, and we can use command dmesg see node message
     169ex.
     170TIPC: Established link <1.1.1:eth0-1.1.3:eth0> on network plane A
     171krg_node_arrival: 2
     172)
     173
     174sudo krgadm cluster start
     175(Kerrighed is running on 7 nodes)
     176
     177top
     178(we cane see all clients' CPU and Memory are combined
     179ex.
     180top - 18:53:16 up 10 min,  2 users,  load average: 0.10, 0.07, 0.04
     181Tasks: 221 total,   1 running, 220 sleeping,   0 stopped,   0 zombie
     182Cpu0  :  0.7%us,  0.3%sy,  0.0%ni, 98.6%id,  0.2%wa,  0.0%hi,  0.1%si,  0.0%st
     183Cpu1  :  0.7%us,  0.3%sy,  0.0%ni, 98.6%id,  0.2%wa,  0.0%hi,  0.1%si,  0.0%st
     184Cpu2  :  0.7%us,  0.3%sy,  0.0%ni, 98.7%id,  0.2%wa,  0.0%hi,  0.1%si,  0.0%st
     185Cpu3  :  0.8%us,  0.4%sy,  0.0%ni, 98.4%id,  0.3%wa,  0.0%hi,  0.1%si,  0.0%st
     186Cpu4  :  0.8%us,  0.4%sy,  0.0%ni, 98.5%id,  0.2%wa,  0.0%hi,  0.1%si,  0.0%st
     187Cpu5  :  0.7%us,  0.4%sy,  0.0%ni, 98.5%id,  0.2%wa,  0.0%hi,  0.1%si,  0.0%st
     188Cpu6  :  0.8%us,  0.3%sy,  0.0%ni, 98.6%id,  0.2%wa,  0.0%hi,  0.1%si,  0.0%st
     189Mem:  14530264k total,  1508584k used, 13021680k free,      560k buffers
     190Swap:  2650684k total,        0k used,  2650684k free,  1282652k cached
     191
     192  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
     193    1 root      15   0  1580  548  480 S    0  0.0   0:00.30 init.orig
     194    2 root      34  19     0    0    0 S    0  0.0   0:00.00 ksoftirqd/0
     195    3 root      RT   0     0    0    0 S    0  0.0   0:00.00 watchdog/0
     196    4 root      10  -5     0    0    0 S    0  0.0   0:00.00 events/0
     197)
     198}}}
     199
     200 * test Kerrighed command
     201{{{
     202sudo krgadm nodes status
     203(ex.
     204[rock@krg002 ~]$ krgadm nodes
     205  0:1   1:1   2:1   3:1   4:1   5:1   6:1
     206)
     207
     208sudo krgcapset -s
     209(ex.
     210Permitted Capabilities: 037777777777
     211        CHANGE_KERRIGHED_CAP, CAN_MIGRATE, DISTANT_FORK, FORK_DELAY
     212        CHECKPOINTABLE, USE_REMOTE_MEMORY, USE_INTRA_CLUSTER_KERSTREAMS
     213        USE_INTER_CLUSTER_KERSTREAMS, USE_WORLD_VISIBLE_KERSTREAMS
     214        SEE_LOCAL_PROC_STAT
     215Effective Capabilities: 01
     216        CHANGE_KERRIGHED_CAP
     217Inheritable Permitted Capabilities: 037777777777
     218        CHANGE_KERRIGHED_CAP, CAN_MIGRATE, DISTANT_FORK, FORK_DELAY
     219        CHECKPOINTABLE, USE_REMOTE_MEMORY, USE_INTRA_CLUSTER_KERSTREAMS
     220        USE_INTER_CLUSTER_KERSTREAMS, USE_WORLD_VISIBLE_KERSTREAMS
     221        SEE_LOCAL_PROC_STAT
     222Inheritable Effective Capabilities: 01
     223        CHANGE_KERRIGHED_CAP
     224)
     225}}}
     226
     227 * Test process migration
     228{{{
     229DRBL Server:
     230mkdir /home/ker ; chmod 777 /home/ker
     231cd /home/ker
     232wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.22.tar.gz
     233tar zxvf linux-2.6.22.tar.gz
     234
     235Client:
     236krgcapset -d +CAN_MIGRATE
     237cd /home/linux-2.6.22
     238sudo make -j 24 bzImage
     239sudo dmesg
     240(you can command dmesg to see message of process migration
     241ex.
     242send_kerrighed_signal: 36647 (Migration Mgr) -> 36885 (bzip2)
     243send_kerrighed_signal: 36647 (Migration Mgr) -> 37449 (cc1)
     244send_kerrighed_signal: 36647 (Migration Mgr) -> 39064 (cc1)
     245send_kerrighed_signal: 36647 (Migration Mgr) -> 39234 (cc1)
     246send_kerrighed_signal: 36647 (Migration Mgr) -> 39269 (cc1)
     247send_kerrighed_signal: 36647 (Migration Mgr) -> 39325 (cc1)
     248send_kerrighed_signal: 36647 (Migration Mgr) -> 39402 (cc1)
     249send_kerrighed_signal: 36647 (Migration Mgr) -> 39465 (cc1)
     250send_kerrighed_signal: 36647 (Migration Mgr) -> 39543 (cc1)
     251send_kerrighed_signal: 36647 (Migration Mgr) -> 39599 (cc1)
     252send_kerrighed_signal: 36647 (Migration Mgr) -> 39660 (cc1)
     253)
     254}}}
     255
     256== Reference ==
     257
     258DRBL
     259http://drbl.sourceforge.net/
     260[[BR]]
     261Kerrighed
     262http:///www.kerrighed.org/wiki/index.php/Main_Page