= '''''Grid_environment''''' = == '''''How to Install Globus Toolkit Based on DRBL''''' == * 1. Introduction[[BR]][[BR]] * The installation guide is written by consulting GT 4.2 Quickstart. We install GT4.2 and DRBL on Debian.[[BR]][[BR]] * 2. Install and set up on the DRBL server and clients[[BR]][[BR]] * At first, you can download gt4.2.0-all-source-installer.tar.gz and build it. After installing, you can test successfully if you follow the instructions(see GT 4.2 Quickstart) to set up your machine. Now we only list some differences and modifications for GT4.2 based on DRBL.[[BR]][[BR]] * part_a) Globus Toolkit:[[BR]][[BR]] * Default shared directory on each machine is $GLOBUS_LOCATION. This directory should be readable and writable by all machines.[[BR]][[BR]] * "$GLOBUS_LOCATION/var" is the only one subdirectory which should be moved to non-NFS mount file system. This is because that globus-schedler-event-generator may doesn't work correctly if the locking of globus-fork.log fails.[[BR]][[BR]] * (1) on the DRBL_server {{{ cd $GLOBUS_LOCATION mv var /etc/ ln -s /etc/var var chmod 622 $GLOBUS_LOCATION/var/globus-for.log (Its permission should be like "-rw--w--w-".) cp /etc/var /tftpboot/nodes/[client_ip]/etc/ -rf }}} * (2) on all DRBL_clients {{{ chmod 622 $GLOBUS_LOCATION/var/globus-for.log (Its permission should be like "-rw--w--w-".) }}} * Each machine has its own directory: /etc/grid-security. This directory stores the signed host(container) keys and certificates and grid-mapfile for user authorization.[[BR]][[BR]] * (1) on the DRBL_server {{{ cp /etc/grid-security /tftpboot/nodes/[client_ip]/etc/ -rf }}} * (2) on each machine {{{ To request and sign its own host certificates. }}} * /etc/grid-security {{{ root@drbl-srv:/etc/grid-security# ls -l drwxr-xr-x 4 root root 4096 2008-09-10 21:45 certificates -rw-r--r-- 1 root root 4625 2008-09-10 20:00 containercert.pem -r-------- 1 root root 891 2008-08-15 22:59 containerkey.pem lrwxrwxrwx 1 root root 61 2008-09-10 21:04 globus-host-ssl.conf -> /etc/grid-security/certificates/globus-host-ssl.conf.71a89a47 lrwxrwxrwx 1 root root 61 2008-09-10 21:04 globus-user-ssl.conf -> /etc/grid-security/certificates/globus-user-ssl.conf.71a89a47 -rw-r--r-- 1 root root 277 2008-08-25 21:10 grid-mapfile lrwxrwxrwx 1 root root 59 2008-09-10 21:04 grid-security.conf -> /etc/grid-security/certificates/grid-security.conf.71a89a47 -rw-r--r-- 1 root root 4625 2008-09-10 20:00 hostcert.pem -rw-r--r-- 1 root root 1367 2008-08-15 01:22 hostcert_request.pem -r-------- 1 root root 891 2008-08-15 01:22 hostkey.pem }}} * /etc/grid-security/certificates {{{ root@drbl-srv:/etc/grid-security/certificates# ls -l -rw-r--r-- 1 root root 1285 2008-09-08 10:16 71a89a47.0 -rw-r--r-- 1 root root 1344 2008-04-09 10:24 71a89a47.signing_policy -rw-r--r-- 1 root root 2625 2008-09-10 14:37 globus-host-ssl.conf.71a89a47 -rw-r--r-- 1 root root 2625 2008-09-10 14:37 globus-user-ssl.conf.71a89a47 -rw-r--r-- 1 root root 1306 2008-04-09 11:27 grid-security.conf.71a89a47 }}} * Each machine should also has some files in its own "/etc". You should copy this files from the DRBL_server to all clients. {{{ /etc/sudoers /etc/services /etc/xinetd.d/myproxy /etc/xinetd.d/gridftp /etc/xinetd.d/globus-gatekeeper }}} * part_b) DRBL:[[BR]][[BR]] * After you install DRBL server, you can follow the example when you execute "drblpush -i" to set up the client environments. Each machine need a host certificate for Globus, and its hostname needs to be fixed. In order to avoid modifying some configuration files(ex:/etc/hosts) every time and avoid starting Globus unsuccessfully, you need to let the DHCP service in DRBL server offer same IP address to the client every time when client boots.[[BR]][[BR]] {{{ root@drbl-srv:/opt/drbl# /opt/drbl/sbin/drblpush -i -l 0 ****************************************************** Now we can collect the MAC address of clients! If you want to let the DHCP service in DRBL server offer same IP address to client every time when client boot, and you never did this procedure, you should do it now! If you already have those MAC addresses of clients, you can put them into different group files (These files number is the same number of networks cards for DRBL service). In this case, you can skip this step. This step helps you to record the MAC addresses of clients, then divide them into different groups. It will save your time and reduce the typos. The MAC addresses will be recorded turn by turn according to the boot of clients, and they will be put into different files according to the network card in server, file name will be like macadr-eth1.txt, macadr-eth2.txt... You can find them in directory /etc/drbl. Please boot the clients by order, make sure they boot from etherboot or PXE! Do you want to collect them ? [y/N] y ****************************************************** OK! Let's do it! request_eth_port:eth1 Stopping dhcp3-server ... Stopping DHCP server: dhcpd3. Stopping tftpd-hpa ... Stopping HPA's tftpd: in.tftpd. *****************************************************. Start detecting MAC address.... Enter 1 or press Enter to view the collecting status. Enter 2 or q to finish collecting and quit. 1 ======================================= 00:0C:29:FA:20:5A 00:0C:29:EB:9E:2F 00:0C:29:1B:19:1E Total: 3 ======================================= Enter 1 or press Enter to view the collecting status. Enter 2 or q to finish collecting and quit. 2 *****************************************************. The collected MAC addresses from [eth1] are saved in file(s) separately: macadr-eth1.txt. These files are saved in directory /etc/drbl. ****************************************************** OK! Let's continue... ****************************************************** Do you want to let the DHCP service in DRBL server offer same IP address to the client every time when client boots (If you want this function, you have to collect the MAC addresses of clients, and save them in file(s) (as in the previous procedure)). This is for the clients connected to DRBL server's ethernet network interface eth1 ? [y/N] y ****************************************************** OK! Please tell me the file name which contains the MAC address of clients line by line for eth1. [macadr-eth1.txt] ****************************************************** What is the initial number do you want to use in the last set of digits in the IP (i.e. the initial value of d in the IP address a.b.c.d) for DRBL clients connected to this ethernet port eth1. [1] 10 ****************************************************** The file name you set is "macadr-eth1.txt". The clients number in this file is 3. We will set the IP address for the clients connected to DRBL server's ethernet network interface eth1 By the MAC address file you set, the IP addresses for the clients connected to DRBL server's ethernet network interface eth1 as: 192.168.0.10 - 192.168.0.12 Accept ? [Y/n] y ****************************************************** OK! Let's continue... ****************************************************** The Layout for your DRBL environment: ****************************************************** NIC NIC IP Clients +-----------------------------+ | DRBL SERVER | | | | +-- [eth0] 192.168.183.129 +- to WAN | | | +-- [eth1] 192.168.0.254 +- to clients group 1 [ 3 clients, their IP | | from 192.168.0.10 - 192.168.0.12] +-----------------------------+ ****************************************************** Total clients: 3 ****************************************************** Press Enter to continue... }}} * You need to modify two files: "/opt/drbl/sbin/drbl-nfs-exports" and "/opt/drbl/sbin/drbl-gen-client-files".[[BR]][[BR]] * (1) /opt/drbl/sbin/drbl-nfs-exports {{{ In "for subnet in $subnet_list" loop, you should add one line. /usr/local/globus-4.2.0 $subnet.*($EXPORTS_NFS_RO_NRS_OPT) In "for ip in `get-client-ip-list`" loop, you should also add one line. /usr/local/globus-4.2.0 $ip($EXPORTS_NFS_RO_NRS_OPT) We assume that $GLOBUS_LOCATION is /usr/local/globus-4.2.0. }}} * (2) /opt/drbl/sbin/drbl-gen-client-files {{{ You only need to add one line. $nfsserver:/usr/local/globus-4.2.0 /usr/local/globus-4.2.0 nfs $FSTAB_NFS_RO_OPT }}} * After modifying the two files, you should run "/opt/drbl/sbin/drblpush -c /etc/drbl/drblpush.conf" to replace current /etc/exports. == '''''References''''' == * [http://www.globus.org/toolkit/docs/4.0/admin/docbook/quickstart.html GT 4.0.x Quickstart] * [http://www.globus.org/toolkit/docs/4.2/4.2.0/admin/quickstart/index.html GT 4.2 Quickstart] * Bugzilla Bug 4431 "freeze by Unsubmitted on personal WS GRAM" * [http://drbl.nchc.org.tw/ DRBL] * "Introduction to Cluster and grid Computing" by Jazz Yao-Tsung Wang * [https://trac.nchc.org.tw/grid/wiki/jazz/DRBL_Xen#KnowIssueofDRBLwithXen Know Issue of DRBL () with Xen]