== '''''Data Challenge''''' == * （宗旨？）：先構想出較有利的機制並透過設計來調整Torque、GXP和Gfarm的運作，盡可能降低資料搬移量並且能取得最佳資源執行工作，更能在某資源fail時，能儘速交由另一資源來接續其工作，而不是重新執行此工作，來達到seamlessly完成工作，更進而加快完成速度。 * 簡易測試範例 * '''$ gxpc e --help''' {{{ Usage: gxpc e [OPTION ...] CMD gxpc mw [OPTION ...] CMD gxpc ep [OPTION ...] file Description: Execute the command on the selected nodes. Option (for mw only): --master 'command' equivalent to e --updown '3:4:command' ... if --master is not given, it is equivalent to e --updown 3:4 ... Options (for e, mw, and ep): --withmask,-m MASK execute on a set of nodes saved by savemask or pushmask --withhostmask,-h HOSTMASK execute on a set of nodes whose names match regexp HOSTMASK --withhostnegmask,-H HOSTMASK execute on a set of nodes whose names do not match regexp HOSTMASK --up FD0[:FD1] collect output from FD0 of CMD, and output them to FD1 of gxpc. if :FD1 is omitted, it is treated as if FD1 == FD0 --down FD0[:FD1] broadcast input to FD0 of gxpc to FD1 of CMD. if :FD1 is omitted, it is treated as if FD1 == FD0 --updown FD1:FD2[:MASTER] if :MASTER is omitted, collect output from FD1 of CMD, and broadcast them to FD2 of CMD. if :MASTER is given, run MASTER on the local host, collect output from FD1 of CMD, feed them to stdin of the MASTER. broadcast stdout of the MASTER to FD1 of CMD. --pty assign pseudo tty for stdin/stdout/stderr of CMD By default, - stdin of gxpc are broadcast to stdin of CMD - stdout of CMD are output to stdout of gxpc - stderr of CMD are output to stderr of gxpc This is as if `--down 0 --up 1 --up 2' are specified. In this case, stdout/stderr are block-buffered by default. You may need to do setbuf in your program or flush stdout/err, to display CMD's output without delay. --pty overwrites this and turn them to line-buffered (by default). both stdout/err of CMD now goto stdout of gxpc (they are merged). CMD's stdout/err should appear as soon as they are newlined. See Also: smask savemask pushmask rmask restoremask popmask }}} * $ '''gxpc use --help''' {{{ Usage: gxpc use [--as USER] RSH_NAME SRC [TARGET] gxpc use --delete [--as USER] RSH_NAME SRC [TARGET] gxpc use gxpc use --delete [idx] Description: Configure rsh-like commands used to login targets matching a particular pattern from hosts matching a particular pattern. The typical usage is `gxpc use RSH_NAME SRC TARGET', which says gxp can use an rsh-like command RSH_NAME for SRC to login TARGET. gxpc remembers these facts to decide which hosts should issue which commands to login which hosts, when explore command is issued. See the section of explore command and the tutorial section of the manual. Examples: gxpc use ssh abc000.def.com pqr.xyz.ac.jp gxpc use ssh abc000 pqr gxpc use ssh abc gxpc use rsh abc gxpc use --as taue ssh abc000 pqr gxpc use qrsh abc gxpc use qrsh_host abc gxpc use sge abc gxpc use torque abc The first line says that, if gxpc is told to login pqr.xyz.ac.jp by explore command, hosts named abc000.def.com can use `ssh' method to do so. How it translates into the actual ssh command line can be shown by `show_explore' command (try `gxpc help show_explore') and can be configured by `rsh' command (try `gxpc help rsh'). SRC and TARGET are actually regular expressions, so the line like the first one can often be written like the second one. The first line is equivalent to the second line as long as there is only one host begining with abc000 and there is only one target beginning with pqr. In general, the specification: gxpc use RSH_NAME SRC TARGET is read: if gxpc is told to login a target matching regular expession TARGET, a host matching regular expression SRC can use RSH_NAME to do so. Note that the effect of use command is NOT to specify which target gxpc should login, but to specify HOW it can do so, if it is told to. It is the role of explore command to specify which target hosts it should login If the TARGET argument is omitted as in the third line, it is treated as if TARGET expression is SRC. That is, the third line is equivalent to: gxpc use ssh abc abc This is often useful to express that ssh login is possible between hosts within a single cluster, which typically have a common prefix in their host names. If the traditional rsh command is allowed within a single cluster, the fourth line may be useful too. If --as user option is given, login is issued using an explicit user name. The fifth line says when gxp attempts to login pqr from abc000, the explicit user name `taue' should be given. You do not need this as long as the underlying rsh-like command will complement it by a configuration file. e.g., ssh will read ~/.ssh/config to complement user name used to login a particular host. qrsh_host uses command qrsh, with an explicit hostname argument to login a particular host (i.e., qrsh -l hostname=...). This is useful in environments where direct ssh is discouraged or disallowed and qrsh is preferred. qrsh also uses qrsh, but without an explicit hostname. The host is selected by the scheduler. Therefore it does not make sense to try to speficify a particular hostname as TARGET. Thus, the effect of the line gxpc use qrsh abc is if targets beginning with abc is given (upon explore command), a host beginning with abc will issue qrsh, and get whichever host is allocated by the scheduler. See Also: explore conf_explore }}} * $ '''gxpc explore --help''' {{{ Usage: gxpc explore [OPTIONS] TARGET TARGET ... Description: Login target hosts specified by OPTIONS and TARGET. Options: --dry dryrun. only show target hosts --hostfile,-h HOSTS_FILE give known hosts by file --hostcmd HOSTS_CMD give known hosts by command output --targetfile,-t TARGETS_FILE give target hosts by file --targetcmd TARGETS_CMD give target hosts by command output --timeout SECONDS specify the time to wait for a remote host's response until gxp considers it dead --children_soft_limit N (>= 2) control the shape of the explore tree. if this value is N, gxpc tries to keep the number of children of a single host no more than N, unless it is absolutely necessary to reach requested nodes. --children_hard_limit N control the shape of the explore tree. if this value is N, gxpc keeps the number of children of a single host no more than N, in any event. --verbosity N (0 <= N <= 2) set verbosity level (the larger the more verbose) --set_default if you set this option, options specified in this explore becomes the default. for example, if you say --timeout 20.0 and --set_default, timeout is set to 20.0 in subsequent explores, even if you do not specify --timeout. --reset_default reset the default values set by --set_default. --show_settings show effective explore options, considering those given by command line and those specified as default values. Execution of an explore command will conceptually consist of the following three steps. (1) Known Hosts: Know names of existing hosts, either by --hostfile, --hostcmd, or a default rule. These are called 'known hosts.' -h is an acronym of --hostfile. (2) Targets: Extract login targets from known hosts. They are extracted by regular expressions given either by --targetfile, --targetcmd, or directly by command line arguments. -t is an acronym of --targetfile. (3) gxpc will attempt to login these targets according to the rules specified by `use' commands. Known hosts are specified by a file using --hostfile option, or by output of a command using --hostcmd. Formats of the two are common and very simple. In the simplest format, a single file contains a single hostname. For example, hongo001 hongo002 hongo004 hongo005 hongo006 hongo007 hongo008 is a valid HOSTS_FILE. If you specify a command that outputs a list of files in the above format, the effect is the same as giving a file having the list by --hostfile. For example, --hostcmd 'for i in `seq 1 8` ; do printf "%03d\n" $i ; done' has the same effect as giving the above file to --hostfile. The format of a HOSTS_FILE is actually a so-called /etc/hosts format, each line of which may contain several aliases of the same host, as well as their IP address. gxpc simply regards them as aliases of a single host, wihtout giving any significance to which columns they are in. Anything after `#' on each line is a comment and ignored. Lines not containning any name, such as empty lines, are also ignored. The above simple format is obviously a special case of this. It is sometimes convenient to specify /etc/hosts as an argument to --hostfile or to specify `ypcat hosts' as an argument to --hostcmd. As a matter of fact, if you do not specify any of --hostfile, --hostcmd, --targetfile, and --targetcmd, it is treated as if --hostfile /etc/hosts is given. Login targets are specified by a file using --targetfile option, --targetcmd option, or by directly listing targets in the command line. Format of them are common and only slightly different from HOSTS_FILE. The format of the list of targets in the command line is as follows. TARGET_REGEXP [N] TARGET_REGEXP [N] TARGET_REGEXP [N] ... where N is an integer and TARGET_REGEXP is any string that cannot be parsed as an integer. That is, it is a list of regular expressions, each item of which may optionally be followed by an integer. The integer indicates how many logins should occur to the target matching TARGET_REGEXP. The following is a valid command line. gxpc explore -h hosts_file hongo00 which says you want to target all hosts beginning with hongo00, among all hosts listed in hosts_file. If, for example, you have specified by `use' command that the local host can login these hosts by ssh, you will reach hosts whose names begin with hongo00. If you instead say gxpc explore -h hosts_file hongo00 2 you will get two processes on each of these hosts. If you do not give any of --targetfile, --targetcmd, and command line targets, it is treated as if a regular expression mathing any string is given as the command line target. That is, all known hosts are targets. Format of targets_host is simply a list of lines each of which is like the list of arguments just explained above. Thus, the following is a valid TARGETS_FILE. hongo00 2 chiba0 istbs sheep which says you want to get two processes on each host beginning with hongo00 and one process on each host beginning with chiba0, istbs, or sheep. Just to illustrate the syntax, the same thing can be alternatively written with different arrangement into lines. hongo00 2 chiba0 istbs sheep Similar to hosts_file, you may instead specify a command line producing the output conforming to the format of TARGETS_FILE. We have so far explained that target_regexp is matched against a pool of known hosts to generate the actual list of targets. There is an exception to this. If TARGET_REGEXP does not match any host in the pool of known hosts, it is treated as if the TARGET_REGEXP is itself a known host. Thus, gxpc explore hongo000 hongo001 will login hongo000 and hongo001, because neither hosts_file nor hosts_cmd hosts are given so these expressions obviously won't match any known host. Using this rule, you may have a file that explicitly lists all hosts and solely use it to specify targets without using separate HOSTS_FILE. For example, if you have a long TARGETS_FILE called targets like: abc000 abc001 ... abc099 def000 def001 ... def049 pqr000 pqr001 ... pqr149 and say gxpc explore -t targets you say you want to get these 300 targets using whatever methods you specified by `use' commands. Unlike HOSTS_FILE, an empty line in TARGETS_FILE is treated as if it is the end of file. By inserting an empty line, you can easily let gxpc ignore the rest of the file. This rule is sometimes convenient when targeting a small number of hosts within a TARGETS_FILE. Here are some examples. 1. gxpc explore -h hosts_file chiba hongo Hosts beginning with chiba or hongo in hosts_file become the targets. 2. gxpc explore -h hosts_file -t targets_file Hosts matching any regular expression in targets_file become the targets. 3. gxpc explore -h hosts_file All hosts in hosts_file become the targets. Equivalent to `gxpc explore -h hosts_file .' (`.' is a regular expression mathing any non-empty string). 4. gxpc explore -t targets_file All hosts in targetfile become the targets. This is simiar to the previous case, but the file format is different. Note that in this case, strings in targets_file won't be matched against anything, so they should be literal target names. 5. gxpc explore chiba000 chiba001 chiba002 chiba003 chiba000, chiba001, chiba002, and chiba003 become the targets. 6. gxpc explore chiba0 Equivalent to `gxpc explore -h /etc/hosts chiba0' which is hosts beginning with chiba0 in /etc/hosts become the targets. Useful when you use a single cluster and all necessary hosts are listed in that file. 7. gxpc explore Equivalent to `gxpc explore -h /etc/hosts' which is in turn equivalent to `gxpc explore -h /etc/hosts .' That is, all hosts in /etc/hosts become the targets. This will be rarely useful because /etc/hosts typically includes hosts you don't want to use. }}} * 以在chiba這個cluster裡為例：(機器有chiba000-157，用到chiba000-003) {{{ $ dach005@chiba000:~$ gxpc e hostname （用gxpc e 來帶入要使用的指令"hostname"，在一開始無login到其他hosts時先查看） chiba000 $ dach005@chiba000:~$ gxpc use ssh chiba （表示可使用ssh指令login 到chiba cluster裡的hosts，即source hosts和target hosts都在此cluster裡） $ dach005@chiba000:~$ gxpc use (可看目前可ssh 的列表) 0 : use ssh chiba chiba $ dach005@chiba000:~$ gxpc explore chiba00[[ 1-3 ]] （可用explore指令來達到真的login到遠端機器裡）（因為語法問題，下指令時無需空格） reached : chiba001 reached : chiba002 reached : chiba003 $ dach005@chiba000:~$ gxpc e hostname （再看一次，可以發現目前我們可以reach到的機器列表） chiba000 chiba003 chiba002 chiba001 $ dach005@chiba000:~$ qsub test_3.sh （Torque 沒搭配GXP時，僅可以在本端機器這邊下命令） 64.chiba000.intrigger.nii.ac.jp $ dach005@chiba000:~$ gxpc e qsub test_3.sh （搭配torque 我們發現就可以同時在本機及遠端機器下執行指令--> chiba000-003是GXP的執行nodes而非執行Torque的執行nodes） 67.chiba000.intrigger.nii.ac.jp 68.chiba000.intrigger.nii.ac.jp 65.chiba000.intrigger.nii.ac.jp 66.chiba000.intrigger.nii.ac.jp }}} * gxpc smask 和 gxpc rmask 的指令測試 (用來挑選某些特定節點) * 可以參考此網址[http://209.85.141.104/search?q=cache:rM5RM5afALIJ:www.ueda.info.waseda.ac.jp/~miwa/pukiwiki/index.php%3F%25B8%25A6%25B5%25E6%25C6%25FC%25BB%25EF+gxpc+smask+rmask&hl=zh-TW&ct=clnk&cd=4&gl=tw 點我] * 在網頁下方(2007年10月13日這篇)執行mpi的例子說明 * 挑選步驟 * 1.送出一個指令，使得在所想選得那些節點上會成功執行的 * 2.送出smask，使得把其它沒成功的節點mask住 * 3.可以用e hostname指令來做確認那些節點是確切被挑選到的 * 4.送出rmask，恢復那些被mask住的節點 * 指令說明 * "gxpc e hostname | grep chiba00[ 02 ]" (只要chiba000和002時)、"e uptime | awk '{ if ($(NF-2) > 0.5) print "H" }' | grep H" 的功能：可挑出我們想要的節點。 * 中括號沒有空格，因為語法關係才將02前後空格 * "gxpc smask" 的功能：用來挑選那些上一個指令執行失敗的節點，把它們set mask。 * "gxpc e hostname" 的功能：用來double check，看目前有哪些節點在選擇中 * "gxpc rmask" 的功能：用來reset mask 回復到原本預設挑選的所有被reached 到的節點。 * 迷思 * 從網頁中的測試範例，我們可以發現他的smask及rmask確實有在運作，但是我有照著做，卻發現沒有作用，似乎無法挑出我要的節點，上一個指令沒有被挑選到的節點（即command失敗）也沒被mask住。 * 不知是否我有測試錯的地方。（待查）