labschedule

Purpose:

starts multiple commands (possibly on multiple hosts or on a multiprocessor environment)

Usage:

labschedule [<option>] ... <executable> [<arguments>]

Options:

-f, --for=<WORDLIST>

loop through all words from <WORDLIST>; in the command line the corresponding loop variable (%1, %2, ...) will be replaced by words from <WORDLIST>. (See the 'for-loop' section below for details.)

--nesting=<N>

only the first <N> loops will form individual 'labrun' calls; the remaining loops will incorporated into one experiment (with executable labschedule --direct ... <executable> ...)

--macro=<LABEL=WORDLIST>

define macro (%LABEL will be expanded to WORDLIST)

--hosts=<HOSTLIST>

execute on hosts <HOSTLIST> (default: localhost)

--maxtasks=<N>

maximum tasks started on one host (default: 1)

--check=<EXP>

condition to test to determine if host can accept another task (typically for multi-user hosts; see the 'check' section below)

-n, --name=<TAG>

<TAG> for log files (default: 'schedule')

-l, --log=<DIR>

<DIR> for log files (default: ./lab_log)

-i, --ignore

ignore error codes from failed experiments, continuing with the rest of the loop (default: abort)

--keep

keep old (incomplete, failed) log files (default: delete them)

--noskip

don't skip previously performed runs (default: skip)

--prefix=COMMAND

prefix for each command (default: ssh %host; cd %curdir)

--nolabrun

don't use 'labrun --log=%logdir --name=%name-%1-%2-...' as command executed

--labrunflag=<OPTION>

additional labrun option; use multiple --labrunflag options for multiple labrun options

-d, --direct

execute loops directly (see 'direct' section below)

-b, --batch

run in background

-p, --print

do not perform the commands, just print them

-v, --verbose

verbose mode

--nolabrc

do not read ~/.labrc and ./labrc

With @FILE or @ FILE (some) command-line options are read from FILE (see section ).

Result:

For every combination of the words in each of the --for commands, an experiment is started. In addition to the log and output files produced by labrun, labschedule keeps track of its own actions in three files: a .log file that logs all relevant actions, a .out file that holds the output of all successful runs, and a .err file that holds the output of all failed runs.

Defining For-Loops:

The word lists for the for-loops can be defined in several ways. The following expressions are possible:

$<ENVVAR>

will be replaced by shell variable <ENVVAR>,

@<FILE>

will be replaced by file <FILE>,

`<COMMAND>`

will be replaced by output of <COMMAND>.

Each of the three statements may be followed by a modifier:
:<PATTERN>
If the regular expression <PATTERN> contains parentheses, only the match corresponding to the parentheses will be taken. Otherwise the first word after the regular expression will be taken. The <PATTERN> may not contain spaces. Use \s instead. Some documentation for regular expressions is given in Section .

Furthermore, the characters ~,* or ? in words will be interpreted as a shell would.

eval(EXPRESSION) will be replaced by the evaluation of the python EXPRESSION, range(X) is a shortcut for eval(range(X)) and will be replaced by the words $0,1,2,3,\dots,(\mbox{\texttt X}-1)$. The functions of the python modules ``random'' and ``math'' can be used.

Expansion of Variables:

Some variables are expanded before executing a command. These can be used in --check, --labrunflag, --prefix, --for, and in the arguments to the experiment.

%curdir

the current directory

%logdir

full path to the log directory

%host

the host for the current experiment

%prefix

either ssh %host cd %curdir; or an empty string

%name

the name, provided by --name (default: schedule)

%maxtasks

number of tasks per host, provided by --maxtasks

%nesting

nesting level, provided by --nesting

%1, %2, ...

will be replaced by the current word from the first, second, and so on --for word list. Here one can use modifiers similar to those of tcsh, (e.g., %1:h). Example (if %1 gives /dir/name.ext):

h)ead	(directory)	%1:h	/dir
t)ail	(name)	%1:t	name.ext
e)xt	(extension)	%1:e	ext
r)oot	(skip ext)	%1:r	/dir/name
s)ubst	(regular expression substitution /from/to/, any character may be delimiter)	%1:s-e(.)-\1E-	/dir/nam.ExEt

More variables can be defined by the --macro=LABEL=WORDLIST option. Valid LABELs are alphanumeric and begin with a letter. Two predefined macros are:

• %idle=`%prefix vmstat 1 2`:(?s).*\D(\d+)
  reads the host's idle percentage (useful for --check) ((?s) makes .* match also newlines, \D is [^0-9] and (\d+) returns the last number printed by vmstat.)
• %check=%idle > 5
  (useful for --check=%check on multi-user systems, see below)

Checking Idle Time:

By default, every host gets assigned %maxtasks tasks. On multiuser systems it may be advisable to check if the host can accept a further task. This can be done with the --check switch:

--check=<EXP>

expression to check, if host can accept another task.

The results of commands, contents of files, and values of environment variables can be incorporated into EXP in the same way they can for for loop values. (See two sections above).

Example:

• --check=`ssh %host w`:load.average < %maxtasks
  reads the load of the host using the w command.
• --check=`ssh %host vmstat 1 2`:[^@]*([0-9]+) > 5
--check=`%prefix vmstat 1 2`:[^@]*([0-9]+) > 5
--check=%idle > 5
  --check=%check
  reads the idle percentage of the host using the vmstat command (all variants resolve to the same command).

Using the --direct switch:

The --direct switch is used internally for performing multiple runs of a program as one experiment when --nesting is used. If both --direct, and --nesting=<LEVEL> are given, labschedule assumes that <LEVEL> loops have already be processed (i.e., the first --for of the command line will be treated as <LEVEL>+1). The main benefit from using --nesting is that output of multiple commands is put in one log and output file.

Nevertheless, the switch is useful for completely different things:

• labschedule --direct --for=*.ps ps2pdf %1 %1:r.pdf
  call ps2pdf on all *.ps files of the current directory
• labschedule -d -f*.jpg sh -c 'djpeg %1|pnmscale 0.5|cjpeg>sml/%1'
  make small jpgs in a different directory (we have to add sh -c because we want the special characters | and > (see Section ) to be interpreted by shell)
• labschedule --direct --for='hert polzin kettner schaefer' finger %1
  finger the explab developers

Examples:

• labschedule --for="def1 def2 def3" -for=range(5) isp %1
  Single host: Start the program isp with the three different parameters, and each command 5 times, summing to 15 executions.
• labschedule --hosts="comp1 comp2 comp3 comp4" -for=*.stp steiner %1
  Compute cluster: For each *.stp file in the current directory, start a command. Use the four mentioned hosts to perform the runs.
• labschedule --maxtasks=20 -check=%check -for=eval(map(lambda s:
  s**2,range(100))) --for=range(2,11,2) calc %1 %2
  Multiprocessor, multi-user system: Start calc with the first 100 square numbers, and for each square number with the five even numbers between 2 and 10 (500 executions in total). Commit maximal 20 tasks at a time, and check if there is still some idle time on the processors.
• labschedule --for="D E F" --for=/dat/%1/*.stp --nesting=1
--labrunflag=--cvs=~/src --labrunflag=--autocvs steiner %2
  For each of the three directories ~/dat/D, ~/dat/E, and ~/dat/F, start an experiment (with the given labrun options. In each of these experiments, start steiner with every *.stp file in the current directory.
• labschedule --for="eval(map(lambda s:randint(1,1000),range(20)))" gugl %1
  Start gugl with 20 random integers between 1 and 1000.