The labrun Tool

The program labrun provides a means to run experiments and automatically log the context of the experiments.

labrun [<options>] <executable> [<arguments>]

The program <executable> should run non-interactively. labrun will first check whether the provided directories (the default is just the current directory) is under CVS and up to date and then, if so, executes the program <executable> with the given arguments <arguments> (if any). <options> might be used to control the behaviour of labrun. Type labrun --help to get more help.

In general, labrun generates at least two files: a .out file that contains the output of the experiment and a .log file that contains certain environment information. Additionally, if the program produces some error messages (written to the standard error stream), these will be recorded in a .err file. All file names are of the form

<executable>-<date>-<time>.[out|log|err]

and can be found in a subdirectory ./lab_log. A link current.[out|log|err] will point to the files for the latest experiment performed.

In the following tutorials, we illustrate how various options of labrun might be used in conjunction with labmex.

$\Rightarrow$ Tutorial 4:

Call labrun on labmex to compile and execute sort-demo with $n=5000$ and $r=10$. Then, check the output file current.out and the log file current.log in ./lab_log.

Solution:

labrun labmex ./sort-demo 5000 10
cd lab_log
... look at the .log and .out files ...

$\Rightarrow$ Tutorial 5:

Find out which option is used to let labrun automatically commit and update the CVS repository if necessary.

Solution:

labrun -a labmex ./sort-demo 5000 10

$\Rightarrow$ Tutorial 6:

Write an option file labrun.opt1 for labrun such that the call

labrun @ labrun.opt1 labmex ./sort-demo 5000 10

does the following: (1) The log files are put into a directory ./res. (2) The tag 5000-10 is used instead of the <date>-<time> stamp. (3) The experiment is executed in batch mode. (4) All data are stored in only one log file.

Solution:

The corresponding file labrun.opt1 would look like this:

labrun.opt1

--log=./res
--tag=5000-10
--batch
--one

$\Rightarrow$ Tutorial 7:

Use the option --comment to add your own comments to the log file. (Type labrun --help-comment to get help.) Try to record the value of some environment variables, e.g., CXXFLAGS, in the log file.

Solution:

The following option file will add a table detailing the free and allocated dynamic memory on the system (result of free) and the version of gcc (result of gcc -v) as a comment to the log file. Moreover, it records the value of the environment variable CXXFLAGS.

labrun.opt2

--comment=Memory=`free`
--comment=Compiler=`gcc -v`
--env=CXXFLAGS

Several experiments might be started by calling labrun several times. However, it is more convenient to put all calls into one file and to separate different experiments via the keyword NEX.

$\Rightarrow$ Tutorial 8:

Write an option file labrun.opt3 such that the call

labrun @ labrun.opt3

performs the following set of experiments: (1a) sort-demo with $n=5000$ and $r=10$. (1b) sort-demo is compiled with the additional optimization flag -O3 and run with $n=5000$ and $r=10$. (2a) sort-demo with $n=10000$ and $r=10$. (2b) sort-demo is compiled with the additional optimization flag -O3 and run with $n=10000$ and $r=10$. Make sure that each call does indeed recompile the source code. Redefine the <date>-<time> tag such that it encodes each experiment's arguments, e.g., you might use 10000-10-O3 for (2b). The log files of all experiments should keep track of the version number of the compiler used. Additionally, each experiment should be launched in verbose mode. All experiments should run in batch mode.

Solution:

labrun.opt3

# experiment (1a)
--comment=Compiler=`gcc -v`
--tag=5000-10-no-O
--verbose
--batch
labmex -c before ./sort-demo 5000 10

NEX

# experiment (1b)
--comment=Compiler=`gcc -v`
--tag=5000-10-O3
--verbose
--batch
labmex -c before -m LFLAGS=-O3 ./sort-demo 5000 10

NEX

# experiment (2a)
--comment=Compiler=`gcc -v`
--tag=10000-10-no-O
--verbose
--batch
labmex -c before ./sort-demo 10000 10

NEX

# experiment (2b)
--comment=Compiler=`gcc -v`
--tag=10000-10-O3
--verbose
--batch
labmex -c before -m LFLAGS=-O3 ./sort-demo 10000 10

Each of the above four experiments uses the three flags --comment=Compiler=`cc-v', --verbose, and --batch. If all your experiments are meant to be run with a common set of command-line options, a global resource file can be created that contains these common options. The program labsetup is available to help you create this resource file. The options recorded in this file will be read will labrun is executed and added to the command-line arguments. In the case of conflicts, command-line arguments take precendence over those recorded in a resource file. For options that may appear more than once on the command line, the union of the command-line and resource file options is used.

$\Rightarrow$ Tutorial 9:

Run the program labsetup to create a global resource file that indicates that labrun should always run in verbose mode and in batch mode and should always record the version of the compiler cc.

Solution:

labsetup session

[...]

labsetup: Settings for labrun

Execute labrun in verbose mode? [n]
y

[...]

Run in the background? [n]
y

[...]

Additional comments to be recorded in log file
   Enter new values one per line.

   Use a '+' at the beginning of the first line to add to the current
   list; otherwise any new values given will replace the current ones.

   An empty line ends the input.
[None]
Compiler='gcc -v'    

[...]

$\Rightarrow$ Tutorial 10:

Now that a global resource file has been created, modify the option file created for Tutorial  by removing the common options of the four experiments and rerun this experiment, putting the resulting log files in a different directory, say lab_log2. Examine the log files from the two sets of experiments to see that the same information is recorded in both.

Solution:

labrun.opt4

# experiment (1a)
--tag=5000-10-no-O
--log=lab_log2
labmex -c before ./sort-demo 5000 10

NEX

# experiment (1b)
--tag=5000-10-O3
--log=lab_log2
labmex -c before -m LFLAGS=-O3 ./sort-demo 5000 10

NEX

# experiment (2a)
--tag=10000-10-no-O
--log=lab_log2
labmex -c before ./sort-demo 10000 10

NEX

# experiment (2b)
--tag=10000-10-O3
--log=lab_log2
labmex -c before -m LFLAGS=-O3 ./sort-demo 10000 10