Output generated by experimentation programs can come in a myriad of forms. However, the output desired for analyzing or presenting results of experiments generally has one of two forms: a table of values or a graph.
Again, we do not wish to restrict users unnecessarily by insisting on a particular form of output that may be difficult to produce for certain programs. Instead, we provide a set of tools that can process text given in any form to produce a particular internal data format. This internal format, called sus (for ``Script-readable User Statistics''), is text-based and easily readable by humans (Section ) so users can produce this format directly with their programs if they wish. The tools that convert from the internal format can also process the data by performing mathematical calculations.
Users are asked to specify which data values to extract from a given input file using key words and regular expressions. The Sus Filter Tools use the powerful Python language expressions for specifying regular expressions and reformatting data. For details on Python, see http://www.python.org.
Two programs, table2sus and text2sus, convert data to the sus format. The first program can be used to convert an ASCII table to the internal format and the second converts a text file in any format to the sus format by selecting from the file the data values indicated by the user. For converting from the sus format, we provide programs that produce
All these programs act as filters by default (reading from standard input and writing to standard output), making it easy to convert any given data file to one of the supported output file formats.
cat prog.out | text2sus num_nodes time | sus2plotAlternatively, input and output files can be specified for each program using the
--outputoptions, respectively. For example, the above command could be rewritten as:
text2sus --input prog.out num_nodes time | sus2plotMultiple
--inputcommands are possible and effect a merging of the input files as if the files were catenated together one after another.
The other command-line options for these tools allow one to specify which data to extract from the file and the format in which to display the output. In the simplest form, the user specifies simply the keyword labels for the desired data, as with the above example. The program will then extract the next word or number after the label as the value to associate with this label in each data record. If you wish to extract a value other than the next word or number to associate with this label, regular expressions can be used on the command line to express the desired value. See the Python documentation or Section for more information on forming regular expressions. See Sections and for examples.
The tools that convert from a sus file allow you to manipulate the data in several ways. In particular, one can
--combineoption). By default, the average of numeric values is taken; string values are always simply concatenated.
0) are processed.
processed in the same order they appear on the command line.
For filtering, sorting, or adding new data fields, the full expresion power of
Python can be used. This includes numerical expressions:
sus2sus '--filter=float(upperbound) > float(lowerbound)*100'(filter out all records for which the upper bound is more than 100 times the lower bound, the quotes ``''' prevent that special characters (``('', ``)'' and ``>'' in this example) are interpreted by the shell, see Sections and for more information on quoting and special characters),
formatting expressions similar to
printf in C:
sus2sus '--add=formattedOpt="%-20f"%float(opt)'(add a formatted opt value, left-justified in a field of 20 characters),
and string manipulation expressions, using Python's string module:
sus2sus '--sort=float(split(time,":"))'(sort the records based on the number of hours in a time field with format HH:MM:SS. Note the float-function that has to be used to sort, e.g. 9 and 10, in the correct order).
This string module, providing functions such as join, split, strip, and replace, is particularly useful for formatting output. We also provide some extensions:
returns <then> if
<exp> is not empty (
'0' or not
defined), otherwise <else>.
sus2sus '--add=solvedText=iff(solved,"solved","not solved")'
Print opt left-justified in a field of minimal length 20 (produces the same result as the example illustrating string formatting above).
Round the time to two digits with a minimum value of 0.01
sus2sus --add 'gap=savediv(float(upVal)-float(loVal),loVal)'
If a variable that is used in an expression is not defined, it will
take on '000' as a default value. The switch
changes this behaviour: