Using GBteSt

The main script is GBteSt.py. Mandatory and optional command-line options are provided. An help message can be printed using:

> ./GBteSt.py --help
usage: GBteSt.py [-h] [-r] [-c] --rootdir ROOTDIR [--rtol RTOL] [--atol ATOL] [-o OUTPUT_FORMAT] [-n OUTPUT_NAME]

options:
  -h, --help            show this help message and exit
  -r, --run             Run test scripts
  -c, --compare         Compare test results
  --rootdir ROOTDIR     Root folder where to store all the test results
  --rtol RTOL           Tolerance for comparing relative differences with floats
  --atol ATOL           Tolerance for comparing absolute differences with floats
  -o OUTPUT_FORMAT, --output_format OUTPUT_FORMAT
                        Format for the comparison output (text or html)
  -n OUTPUT_NAME, --output_name OUTPUT_NAME
                        Name of the output file (to be stored in --rootdir)

The different options are:

  • --run: run the tests defined by the configuration files

  • --compare: compare the results obtained via the --run option. Each dataset of the result files are compared using:

    \(|result\_1 - result\_2| < a_{tol}+ r_{tol} \cdot |result\_1|\),

    where atol and rtol are defined below. Setting atol > 0 and rtol = 0 compares the results using and absolute difference, while atol = 0 and rtol > 0 compares them using a relative difference.

  • --rootdir: root directory where to store the test results and the summary file.

  • --rtol: tolerance for the relative difference.

  • --atol: tolerance for the absolute difference.

  • --output_format: format in which the summary is generated. Options are “html” or “terminal”.

  • --output_name: name of the file containing the summary of the test.

Testing workflow

  1. First, you need to setup your tests according to the [configuration page](configuration.md). Note that the executables must be already compiled.

  2. Once the test setup is configured, you can run the tests using:

    ./GBteSt.py --rootdir <rootdir> --run

    This will launch all the test parametrized in the configuration file.

  3. When the tests have all run, you can compare the results and generate a summary report using:

    ./GBteSt.py --rootdir <rootdir> --compare --rtol <rtol> --atol <atol> -o ''

    By default, this will print the summary to the terminal. If you wish to generate an HTML report, add the -o and -n options.