Directory structure
 * benchmark-runner: experiment scripts
 * benchmarks: benchmark set
 * cbmc: CBMC/CEGIS source directory

"compile-and-run-experiment.sh" compiles CBMC/CEGIS, runs the full benchmark set with the two-stage engine and displays the results in CSV format on the console output. To execute it, type the following comands:

chmod +x compile-and-run-experiment.sh
./compile-and-run-experiment.sh


The script only shows a summary of runtimes and success status in the CSV file, which is the same result data as we provide in the paper.  All further details (e.g. the synthesised controller coefficients) can be found in the generated log files in each benchmark directory.  Please note that running the benchmarks is very time-consuming; the expected runtime for the whole benchmark set in the two-step engine is 24-48h, and 48-96h for the one-step engine (where the majority of benchmarks time out).  The one-step back-end is disabled by default in compile-and-run-experiment.sh and can be enabled on line 18 in the script.

The benchmark files are comprised in a single .c file which is optimised for input into CBMC.  Each benchmark contains the same closed loop system implementation, expressing the controller stability property as described in the paper.  Features which change for each benchmark file are the globally declared struct variables "impl" (contains controller precision), "controller" (controller coefficients to replace by stable values), "plant" (plant coefficients and uncertainty).

Requirements:
 * Linux OS
 * GNU bash >= 4.3
 * gcc and g++ (>= 6.3.1 recommended)
 * GNU Bison >= 3.0.4
 * GNU Make >= 4.1
 * GNU patch >= 2.7.5
 * flex >= 2.6.0
 * git >= 2.9.3
 * libzip >= 1.1.3
 * zlib >= 1.2.11
