System testing scientific software
Extracting hydrocarbons from the bottom of the North Sea is a complex business, consequently lot of effort is put into making this as safe as possible. Most marine operations are these days simulated using specialized software before they are executed. In order to help ensuring that these simulations are giving the correct results we are adding system tests in addition to the usual unit and integration tests.
In this presentation I would like to show how we at the Norwegian Marine Research Institue (MARINTEK) use Xtext to implement a mechanism to describe such tests. Test specifications are written in a language very similar to Markdown, intermingled with basic scripting and assertions. When processed, tests will be executed and documentation produced. This documentation is automatically included in the release notes as a part of the final PDF document.
The tests specification editor and tools are integrated into the product itself, enabling end users of the system to write their own tests as well. This can be used to verify that their own simulations produce the same results after changes to the software stack.