Advanced FreeCAD test system

This page is dedicated to the Google Summer of Code project regarding the enhancement of FreeCADs test system.

Outline

FreeCAD as an CAE application has a high level of complexity, both in its source code and also in its user interaction. To ensure a certain level of quality automatic testing is essentail. However, as an open source application with spare time coders only this part of the project has not seen very much attention. One of the major reasons is the low-level handling required to write test cases. All actions to trigger, every result fetching and every single comparison needs to be hand coded. This makes it cumbersome to provide a test for every created functionality and patially impossible to do so if deep document comparisons are needed. For example he Part an PartDesign workbench: An automated test for document objects require the resulting topology shape to be analysed. This is an tremendous part and cannot be handled on a per test basis.

This project aims at reducing the work required to write meaningful tests. This should be accomplished by providing a infrastructure for result file storage and special "comparators" which compare the stored result files with the test result for equality.

Details

1. Create a result file infrastructure for the test system. It should allow to save an arbitrary number of files together with the test itself where the expected results are stored. It is intended to have one result file for each comparator used. The infrastructure should make the storage (file structure), loading and handling easy. It furthermore should define a specification for the generic file content, e.g. which comparator to use for it etc.
2. Create a infrastructure for comparators and provide a few important ones. The comparators should be able to read in a result file and compare the available test output with it. As every workbench requires different types of comparisons the comparators need to be provided by the workbench itself, as well as possible top-level ones. The test infrastructure needs to be adopted to work with such wokrbench specific types. Furthermore there needs to be a way to generate result files for comparators. this can be done either by themself or by a different class.
   1. Implement a global comparator for the document structure: It stores the Document object structure with all properties in a result file and compares the available document after a test run with it
   2. Implement a Part workbench comparator for shapes: It stores data about a certain TopoShape in a result file suitable for comparison, e.g. number of edges/vertices/faces, properties like area, mass, center of gravity.
3. Create a Wiki page describing the working of the test system and how to create tests and new comparators