Testing Inkscape

Testing Inkscape

Inkscape is a young project and the emphasis is still on adding features. Nonetheless it is gratifying that the stability of Inkspace has been steadily rising with each release.

The most important part of 'Testing' is simply to use Inkscape for normal work -- confirming that Inkscape has reached this level of maturity, exercise the new features and verify that the application works as expected.

Report a bug if you find anything that does not behave as it should. A bug report should include at least a step-by-step description of how to trigger the bug and/or a test file that demonstrates the bug (the smaller/more focussed the test file the better).

? Follow up

Users

The field is wide open. We are keen to receive bug reports and feature requests (in the form of a bug report). These often require analysis, clarification and further action. Anyone can do this. Better still would be to provide patches for any part of the application that is not up to the standard you expect - it is confirmation that the project is evolving. Note that serious testing should be done with an 'unstable' build, either one that you made yourself (see BuildingInkscape), or a snapshot that you have downloaded. We would also like to hear about areas in which we do not have parity with comparable applications. If you find that you are coming up with interesting ideas concerning shortcomings in Inkscape, or plans for its future, get involved with the Inkscape testers group.

We need people to create and update documentation, online help, tutorials and screen shots. Noting defects in these is a perfectly valid form of testing - we do not want releases to go out with obsolete documentation.

Inkscape Testers

A community of Inkscape testers has grown up which has its own mailing list, and it is to be hoped that this will spearhead all work on usability and human factors. This group should be your first port of call for these areas:

• ComplianceTesting
• RegressionTesting
• InteroperabilityTesting
• UsabilityTesting
• PerformanceTesting
• HIG compliance

See also TestingFramework.

Note: Bryce? Jon? shouldn't the whole of that page be merged here? Or is it better to have this info in two pieces. IMHO wiki pages should not be made too long.

Developers

Running unit tests

There are now some UnitTests which should be performed before checking in. These may take some time to complete, and so this cannot be made a requirement for each build (Test Driven Development), nonetheless everyone is on their honour not to 'break the build' by committing code that does not pass these tests. You can execute them by:

• Linux: Just run 'make check', it will build and run them. (Does this work on MacOS X?)
• Windows: Use 'buildtool distcxxtests' (where buildtool is built using 'g++ -O3 -o buildtool buildtool.cpp) to build the tests, then go to the 'inkscape' subdirectory that was just created and execute cxxtests.

Cxxtests will generate two (more or less equivalent) result files, an XML file and a text file with the extension 'log'. On Linux these files are created for each directory that contains unit tests, on Windows only one of each is created (cxxtests.log/xml).

Creating unit tests

Inkscape uses the CxxTest framework. To enhance, modify or extend existing unit tests, just edit the existing test file (....-test.h).

The easiest way to create a new test in a directory which already has some unit tests is to simply copy one of the existing test files, strip it (remove anything specific and rename the class, constructors, etc.) and add some test methods. Take the time to look at the different ASSERT statements CxxTest supports, the TSM_ variants can be especially useful for example when you want to test a lot of different cases. Important: to make everything build correctly you have to do the following:

• Add the file to the right (already existing) group in the cxxtest target in build.xml
• Add the file to the dir_test_dir_includes variable in dir/Makefile_insert (watch the backslashes at the end of the lines!).

For creating a unit test in a directory which does not have any unit tests yet you basically proceed in the same way as above, except that you now also have to:

• Add a cxxtestpart group to the cxxtest target in build.xml (just copy and modify an existing one).
• Add the corresponding .o file to the exclude list of the lib target in build.xml and to the include list of the linkcxxtests target.
• Add some boiler-plate stuff to dir/Makefile_insert (see existing Makefile_insert's, for example in display/, svg/ and xml/). Be sure to rename everything correctly and that the list of includes and the ..._LDADD variable are correct.
• Make the following changes to Makefile.am (watch the backslashes at the end of lines!):
  • Add dir/libtest-dir.a to check_LIBRARIES
  • Add dir/test-dir$(EXEEXT) to TESTS
  • Add dir/test-dir to check_PROGRAMS
  • Add dir/test-dir.log and dir/test-dir.xml to the commandline in distclean-local
  • TODO: Someone who knows exactly how Makefiles work should consider streamlining this process.

Build report

There is an 'inkscape build report; which is sent regularly to the inkscape-tester list (and periodically to the developer list, when new problems are seen) that gives a count of warnings spotted in the code.

• Smoketests
• Defects in the build system