Difference between revisions of "TestingFramework"

From Inkscape Wiki
Jump to navigation Jump to search
m (Update URL)
(Replaced content with "removed - content was outdated")
Tags: Replaced Visual edit
Line 1: Line 1:
== Inkscape Testing Framework ==
removed - content was outdated
 
In order to detect and identify bugs quickly and reliably, we should set up a testing framework for Inkscape.
 
There are several different ways to test a desktop Linux application.  Each method provides different kinds of
information about the codebase and are each valuable in different ways.  A multi-pronged approach that uses
more than one kind of testing is expected to gain the best results.
 
=== Compile Testing ===
 
In compile testing, the codebase is repetitively checked out and compiled in one or more different ways.
For example, with different compilers or for different platforms (Linux, Windows, Mac OSX, ...)  Errors in
configuration, compilation, linking, or basic executation are detected, collected, and reported on a periodic
basis (such as daily compile reports).
 
It may be possible to set up compile testing using [[SourceForge]]'s compile farm.  Another option to
investigate is the Mozilla project's "Tinderbox".
 
=== Unit Testing ===
 
Unit testing is implemented at the file level.  Each *.c or *.cpp file is provided with a test file containing
calls for each of the functions, and that exercises the API, including a wide variety of error situations such
as NULL pointers, buffer overflows, very large or very small floating point numbers, zero or negative int's,
uninitialized objects, etc.
 
The intention of unit tests is to provide sanity checks that the developer can use as they implement the
code.  Ideally, when implementing a new code file, the programmer could first implement the unit test
from a spec, and then work on implementing the code to pass the unit test; this is the development
style advocated in the Extreme Programming philosophy.
 
=== Regression Testing ===
 
In regression testing, the philosophy is to implement test cases for each bug found, and to run these
test cases against the codebase to determine that bug fixes actually fix the bug, and to ensure in the
future that the bugs do not re-occur. 
 
This can be implemented by assembling a repository of "problem" SVG files that have caused Inkscape
to error in programmatically detectable manners (such as crashes or emission of invalid SVG or PNG's).
 
=== GUI Testing ===
 
To eneable a11y in Inkscape for testing with dogtail, start Inkscape with the following command.
* GTK_MODULES=gail:atk-bridge inkscape
 
=== Fault Response Testing ===
 
In fault response testing, the application is put into various critical situations and the behavior
detected and compared against expectations.  Examples include out-of-disk-space, out-of-memory,
response to different signals, loading and saving very large files, starting up with invalid or unreadible
config files, and so forth.
 
=== Specification Compliance Testing ===
 
In spec testing, the application is run against various input cases and caused to generate output that
can be tested against "known" correct data.  For instance, the [[W3C]] produces an SVG compliance test
suite with SVG files and the expected rendered output as PNG's.  Inkscape can be run on each of the
SVG's from the commandline and caused to save the rendering as a PNG.  An image-difference tool
can then be used to calculate a diff-score between the emitted PNG and the [[W3C]] image.
 
Similarly, this process can be used against other applications such as Batik or librsvg, to check
inter-application conformance on the same SVG files.  Ideally, all SVG applications should render
a given SVG file identically. 
 
=== Performance Testing ===
 
In performance testing the application is measured as to how well it works against various criteria such
as start-up time, output quality, processing speed, etc. 
 
=== Valgrind Memory Leak Testing ===
 
What I did here is simply ran inkscape through valgrind with the following command:
 
    valgrind -v --leak-check=yes --leak-resolution=high --num-callers=15 --show-reachable=yes --log-file=inkscape.log src/inkscape
 
I then drew a few objects, edited some points and exit.
 
What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)
 
Note: Turn off Inkscape's garbage collection before starting valgrind, e.g.
    export _INKSCAPE_GC=disable
(As suggested John Bintz)[http://www.coswellproductions.com/quick-drying-paint-profiling-inkscape-using-valgrind]

Revision as of 14:40, 26 May 2025

removed - content was outdated

Retrieved from "https://wiki.inkscape.org/wiki/index.php?title=TestingFramework&oldid=123233"