TestingFramework - Revision history

Mgmax: mark as outdated

2025-05-31T11:58:47Z

mark as outdated

← Older revision		Revision as of 11:58, 31 May 2025
Line 1:		Line 1:
	removed ~~- content~~ was outdated		{{Outdated\|message=The content of this specific page was removed because it was completely outdated and no longer applicable.}}

Mgmax: Replaced content with "removed - content was outdated"

2025-05-26T14:40:02Z

Replaced content with "removed - content was outdated"

← Older revision		Revision as of 14:40, 26 May 2025
Line 1:		Line 1:
	~~== Inkscape Testing Framework ==~~		removed - content was outdated

	~~In order to detect and identify bugs quickly and reliably, we should set up a testing framework for Inkscape.~~

	~~There are several different ways to test a desktop Linux application. Each method provides different kinds of~~
	~~information about the codebase and are each valuable in different ways. A multi~~-~~pronged approach that uses~~
	~~more than one kind of testing is expected to gain the best results.~~

	~~=== Compile Testing ===~~

	~~In compile testing, the codebase is repetitively checked out and compiled in one or more different ways.~~
	~~For example, with different compilers or for different platforms (Linux, Windows, Mac OSX, ...) Errors in~~
	~~configuration, compilation, linking, or basic executation are detected, collected, and reported on a periodic~~
	~~basis (such as daily compile reports).~~

	~~It may be possible to set up compile testing using [[SourceForge]]'s compile farm. Another option to~~
	~~investigate is the Mozilla project's "Tinderbox".~~

	~~=== Unit Testing ===~~

	~~Unit testing is implemented at the file level. Each .c or .cpp file is provided with a test file containing~~
	~~calls for each of the functions, and that exercises the API, including a wide variety of error situations such~~
	~~as NULL pointers, buffer overflows, very large or very small floating point numbers, zero or negative int's,~~
	~~uninitialized objects, etc.~~

	~~The intention of unit tests is to provide sanity checks that the developer can use as they implement the~~
	~~code. Ideally, when implementing a new code file, the programmer could first implement the unit test~~
	~~from a spec, and then work on implementing the code to pass the unit test; this is the development~~
	~~style advocated in the Extreme Programming philosophy.~~

	~~=== Regression Testing ===~~

	~~In regression testing, the philosophy is to implement test cases for each bug found, and to run these~~
	~~test cases against the codebase to determine that bug fixes actually fix the bug, and to ensure in the~~
	~~future that the bugs do not re-occur.~~

	~~This can be implemented by assembling a repository of "problem" SVG files that have caused Inkscape~~
	~~to error in programmatically detectable manners (such as crashes or emission of invalid SVG or PNG's).~~

	~~=== GUI Testing ===~~

	~~To eneable a11y in Inkscape for testing with dogtail, start Inkscape with the following command.~~
	* GTK_MODULES=gail:atk-bridge inkscape

	~~=== Fault Response Testing ===~~

	~~In fault response testing, the application is put into various critical situations and the behavior~~
	~~detected and compared against expectations. Examples include out-of-disk-space, out-of-memory,~~
	~~response to different signals, loading and saving very large files, starting up with invalid or unreadible~~
	~~config files, and so forth.~~

	~~=== Specification Compliance Testing ===~~

	~~In spec testing, the application is run against various input cases and caused to generate output that~~
	~~can be tested against "known" correct data. For instance, the [[W3C]] produces an SVG compliance test~~
	~~suite with SVG files and the expected rendered output as PNG's. Inkscape can be run on each of the~~
	~~SVG's from the commandline and caused to save the rendering as a PNG. An image-difference tool~~
	~~can then be used to calculate a diff-score between the emitted PNG and the [[W3C]] image.~~

	~~Similarly, this process can be used against other applications such as Batik or librsvg, to check~~
	~~inter-application conformance on the same SVG files. Ideally, all SVG applications should render~~
	~~a given SVG file identically.~~

	~~=== Performance Testing ===~~

	~~In performance testing the application is measured as to how well it works against various criteria such~~
	~~as start-up time, output quality, processing speed, etc.~~

	~~=== Valgrind Memory Leak Testing ===~~

	~~What I did here is simply ran inkscape through valgrind with the following command:~~

	~~valgrind -v --leak-check=yes --leak-resolution=high --num-callers=15 --show-reachable=yes --log-file=inkscape.log src/inkscape~~

	~~I then drew a few objects, edited some points and exit.~~

	~~What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)~~

	~~Note: Turn off Inkscape's garbage collection before starting valgrind, e.g.~~
	~~export _INKSCAPE_GC=disable~~
	~~(As suggested John Bintz)[http://www.coswellproductions.com/quick-drying-paint-profiling-inkscape-using-valgrind]~~

Johncoswell: Update URL

2011-02-13T22:42:29Z

Update URL

← Older revision		Revision as of 22:42, 13 February 2011
Line 79:		Line 79:
	Note: Turn off Inkscape's garbage collection before starting valgrind, e.g.		Note: Turn off Inkscape's garbage collection before starting valgrind, e.g.
	export _INKSCAPE_GC=disable		export _INKSCAPE_GC=disable
	(As suggested John ~~Coswell~~)[http://www.coswellproductions.com~~/wordpress/2008/04/30~~/quick-drying-paint-profiling-inkscape-using-valgrind/]		(As suggested John Bintz)[http://www.coswellproductions.com/quick-drying-paint-profiling-inkscape-using-valgrind]

Dvlierop: /* Valgrind Memory Leak Testing */

2008-12-06T14:03:54Z

Valgrind Memory Leak Testing

← Older revision		Revision as of 14:03, 6 December 2008
Line 76:		Line 76:

	What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)		What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)

			Note: Turn off Inkscape's garbage collection before starting valgrind, e.g.
			export _INKSCAPE_GC=disable
			(As suggested John Coswell)[http://www.coswellproductions.com/wordpress/2008/04/30/quick-drying-paint-profiling-inkscape-using-valgrind/]

Dvlierop: /* Valgrind Memory Leak Testing */

2008-12-06T13:56:49Z

Valgrind Memory Leak Testing

← Older revision		Revision as of 13:56, 6 December 2008
Line 71:		Line 71:
	What I did here is simply ran inkscape through valgrind with the following command:		What I did here is simply ran inkscape through valgrind with the following command:

	valgrind -v --leak-check=yes --leak-resolution=high --num-callers=15 --show-reachable=yes --~~logfile~~=inkscape.log src/inkscape		valgrind -v --leak-check=yes --leak-resolution=high --num-callers=15 --show-reachable=yes --log-file=inkscape.log src/inkscape

	I then drew a few objects, edited some points and exit.		I then drew a few objects, edited some points and exit.

	What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)		What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)

Acspike at 18:34, 6 July 2006

2006-07-06T18:34:15Z

← Older revision		Revision as of 18:34, 6 July 2006
Line 37:		Line 37:
	This can be implemented by assembling a repository of "problem" SVG files that have caused Inkscape		This can be implemented by assembling a repository of "problem" SVG files that have caused Inkscape
	to error in programmatically detectable manners (such as crashes or emission of invalid SVG or PNG's).		to error in programmatically detectable manners (such as crashes or emission of invalid SVG or PNG's).

			=== GUI Testing ===

			To eneable a11y in Inkscape for testing with dogtail, start Inkscape with the following command.
			* GTK_MODULES=gail:atk-bridge inkscape

	=== Fault Response Testing ===		=== Fault Response Testing ===

Conversion script: link fix

2006-01-22T02:33:48Z

link fix

New page

== Inkscape Testing Framework ==

In order to detect and identify bugs quickly and reliably, we should set up a testing framework for Inkscape.

There are several different ways to test a desktop Linux application. Each method provides different kinds of
information about the codebase and are each valuable in different ways. A multi-pronged approach that uses
more than one kind of testing is expected to gain the best results.

=== Compile Testing ===

In compile testing, the codebase is repetitively checked out and compiled in one or more different ways.
For example, with different compilers or for different platforms (Linux, Windows, Mac OSX, ...) Errors in
configuration, compilation, linking, or basic executation are detected, collected, and reported on a periodic
basis (such as daily compile reports).

It may be possible to set up compile testing using [[SourceForge]]'s compile farm. Another option to
investigate is the Mozilla project's "Tinderbox".

=== Unit Testing ===

Unit testing is implemented at the file level. Each *.c or *.cpp file is provided with a test file containing
calls for each of the functions, and that exercises the API, including a wide variety of error situations such
as NULL pointers, buffer overflows, very large or very small floating point numbers, zero or negative int's,
uninitialized objects, etc.

The intention of unit tests is to provide sanity checks that the developer can use as they implement the
code. Ideally, when implementing a new code file, the programmer could first implement the unit test
from a spec, and then work on implementing the code to pass the unit test; this is the development
style advocated in the Extreme Programming philosophy.

=== Regression Testing ===

In regression testing, the philosophy is to implement test cases for each bug found, and to run these
test cases against the codebase to determine that bug fixes actually fix the bug, and to ensure in the
future that the bugs do not re-occur.

This can be implemented by assembling a repository of "problem" SVG files that have caused Inkscape
to error in programmatically detectable manners (such as crashes or emission of invalid SVG or PNG's).

=== Fault Response Testing ===

In fault response testing, the application is put into various critical situations and the behavior
detected and compared against expectations. Examples include out-of-disk-space, out-of-memory,
response to different signals, loading and saving very large files, starting up with invalid or unreadible
config files, and so forth.

=== Specification Compliance Testing ===

In spec testing, the application is run against various input cases and caused to generate output that
can be tested against "known" correct data. For instance, the [[W3C]] produces an SVG compliance test
suite with SVG files and the expected rendered output as PNG's. Inkscape can be run on each of the
SVG's from the commandline and caused to save the rendering as a PNG. An image-difference tool
can then be used to calculate a diff-score between the emitted PNG and the [[W3C]] image.

Similarly, this process can be used against other applications such as Batik or librsvg, to check
inter-application conformance on the same SVG files. Ideally, all SVG applications should render
a given SVG file identically.

=== Performance Testing ===

In performance testing the application is measured as to how well it works against various criteria such
as start-up time, output quality, processing speed, etc.

=== Valgrind Memory Leak Testing ===

What I did here is simply ran inkscape through valgrind with the following command:

valgrind -v --leak-check=yes --leak-resolution=high --num-callers=15 --show-reachable=yes --logfile=inkscape.log src/inkscape

I then drew a few objects, edited some points and exit.

What I would like to see is people investigate the memory problems listed here and as they are resolved, we remove them from the list until we don't have any more. Then we can try to find more! :)