ExtensionsSystem

This is a copy of the original spec for the extensions system. Part of this was accepted into the Sodipodi codebase. This might be replaced by a more full-fledged plugin system in Inkscape at some point. The original patch is at http://developer.osdl.org/bryce/patches/sodipodi/extension_02/

Sodipodi Simple Extension System

This specification describes a design for implementing extension capabilities in Sodipodi, a 2D vector-based drawing application that uses the Scalable Vector Graphics (SVG) file format and the GNOME graphical user interface system.

The concept for this extension system is simple: The user selects some drawing elements within Sodipodi, and then chooses an extension program to run on them. Sodipodi renders the textual SVG code for the elements and then pipes this data to the stdin of the extension program. The extension performs its operations and prints the resulting SVG text to its stdout, which Sodipodi captures, parses, and displays. This can either be added to a new document or the same document, and if the latter, it can either replace the original selections or be added atop it.

The idea of "pipes" should resonate well with experienced UNIX users, because of the value they provide in general day-to-day use at the commandline. This approach allows small general purpose tools to be quickly combined to perform some specific purpose.

This is not a sophisticated, ultra-powerful, or high-performance system, but it is not intended to be so. The primary objectives this design seeks to achieve are simplicity of interface, language independence, and maximum flexibility. We aim to permit even neophyte programmers to be able to quickly throw together a little script to conduct some necessary transformation, without needing to first learn some new programming technologies. We also aim to enable more experienced software engineers to augment Sodipodi's functionality in a modular fashion that is independent of core development. And further, we hope that by allowing extension-writing to be done in a multiplicity of languages that we can take advantage of powerful functionalities already available via other languages (such as Perl's wealth of CPAN modules, or Java's XML manipulation technologies, or C++-based graphic manipulation libraries). This system may not be able to intrinsically meet 100% of the desires one would wish for in a powerful C-based plug-in system, but it hopefully will meet a sizable chunk of the needs, and should be able to do so with minimal impact on Sodipodi's internal development.

Key Requirements

• Extensions can be added to sodipodi without compiling it
• Extensions can be invoked from the GUI Sodipodi
• Extensions can be invoked from the commandline Sodipodi
• Extensions can be organized into one or more menus and toolboxes
• Quantity of extensions shall not delay program initialization
• Extensions (incl init) shall not run in background
• Authoring of an extension should be straightforward & easy to grok
• Extensions shall operate on entire SVG documents
• Extensions shall operate on fragments of SVG documents
• Extensions shall operate on individual SVG entities
• It shall be possible to write extensions in most any programming language

Architecture of a Compliant Extension

The extension is implemented as a commandline filter that takes some arbitrary SVG through stdin, conducts some operations, and returns processed SVG through stdout.

The benefit of the stdin/stdout approach is that it allows them to be pipe chained. For example, if one wished to modify a stock diagram of a data center lab by indicating status of particular machines with color, one might approach it via a complex commandline such as:

$ cat lab_diagram.svg \
     | svg_highlight --stroke-color=yellow --id=rack_24 --id=rack_42 \
     | svg_highlight --stroke-color=red --stroke-width=2px --id=rack_13 \
     | svg_delete  --id=main_border --id="legend_*" \
     | svg_change_text --id=title_text --sed="s/Lab Diagram/Rack Status/" \
     | sodipodi -z --export-png lab_diagram__rack_status.png
                                                                               

Extensions can also be run at the commandline as stand-alone filter programs. They are able to take a saved Sodipodi SVG file as input and emit a changed files with exactly the same effects as if it were invoked on the entire document within the Sodipodi GUI.

By being runnable from the commandline, these filters have a much larger applicability because they can be useful tools in their own right. For instance, the above example could be tied to output from a status sampler program such as Nagios to provide a dynamic graphic of the data center's machine status.

Each extension has a set of extension-specificit 'parameters' associated with it. These parameters are used to control the extension's behavior. Each parameter is indicated via gnu-style, double-dash option syntax (i.e., --blah --foo=bar).

If Sodipodi is running in GUI mode and some parameters for an invoked extension require or permit user specification at runtime, then a GUI dialog box will be displayed with the appropriate widgets shown to allow data specification by the user.

The extension also accepts one or more filenames on the commandline. If any filenames are provided, the extension will open and process those files the same as if they'd been cat'ed to STDIN. (This is the standard behavior for Perl scripts when using the common `while (<>) {` syntax.)

The extension is required to provide an exit status value when it completes. These are as follows:

<0 - Extension-specific error
0  - Success
1  - General failure
2  - Memory error
3  - File I/O error
4  - Math error
5  - Input not understood (not valid SVG?)
6  - Could not operate on any objects in this data stream
7-127 - Reserved
                                                                               

If an extension returns any error value other than 0, it can also print a textual error description (or stream) to STDERR. This could be used by Sodipodi to display error information to the user through a GUI or stderr, and/or recorded into an error log.

In any case where the return value is non-zero, anything emitted to STDOUT is discarded and ignored.

Parseable Error Language

If the first three characters of the STDERR stream match the characters '#%%', then the stream is considered to be 'parseable error language' text. The remainder of the line following the '#%%' signifier is used to specify the format and/or parsing options for the stream. For instance, if the first line matches '#%% INI', then the stream will be interpreted as a line-oriented list of key=value pairs separated by [sections]. The range of options supported by Sodipodi will be limited (assuming any are available at all), so extension developers should take care to select syntaxes that are well supported.

This 'parseable error language' system is completely optional for extensions, and is not recommended for typical extensions. It is provided as a mechanism for allowing better interaction between an extension and its caller than is available simply through error codes. For instance, this could allow returning a list of per-entity error messages, that Sodipodi could pop-up

For example, it could allow returning a long list of error messages, with one message declared the "main error" and the others provided for reference; or it could allow returning per-entity error messages, that the calling application could display in association with those objects; or it could permit returning error information for input dialogs (e.g., if the user has provided invalid input data), allowing them to re-edit and re-run the extension.

Extension Registration

Extension registration is controlled and cached by Sodipodi and occurs automatically when Sodipodi is started up.

Each extension is housed in its own subdirectory and is composed of the following files:

• Toolbutton image
• Config file (XML)
• Main executable
• Additional code files (optional)
• README, COPYING, AUTHORS, Changes, etc.

The process of installation of these files is left to the responsibility of the extension in question. For instance, they could be installed via RPM, make, or even generated by the script itself (e.g., like 'shar'). If the extension does not provide a toolbar image or config file, generic defaults will be used.

Sodipodi maintains a global XML config file with information about each extension, taken from the extension's config file. During program initialization, it will look in the various places where Sodipodi extensions can be installed, for any config files with timestamps newer than the last time it loaded them. If it finds such a file, it will re-parse that file and load it in place of the old data. The reason for using this procedure is because it is believed that loading and parsing a single config file, coupled with simple directory checking operations will be significantly faster than parsing individual extension config files during each activation.

In the master config file, each extension is registered with the following information:

• Extension title
• State (active / inactive / etc.)
• Description
• Path (e.g., /usr/share/sodipodi/extensions/foobar/)
• Toolbutton image filename
• Executable filename
• Help document filename
• List of parameters with type and default value
• Class(es) of data it acts on (text, nodes, etc.)
• Whether data is replaced, added, or added to new doc
• Toolbox menu location
• Toolbox menu state (closed, etc.)
• Timestamp of last loading

Sodipodi includes some information about extensions that are not necessarily present in the extension config file. In addition, the extension may incorporate other information into its config file that Sodipodi may ignore and not load into the master config.

Overriding of some of this information is optionally permitted via the user's preferences file. These overrides are always preserved (on a per-user basis) even if the global config file is regenerated. This capability is included to permit, for instance, per-user reorganization of the toolbox menu structure. The management of this preference information is outside the scope of this document.

The benefit of loading from a global config file is that generally one need only perform a single file-parse step during program initialization, which should help avoid excessive paging, etc. It is particularly advantageous if one is loading a large number (hundreds) of plugins, or are running in a very tight-memory situation.

Extensions are not allowed to modify this global config file directly. They instead can update their personal config file (during installation or upgrades), and Sodipodi will re-parse these as appropriate. Sodipodi will not make any changes to the individual extension's config file directly. In any case where config information needs to be stored in the extension's config file, Sodipodi will supply this information through the normal commandline-option-based interface, and the extension will be responsible for making the appropriate alterations to its config file.

A command will also be provided within Sodipodi's menubar to permit regeneration of the master config file from all of the extension configs. This command will also be available as a commandline option for the sodipodi executable.

# Remaining Questions
#
# What format/schema is the master config file?
#
# What API (i.e., std cmdline option) should be used to specify
# config values to add to the extension's config file?
                                                                               
                                                                               

Extension Operation

When an extension is invoked from Sodipodi, the currently selected SVG elements are serialized into textual SVG document fragments and piped to the extension executable's stdin. Sodipodi receives the resulting stdout from the pipe in textual SVG form. This is reparsed into internal data elements by a method in repr-io.c in Sodipodi. If this parsing is successful and results in valid objects, the original elements are deleted and replaced with the ones generated by the script.

The operations performed by the extension are treated as a single conceptual "change" for purposes of bookkeeping undo/redo, cache, etc.

Each extension specifies the classes and quantities of SVG elements it can operate on in its config file. Sodipodi could use this information, for example, to determine when to "grey out" and invalidate buttons of extensions that cannot be used on the current selection. Note that the extension mustn't assume that the data it is passed is validated to meet these criteria before being called, and should always perform data filtering and validation itself, on the data it is given; consider for instance a case where the extension is used on the commandline on a SVG file directly.

Any valid SVG element (line, circle, rectangle, text, etc.) recognized by Sodipodi may be used for the class specification. In addition, individual nodes and the document itself (with no contents) are valid types for extensions to operate on. A method in sp-obect-repr.c will be used to get a dynamic list of element classes.

Extensions may specify classes higher in the inheritance chain, so that for example one can specify that it operates on 'shapes' in general, including rects, lines, spirals and whatever new types there may be added in future. Quantities can be specified as ranges and/or comma separated lists, for instance, "1+", "2", "2,4,6,8,10", or "2,4-42".

SVG DOM syntax is used to specify the naming/inheritance. Sodipodi will implement some DOM name queries to support this.

Note that as of this writing, Sodipodi does not implement button-invalidation, but something like Qt 'actions' may be implemented to support this in the future.

# Remaining Questions
#
# Is it possible to pass a nodal entity and specify a subset of its
# nodes to operate upon?  How?
                                                                               
                                                                               

Extension Interface Specification =

Some extensions can act directly on the elements selected, but many will require further information to quantify & specify the parameters that control their operation. For example, an extension that stretches elements would need to know what percentage to stretch the object. Another might need to know a color to use, or replacement text for search and replace, etc.

Rather than require each extension to implement GUI behaviors independently (using Gtk bindings, Tk, etc.) this design includes provision for extensions to provide interface "descriptions", which Sodipodi can parse and use to dynamically generate dialogs for the extension. In addition to simplifying the work facing extension authors, this will provide more consistency of interface to end users, and avoid a class of portability issues, should one wish, for example, to use the scripts on a non-UNIX O/S.

This interface description language must provide easy specification of the widgets and text for the dialog, such as buttons, input boxes, color/alpha selectors, dropdown boxes, radio buttons, checkboxes, selection lists, gradient boxes, preview thumbnails, tree displays (e.g., for XML editing, etc.). It must allow precise positioning and sizing of the widgets, but also allow auto-layout (so that one can simply provide a list of text+widget_type pairs and the widgets placed onto the dialog in a running list. This way, extension authors can specify the dialogs with a minimum of fuss, but can also have more exact control over the layout of the form if they wish.

Parameter settings are not expected to be kept from one invocation of the extension to the next. If it is found to be necessary, it should be provided via Sodipodi, rather than by the extension.

# Remaining Questions:
#
# Should the interface definition be taken from config file
# or queried from the script?
#
# What already-existing interface abstraction languages exist?
#
# How would multi-dialog interfaces be implemented?
                                                                               
                                                                               

Extension Examples

The following cases are provided as "use cases" to help conceive the way the extension system should/could work. It should be clear that some of these functions, while of obvious utility to certain userbases, are too application specific to be worth incorporating into Sodipodi's core.

Spellcheck: Checks spelling of text in document and changes color of misspelled words to red. This would be invaluable if one is producing

Translate: Converts selected text into alternative language using separate translation application.

Align_to_grid: Adjusts all selected elements to line up to the document grid. Resizes objects so that all corners touch on grid corner points.

Data_center_status: Queries database of computer hardware and generates a map of the machine room using color to indicate state of computer racks.

Fractalize: Changes a nodal line or shape's border to be roughened using fractals. This could be useful for creation of realistic-looking coastlines in a map.

Weave: Takes two overlapping curved lines and "weaves" them to resemble a scroll-work rope such as is common in Celtic art.

-- Bryce W. Harrington