Difference between revisions of "Pangoification"

From Inkscape Wiki
Jump to navigation Jump to search
m (Reverted edits by Sy9Iav (Talk); changed back to last version by GigaClon)
(more wikifi)
 
(3 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction to the Inkscape 0.39 Pango Rearchitecting Effort ==
== Introduction to the Inkscape 0.39 Pango Rearchitecting Effort ==


Wondering what the heck [http://pango.org/ Pango] is, why we want it in the codebase, what
Wondering what the heck [http://pango.org/ Pango] is, why we want it in the codebase, what needs to be done, and/or what's been done so far?  Then this document is for you.  We'll try to cover the basics to get you oriented, and provide some links to more info.
needs to be done, and/or what's been done so far?  Then this document is
for you.  We'll try to cover the basics to get you oriented, and provide
some links to more info.


=== Contents ===
=== Contents ===
Line 29: Line 26:


=== What is Pango and what does it do? ===
=== What is Pango and what does it do? ===
Pango is a library that provides layout and rendering of
Pango is a library that provides layout and rendering of internationalized text, that uses Unicode for its encoding and aims for
internationalized text, that uses Unicode for its encoding and aims for
supporting output in all the world's major languages.
supporting output in all the world's major languages.


Pango's principle use has been for internationalizing the text in Gnome
Pango's principle use has been for internationalizing the text in Gnome GUI's. [http://www.pango.org/gallery.shtml See the gallery].
GUI's. See the gallery:  http://www.pango.org/gallery.shtml


From the Pango 1.4 release announcement:
From the Pango 1.4 release announcement:


  Pango is designed to be modular; the core Pango layout can be used
{| width="50%" align="left" cellpadding="10" style="border:1px #AAAAAA solid;background-color: #F9F9F9; "
   with three different font backends:
| align="justify" | Pango is designed to be modular; the core Pango layout can be used  with three different font backends:
#Client-side fonts on X using the Xft2 library
#Direct rendering of scalable fonts using the FreeType library
#Native fonts on Microsoft platforms


  - Client-side fonts on X using the Xft2 library
Pango-1.4.0 ships with a wide selection of modules, including modules for Hebrew, Arabic, Hangul, Thai, and a number of Indic  scripts. Virtually all of the world's major scripts are supported.
  - Direct rendering of scalable fonts using the FreeType library
  - Native fonts on Microsoft platforms


  Pango-1.4.0 ships with a wide selection of modules, including modules
As well as the low level layout rendering routines, Pango includes PangoLayout, a high level driver for laying out entire blocks of text, and routines to assist in editing internationalized text.
  for Hebrew, Arabic, Hangul, Thai, and a number of Indic
  scripts. Virtually all of the world's major scripts are supported.


  As well as the low level layout rendering routines, Pango includes
More information about Pango is available from http://www.pango.org/
  PangoLayout, a high level driver for laying out entire blocks of text,
|}
  and routines to assist in editing internationalized text.
<br clear="all"/>


  More information about Pango is available from http://www.pango.org/"
In other words, Pango is a glue between platform-specific font subsystems and application-specific renderers, with strong
 
In other words, Pango is a glue between platform-specific font
subsystems and application-specific renderers, with strong
internationalization support.
internationalization support.


=== What does Inkscape currently have that it'd replace? ===
=== What does Inkscape currently have that it'd replace? ===
Inkscape currently uses a custom-written library called libnrtype for doing typeface rendering.  This library provides wrappers around other platform-specific font subsystems, and allows us to do things like convert text to curves by getting the outlines of individual glyphs, look up typeface metrics, and convert CSS style representations to internal values for font weight, style, etc.


Inkscape currently uses a custom-written library called libnrtype for
The library includes hooks to Xft, Gnome-Print, and Win32. It has dependencies into libnr.  As part of the Inkscape renderer work, livarot has also been added as a dependency for this.
doing typeface rendering.  This library provides wrappers around other
platform-specific font subsystems, and allows us to do things like
convert text to curves by getting the outlines of individual glyphs,
look up typeface metrics, and convert CSS style representations to
internal values for font weight, style, etc.


The library includes hooks to Xft, Gnome-Print, and Win32. It has
We aren't sure if anyone other than Inkscape and Sodipodi use this library. It was originally written by Lauris for Sodipodi. We've been maintaining it as things like livarot have been introduced, but the Inkscape team are not actively developing it anymore.
dependencies into libnr. As part of the Inkscape
renderer work, livarot has also been added as a dependency for this.


We aren't sure if anyone other than Inkscape and Sodipodi use this
In other words, libnrtype is an efficient internal library that enables us to work with fonted text as first-class drawing objects, but unfortunately it does not have the complex support for internationalization that we need.   
library.  It was originally written by Lauris for Sodipodi.  We've been
maintaining it as things like livarot have been introduced, but the Inkscape team
are not actively developing it anymore.
 
In other words, libnrtype is an efficient internal library that enables
us to work with fonted text as first-class drawing objects, but unfortunately
it does not have the complex support for internationalization that we need.   




=== What will Inkscape be able to get from Pango? ===
=== What will Inkscape be able to get from Pango? ===
Pango looks like it will bring significant value to us by allowing us to provide much better Internationalized font support, and in doing so, enable us to close a huge chunk of old bug reports.


Pango looks like it will bring significant value to us by allowing us to
It also replaces a sizeable chunk of code in Inkscape and instead of the extra overhead of maintaining our own specific code we get all the benefits of sharing a standard, stable, well maintained codebase that is pivotal to the Gnome desktop and has many more resources being put into it. This will result in a net reduction in the size of our codebase, which is a _good_ thing; less code means fewer places where bugs can appear and suck our time away from more important things.   
provide much better Internationalized font support, and in doing so,
enable us to close a huge chunk of old bug reports.
 
It also replaces a sizeable chunk of code in Inkscape and instead of the extra  
overhead of maintaining our own specific code we get all the benefits of sharing  
a standard, stable, well maintained codebase that is pivotal to the Gnome desktop  
and has many more resources being put into it.
This will result in a net reduction in the size of our codebase, which is a
_good_ thing; less code means fewer places where bugs can appear and
suck our time away from more important things.   


Specifically, beyond what we currently have, Pango gives us:
Specifically, beyond what we currently have, Pango gives us:
Line 100: Line 69:
* Internationalization support
* Internationalization support
* An actively maintained library (last release 16 Mar 2004)
* An actively maintained library (last release 16 Mar 2004)
* Documentation (Ref Man:  http://developer.gnome.org/doc/API/2.0/pango/)
* Documentation ([http://developer.gnome.org/doc/API/2.0/pango/ Man])
* Modular system for renderers gives us a clean interface for hooking Inkscape's renderer to.
* Modular system for renderers gives us a clean interface for hooking Inkscape's renderer to.
* A text layout system (left/right align, tab stops, etc.) - may not be powerful enough to meet our needs, though.
* A text layout system (left/right align, tab stops, etc.) - may not be powerful enough to meet our needs, though.
Line 108: Line 77:
=== How would Pango fit into Inkscape? ===
=== How would Pango fit into Inkscape? ===


While Pango provides a lot that libnrtype does not, there are some things
While Pango provides a lot that libnrtype does not, there are some things we'll need to consider:
we'll need to consider:


* Text-to-path:  Does Pango allow us to create outlines of text?
* Text-to-path:  Does Pango allow us to create outlines of text?
Line 115: Line 83:
* Any other unknown unknowns
* Any other unknown unknowns


These things aside, though, it could be as simple as just switching out
These things aside, though, it could be as simple as just switching out the libnrtype calls with the corresponding ones from pango, and then test the heck out of it.
the libnrtype calls with the corresponding ones from pango, and
then test the heck out of it.


One approach to doing the conversion is to simply start going through
One approach to doing the conversion is to simply start going through libnrtype, function by function, and changing the routines to become pass-thru calls to the corresponding Pango calls, where feasible. Then, whatever is left-over we
libnrtype, function by function, and changing the routines to become
#place in another location in Inkscape
pass-thru calls to the corresponding Pango calls, where feasible. Then,
#package up and submit as a patch to help improve Pango in the long-term.
whatever is left-over we a) place in another location in Inkscape, and
b) package up and submit as a patch to help improve Pango in the
long-term.




=== Who's done what so far with the old code, and with Pango? ===
=== Who's done what so far with the old code, and with Pango? ===
Nathan has the most experience with text layout issues, and has done some work on libnrtype in the past.  Several of the core developers have touched on text/font areas of the code, and can provide some guidance around the edges of the code. Several team members have had some limited experience with Pango via its use through the GUI interface, but we don't yet have an experience base built up in it.


Nathan has the most experience with text layout issues, and has done
Beyond that, while we all are in consensus that this work needs done, that work has not begun in earnest yet.  So even someone with no
some work on libnrtype in the past.  Several of the core developers have
knowledge of fonts or Inkscape internals should be able to catch up to speed with the rest of us pretty darn quick.  ;-)
touched on text/font areas of the code, and can provide some guidance
around the edges of the code.  Several team members have had some limited
experience with Pango via its use through the GUI interface, but we
don't yet have an experience base built up in it.


Beyond that, while we all are in consensus that this work needs done,
Inkscape already indirectly depends on Pango because [http://sourceforge.net/mailarchive/message.php?msg_id=7809807 GTK depends on Pango].
that work has not begun in earnest yet.  So even someone with no
knowledge of fonts or Inkscape internals should be able to catch up to
speed with the rest of us pretty darn quick.  ;-)
 
Inkscape already indirectly depends on Pango because  
<a href="http://sourceforge.net/mailarchive/message.php?msg_id=7809807"
title="GTK depends on Pango">Pango it is an internal dependency of GTK</a>


=== Okay, what do we need to do? ===
=== Okay, what do we need to do? ===
Line 167: Line 120:


=== Pangoification Discussion ===
=== Pangoification Discussion ===
This is a summary of a discussion between mental, bulia, fred, and bryce on Jabber 2004-06-05 regarding adoption of Pango and the issues involved.


This is a summary of a discussion between mental, bulia, fred, and bryce
The principle issue we are facing at this time is that while many of the Pango issues we've identified seem to have doable work-arounds, it's become a little unclear what our eventual goal design-wise is with it in the long term.  Specifically, Pango likes to grid-fit its layouts, which makes it a bit inflexible for things like getting undistorted outlines and layouts.
on Jabber 2004-06-05 regarding adoption of Pango and the issues involved.
 
The principle issue we are facing at this time is that while many of the
Pango issues we've identified seem to have doable work-arounds, it's
become a little unclear what our eventual goal design-wise is with it in
the long term.  Specifically, Pango likes to grid-fit its layouts, which
makes it a bit inflexible for things like getting undistorted outlines
and layouts.




==== Vertical Text ====
==== Vertical Text ====


Pango does well with horizontal text but Fred had a question about how
Pango does well with horizontal text but Fred had a question about how multi-byte characters are handled.  E.g., an A with an umlaut requires two bytes.  Would this render as a single character entity or as two? Certainly we wouldn't want the system to resort to putting the A and umlaut on separate lines.
multi-byte characters are handled.  E.g., an A with an umlaut requires
two bytes.  Would this render as a single character entity or as two?
Certainly we wouldn't want the system to resort to putting the A and
umlaut on separate lines.


Mental explained that Pango provides routines for determining how many
Mental explained that Pango provides routines for determining how many "codepoints" makes up a particular character.  When you pass this routine a UTF-8 string pointer, it will return the pointer advanced by the next full character.  A codepoint can consist of one or more bytes. one or more codepoints define a 'glyph'. One or more glyphs define a character.
"codepoints" makes up a particular character.  When you pass this
routine a UTF-8 string pointer, it will return the pointer advanced by
the next full character.  A codepoint can consist of one or more bytes.
one or more codepoints define a 'glyph'. One or more glyphs define a
character.


A glyph like an umlauts is an example of a "composing character". Fonts
A glyph like an umlauts is an example of a "composing character". Fonts combine these composing characters with other base characters to form the various characters particular to a given language.
combine these composing characters with other base characters to form
the various characters particular to a given language.


It looks like pango_layout_move_cursor_visually() would be used to
It looks like pango_layout_move_cursor_visually() would be used to determine the positioning of vertical text. This won't be simple if we don't use the Pango glyph positioning.  For more info see: [http://developer.gnome.org/doc/API/2.0/pango/pango-Layout-Objects.html#pango-layout-move-cursor-visually]
determine the positioning of vertical text. This won't be simple if we
don't use the Pango glyph positioning.  For more info see:
http://developer.gnome.org/doc/API/2.0/pango/pango-Layout-Objects.html#pango-layout-move-cursor-visually


For our own work this suggests we may need to create PangoLayouts to do
For our own work this suggests we may need to create PangoLayouts to do our own specialized layout.
our own specialized layout.


UPDATE:  Fred has successfully tested it out. He says it's only repositioning the text
'''UPDATE''':  Fred has successfully tested it out. He says it's only repositioning the text vertically; the tricky part is to get the correct characters.
vertically; the tricky part is to get the correct characters.


==== Kerning Abilities ====
==== Kerning Abilities ====
Control over font kerning is important for Inkscape yet it is uncertain
Control over font kerning is important for Inkscape yet it is uncertain if Pango provides this control.  For letter spacing we need to be able to:
if Pango provides this control.  For letter spacing we need to be able
to:


* Measure strings
* Measure strings
Line 219: Line 147:
* Glyph width (for things like text-on-path)
* Glyph width (for things like text-on-path)


Currently, sp-chars creates a NRArenaGlyphGroup for each string, tied to
Currently, sp-chars creates a NRArenaGlyphGroup for each string, tied to one NRRasterFont, but for things like text-on-path we'll want to have unique transforms for each character.  We do not want to have to create a separate GlyphGroup for each glyph though, so need a better solution to this.
one NRRasterFont, but for things like text-on-path we'll want to have
unique transforms for each character.  We do not want to have to create
a separate GlyphGroup for each glyph though, so need a better solution
to this.


The question came up about whether we can eliminate NRRasterFont. This
The question came up about whether we can eliminate NRRasterFont. This appears to be a necessary step in the pipline, because it is where the transform is taken into account and where glyph caching occurs. There is a question about whether glyph caching is really necessary, or if we could do direct rendering and use performance analysis to determine where such optimizations are really necessary, but since text always has such massive redundancy, caching is likely to prove quite necessary.
appears to be a necessary step in the pipline, because it is where the
transform is taken into account and where glyph caching occurs. There
is a question about whether glyph caching is really necessary, or if we
could do direct rendering and use performance analysis to determine
where such optimizations are really necessary, but since text always has
such massive redundancy, caching is likely to prove quite necessary.


[[Category:Wiki Attic]]
[[Category:Wiki Attic]]


==== Win32 Testing ====
==== Win32 Testing ====
We've very concerned that bugs in font handling are going to reproduce
We've very concerned that bugs in font handling are going to reproduce quickly on different platforms unless they're closely tracked by developers on those platforms.  Is adopting Pango worth enduring those platform issues?
quickly on different platforms unless they're closely tracked by
developers on those platforms.  Is adopting Pango worth enduring those
platform issues?




Latest revision as of 17:50, 3 December 2011

Introduction to the Inkscape 0.39 Pango Rearchitecting Effort

Wondering what the heck Pango is, why we want it in the codebase, what needs to be done, and/or what's been done so far? Then this document is for you. We'll try to cover the basics to get you oriented, and provide some links to more info.

Contents

  1. What does Inkscape really need for font rendering (perfect world)?
  2. What is Pango and what does it do?
  3. What does Inkscape currently have that it'd replace?
  4. What will Inkscape be able to get from Pango?
  5. How would Pango fit into Inkscape?
  6. Who's done what so far with the old code, and with Pango?
  7. Okay, what do we need to do?
  8. Pangoification Discussion 2004-06-05


What does Inkscape really need for font rendering (perfect world)?

What Inkscape needs from its font rendering subsystem is:

  • Regular font rendering, including a wide range of different font metrics.
  • Extraordinarily good support for internationalized fonts (many of the bugs left from the 0.38 Bug Hunt were due to inadequacies in font handling.)
  • Great support for generating outlines of text objects
  • Good support for us to build text layout algorithms (like text-along-a-path) onto.
  • Provided by a well-documented, well-supported, widely-available library that is actively maintained and extended by people other than us, who are very responsive to Inkscape's needs. Ideally the library would add no additional dependencies on us.


What is Pango and what does it do?

Pango is a library that provides layout and rendering of internationalized text, that uses Unicode for its encoding and aims for supporting output in all the world's major languages.

Pango's principle use has been for internationalizing the text in Gnome GUI's. See the gallery.

From the Pango 1.4 release announcement:

Pango is designed to be modular; the core Pango layout can be used with three different font backends:
  1. Client-side fonts on X using the Xft2 library
  2. Direct rendering of scalable fonts using the FreeType library
  3. Native fonts on Microsoft platforms

Pango-1.4.0 ships with a wide selection of modules, including modules for Hebrew, Arabic, Hangul, Thai, and a number of Indic scripts. Virtually all of the world's major scripts are supported.

As well as the low level layout rendering routines, Pango includes PangoLayout, a high level driver for laying out entire blocks of text, and routines to assist in editing internationalized text.

More information about Pango is available from http://www.pango.org/


In other words, Pango is a glue between platform-specific font subsystems and application-specific renderers, with strong internationalization support.

What does Inkscape currently have that it'd replace?

Inkscape currently uses a custom-written library called libnrtype for doing typeface rendering. This library provides wrappers around other platform-specific font subsystems, and allows us to do things like convert text to curves by getting the outlines of individual glyphs, look up typeface metrics, and convert CSS style representations to internal values for font weight, style, etc.

The library includes hooks to Xft, Gnome-Print, and Win32. It has dependencies into libnr. As part of the Inkscape renderer work, livarot has also been added as a dependency for this.

We aren't sure if anyone other than Inkscape and Sodipodi use this library. It was originally written by Lauris for Sodipodi. We've been maintaining it as things like livarot have been introduced, but the Inkscape team are not actively developing it anymore.

In other words, libnrtype is an efficient internal library that enables us to work with fonted text as first-class drawing objects, but unfortunately it does not have the complex support for internationalization that we need.


What will Inkscape be able to get from Pango?

Pango looks like it will bring significant value to us by allowing us to provide much better Internationalized font support, and in doing so, enable us to close a huge chunk of old bug reports.

It also replaces a sizeable chunk of code in Inkscape and instead of the extra overhead of maintaining our own specific code we get all the benefits of sharing a standard, stable, well maintained codebase that is pivotal to the Gnome desktop and has many more resources being put into it. This will result in a net reduction in the size of our codebase, which is a _good_ thing; less code means fewer places where bugs can appear and suck our time away from more important things.

Specifically, beyond what we currently have, Pango gives us:

  • Internationalization support
  • An actively maintained library (last release 16 Mar 2004)
  • Documentation (Man)
  • Modular system for renderers gives us a clean interface for hooking Inkscape's renderer to.
  • A text layout system (left/right align, tab stops, etc.) - may not be powerful enough to meet our needs, though.
  • Already exists as a dependency for Inkscape due to Gtk, so adds no additional dependencies for our users.


How would Pango fit into Inkscape?

While Pango provides a lot that libnrtype does not, there are some things we'll need to consider:

  • Text-to-path: Does Pango allow us to create outlines of text?
  • Different function calls / changed API
  • Any other unknown unknowns

These things aside, though, it could be as simple as just switching out the libnrtype calls with the corresponding ones from pango, and then test the heck out of it.

One approach to doing the conversion is to simply start going through libnrtype, function by function, and changing the routines to become pass-thru calls to the corresponding Pango calls, where feasible. Then, whatever is left-over we

  1. place in another location in Inkscape
  2. package up and submit as a patch to help improve Pango in the long-term.


Who's done what so far with the old code, and with Pango?

Nathan has the most experience with text layout issues, and has done some work on libnrtype in the past. Several of the core developers have touched on text/font areas of the code, and can provide some guidance around the edges of the code. Several team members have had some limited experience with Pango via its use through the GUI interface, but we don't yet have an experience base built up in it.

Beyond that, while we all are in consensus that this work needs done, that work has not begun in earnest yet. So even someone with no knowledge of fonts or Inkscape internals should be able to catch up to speed with the rest of us pretty darn quick.  ;-)

Inkscape already indirectly depends on Pango because GTK depends on Pango.

Okay, what do we need to do?

For specific tasks, please refer to the Inkscape Roadmap in the wiki site, but in general here is roughly what will need to be done:

a) Get up to speed in Pango. Get familiar with the reference manual. [Done]

b) Review the libnrtype code.

c) Replace calls of libnrtype routines that have close analogs in Pango with their Pango equivalents. If there are no remaining calls to the libnrtype routines, remove them from the codebase.

d) Analyze the remaining libnrtype routines to see if they can be refactored into Pango calls to get the equivalent behavior, or alternately examine if the calling code could be refactored to use Pango instead of libnrtype.

e) Re-abstract remaining libnrtype code that can't be rewritten as Pango calls and place in a new area of the codebase outside libnrtype. E.g., "nr-type-pango". Consider if it would be appropriate to also submit that chunk of code back to Pango as an added feature; if so, package it as a patch and send it to the Gtk mailing list (or a <a href="http://bugzilla.gnome.org/enter_bug.cgi?product=pango">patch tracker</a> if you can find one). For example, it could be _quite_ handy to refactor out the glyph outlining and text attributes code into some adaptor objects/functions that works with Pango.

f) Start employing Pango for doing text layout for e.g. SPText and friends.

For reference see <a href="http://sourceforge.net/mailarchive/forum.php?thread_id=4172159&forum_id=36054">mental's post about how to do Pango conversion</a> from the <a href="http://inkscape.org/mailing_lists.php">inkscape-devel@ mailing list</a>.


Pangoification Discussion

This is a summary of a discussion between mental, bulia, fred, and bryce on Jabber 2004-06-05 regarding adoption of Pango and the issues involved.

The principle issue we are facing at this time is that while many of the Pango issues we've identified seem to have doable work-arounds, it's become a little unclear what our eventual goal design-wise is with it in the long term. Specifically, Pango likes to grid-fit its layouts, which makes it a bit inflexible for things like getting undistorted outlines and layouts.


Vertical Text

Pango does well with horizontal text but Fred had a question about how multi-byte characters are handled. E.g., an A with an umlaut requires two bytes. Would this render as a single character entity or as two? Certainly we wouldn't want the system to resort to putting the A and umlaut on separate lines.

Mental explained that Pango provides routines for determining how many "codepoints" makes up a particular character. When you pass this routine a UTF-8 string pointer, it will return the pointer advanced by the next full character. A codepoint can consist of one or more bytes. one or more codepoints define a 'glyph'. One or more glyphs define a character.

A glyph like an umlauts is an example of a "composing character". Fonts combine these composing characters with other base characters to form the various characters particular to a given language.

It looks like pango_layout_move_cursor_visually() would be used to determine the positioning of vertical text. This won't be simple if we don't use the Pango glyph positioning. For more info see: [1]

For our own work this suggests we may need to create PangoLayouts to do our own specialized layout.

UPDATE: Fred has successfully tested it out. He says it's only repositioning the text vertically; the tricky part is to get the correct characters.

Kerning Abilities

Control over font kerning is important for Inkscape yet it is uncertain if Pango provides this control. For letter spacing we need to be able to:

  • Measure strings
  • Count characters
  • Handle pairwise kerning
  • Glyph width (for things like text-on-path)

Currently, sp-chars creates a NRArenaGlyphGroup for each string, tied to one NRRasterFont, but for things like text-on-path we'll want to have unique transforms for each character. We do not want to have to create a separate GlyphGroup for each glyph though, so need a better solution to this.

The question came up about whether we can eliminate NRRasterFont. This appears to be a necessary step in the pipline, because it is where the transform is taken into account and where glyph caching occurs. There is a question about whether glyph caching is really necessary, or if we could do direct rendering and use performance analysis to determine where such optimizations are really necessary, but since text always has such massive redundancy, caching is likely to prove quite necessary.

Win32 Testing

We've very concerned that bugs in font handling are going to reproduce quickly on different platforms unless they're closely tracked by developers on those platforms. Is adopting Pango worth enduring those platform issues?


References

-- Bryce 4/16/04
-- Alan 18/04/04 UTC, various minor edits
Retrieved from "https://wiki.inkscape.org/wiki/index.php?title=Pangoification&oldid=74228"