Created attachment 24897 [details] [review]
add PANGO_SCRIPT_PREFERRED

Created attachment 24899 [details] [review]
modifications to use PANGO_SCRIPT_PREFERRED

I'm not entirely sure that that extra script is needed; the way
I'd expect it to work is that the SILGraphite shaper would
simply advertise to cover all scripts exactly, with
PANGO_SCRIPT_COMMON, "*" (as oppposed to the basic shaper
which covers them approximately with PANGO_SCRIPT_COMMON,   "")

Then the Graphite shaper can use the "covers" method to say
whether it wants a particular font or not.

The way that that I set things up for Pango-1.4 is that 
if we have three shapers that handle arabic - 
Arabic (exact) Graphite (exact) and basic (fallback)
and two fonts A, B, the sequence is:

 arabic.covers (A, char)
 graphite.covers (A, char)
 arabic.covers (B, char)
 graphite.covers (B, char)
 basic.covers (A, char)
 basic.covers (B, char)

So, if font A is a graphite font, you get the graphite shaper,
if it is a opentype font, you get the opentype shaper.

There is a pretty big problem with this approach - fontconfig
prunes the list of returned fonts; if font A was a Quickdraw GX
font, then neither the Graphite nor the OpenType shapers would
ever see font B.

The solution to that requires some extra API to let shapers
influence the sorting of the returned lists. I don't know
the exact details. But this shouldn't keep you from experimenting
with a Graphite shaper; just make sure that the fonts you
want to use are specified explicitely or sort on your system
ahead of "chart" fonts that cover the characters but don't
have useful layout tables.

>I'm not entirely sure that that extra script is needed; the way
>I'd expect it to work is that the SILGraphite shaper would
>simply advertise to cover all scripts exactly, with
>PANGO_SCRIPT_COMMON, "*" (as oppposed to the basic shaper
>which covers them approximately with PANGO_SCRIPT_COMMON,   "")

I found that that wasn't enough. The basic-fc shaper covers most
scripts exactly (and I'd guess the script specific ones cover the
rest) and pango_map_get_engines and pango_map_get_engine only use an
exact common if an exact script match isn't found.

Another option could be to change the coverage of the basic shaper to
just be PANGO_SCRIPT_COMMON, "". However, it will still be an issue
for the scripts covered by the other shapers.

Or maybe in pango_map_get_engines both the common and entry engines
should be appended (currently it is one or other)-  that sounds like
what you are describing for covers to be checked on multiple shapers.

> arabic.covers (A, char)
> graphite.covers (A, char)

If both the shapers have coverage is it defined which one will be
used? I know that the SIL font folks are putting OT tables into the
fonts as well as graphite ones so they have some functionality on
non-graphite enabled apps.

If there are particular scripts that Graphite can do a better
job on than 'basic', then they probably shouldn't be marked
"*" for the basic shaper. In generally, only scripts where
I thought that the basic shaper was sufficient for fully
correct rendering got "*". What scripts are you having
issues with?

In terms of the fonts with both sets of tables, I don't have
a good solution there. We could add some sort of priority
system where shapers were sorted by how "good" they claimed
to be, but I don't see how a shaper author knows how
"good" their shaper is compared to other shapers that might
not yet be written...

The one I'm having an issue with is Latin. The test font that I'm
using is piglatin. It is a real issue though as stacked diacritics on
a latin font is something that graphite can be used for. Graphite
isn't at the moment better for any particular script (although it
might be at some point if a good script-specific font is made using
it, like Myanmar), it has the potential to add extra features to an
existing script. In particular to add it for a particular language
group that uses a modified script.
Graphite depends on the font rather than the script.

The PANGO_SCRIPT_PREFERRED was my idea for the graphite shaper to say
that its shaper should override any other if the font can handle it.
I'm assuming that the font designer will cover at least the same as
the opentype part of the font in the graphite part.

However, afaict it won't be defined which one will win if another
shaper comes along and wants to be preferred as well. OTOH it is the
same at the moment if more than one shaper wants to be exact for a
script or exact for common.

Alternatively, maybe there could be extra PangoCoverageLevels.

Also, if pango is choosing between different fonts rather than
choosing a shaper for a specific font I'd guess it would be more
likely to want to choose a non-graphite font, or at least not
necessarily choose the graphite one. Are fonts chosen on the basis of
 coverage or is the font chosen by script and then the shaper chosen
by coverage?

The way I'm doing the graphite pango module is to install it to
$(pkglibdir)/pango/$(PANGO_MODULE_VERSION) and then users or sysadmin
can decide to add it or not to the ModulesPath in the pangorc.

Maybe there could be a way to specify relative preferences for shapers
in the pangorc?

Writeup of what I think the blocking issues here are at:

http://mail.gnome.org/archives/gtk-i18n-list/2004-December/msg00020.html

Moving off 1.8 milestone and marking Priority High as something I'd like
to get to for the next devel cycle.

Thanks for your help at GUADEC. I'm pretty sure now that these patches are
unnecessary - it is just important to get the graphite module to the bottom of
the list.

I'm leaving this bug open in case there are changes I'll want to request to get
the cursor tracking right as I want to investigate that further. I'm going to
have another look at the win32 uniscribe module code to see if that helps to get
it working out of the box.

The other thing I'm thinking about is submitting a version of the graphite
module for inclusion in pango. It would load the silgraphite library dynamically
if it is available (like basic-win32 does for usp10.dll). I would expect a
small->negligible performance hit on systems without graphite as
graphite_engine_covers (which would check for has_silgraphite and return false)
would have to be called every time. Does this sound like an acceptable hit? Or
should I just keep the graphite module separate?

Regards,
Daniel

Any news on this?  Having the Graphite module included would be nice.  We can
work out that little bit about what to do if Graphite is not available later.  I
think what happens is that the pango.modules wont list it and you need to
regenerate it after installing graphite.  No matter which route we take, an
initial patch is needed.

I've almost got this ready - a module that is split out that doesn't depend on
graphite, but loads a helper library dynamically. Will attach patch soon.

Created attachment 55820 [details] [review]
mods to build system to add graphite module

This part adds graphite module to the build

Created attachment 55821 [details]
graphite module

graphite shaping and breaking modules that loads a helper library dynamically
(which uses graphite) if it is available

Ok, where can I get this libpangographite that the modules dlopens?

And I'm not seeing why you have to duplicate the break.c code.  Would you please
explain a bit what is the code doing?

Thanks.

a snapshot from svn of pangographite is available at
http://www.sf.net/projects/silgraphite

The code adds a call to graphite_real_break in the middle of
pango_default_break_for_graphite which allows graphite to do part of breaking -
setting is_cursor_position, is_line_break, is_char_break. The rest of the
algorithm is the same currently.

Created attachment 58615 [details] [review]
mods to build system to add graphite module

update patch against current/recent cvs.

Created attachment 58616 [details]
graphite module directory

unpack module to modules/

updated break module to just call pango_default_break then do graphite stuff on the result so it is cleaner and easier to maintain.

src RPMs of graphite are available at via https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=176071
I'll co-ordinate with Michael to get the latest package made as well including the pangographite support lib but not the pango module.

Ok, with calling pango_default_break now, I'm a lot happier with your patch.  I will be looking into it sometime soon.  Thanks.

Behdad has said that it is ok for the C++ dependency of libpangographite to go into pango so it is ok for libgraphitepango to be merged back into the module. Also that the shaping and breaking modules can be merged together to share the caching. I'll just merge those back in and make sure it works.

Thanks,
Daniel

Created attachment 69132 [details] [review]
Build system additions for graphite

changes to configure.in to detect graphite, and Makefile.am to build it
configure.in changes aren't complete (need to decide what to say when graphite isn't found)

You also might prefer to just use AC_PROG_CXX whether or not graphite is there, or to depend on the AC_CHECK_PROGS(CXX, [$CCC c++ g++ gcc CC cxx cc++ cl], gcc) line 105

Created attachment 69133 [details]
graphite module directory

libpangographite has been integrated into this. There are bugs for ligatures which have been introduced by decisions for burmese. We will look into them.

Created attachment 76712 [details]
pangographite integration module

includes unassigned codepoint support, broken hexboxes

So, you only try to do a hexbox if the shaping fails completely?  

hexbox works in HEAD assuming fix in pangographite that gets the glyph box width correct for unknown glyphs.

any progress?

This is obsolete.  Enabled graphite2 shaper in harfbuzz instead.