On the other hand... please note that it's exactly the non-TrueType fonts that 
cause(d) lots of problems in GIMP 2.0 on Windows. See the ever-so-popular bug 
#132366, for instance. Do we really want to open up the non-TrueType can of 
worms for other applications, too? 

Actually, I'm thinking about Fixedsys font available to X-Chat-win32 users. Many
new users come from mIRC, which uses Fixedsys font as default, which is the best
monospaced font for this purpose (Courier New font is very thin), and they (me
too) miss the font.

From the bug mentioned above I see bitmap font has caused crashes, but it was
not Fixedsys (which comes with all Windows installations). Maybe, for the very
least, only Fixedsys (and not other bitmap fonts) would be made available to
pango? Also, it seems that OpenType w/Postscript outlines fonts are untested, so
maybe it would be a good time to start? They work on Linux, at least.

One more point: pango font rendering performance is still very poor, it's very
noticeable when using X-Chat (in its xtext widget). Bitmap fonts are faster even
in native Windows applications, so making Fixedsys available could make X-Chat
(I hope noticeably) faster.

> pango font rendering performance is still very poor

With "still", what version are you referring to? If you are using Pango 1.2.x, 
and are sure you will handle only non-complex test (i.e. just Latin, Cyrillic, 
Greek, Chinese etc, and no Hebrew, Arabic, Indic, Thai etc), have you tried 
setting the PANGO_WIN32_NO_UNISCRIBE environment variable (to any value)? It 
might have a dramatic effect.

Weren't the crashes with the FT2 backend, not the win32 backend?

I don't so much mind that bitmap fonts are missing, but the lack of support for
Type1 and PS OpenType fonts in Win32 is a bit annoying for me, as they're
supported not only by Unix GTK but also at the system level in Windows 2000 and
above. I have quite a few Adobe fonts I'd really like to be able to use in xchat
and gaim.

Where can one find (legally) some sample Type1 fonts to use in Windows (and
whatever machinery necessary to make them usable from the Win32 API)?

http://www.handyarchive.com/free/type1/

Classic PS Fonts, for example. They are supported by Win 2000 and XP (and maybe NT).

I had a look at adding non-truetype font support to pangowin32. It is not
straightforward at all. 

Well, it's straightforward to take out the single test in pangowin32-fontmap.c
that now skips non-TrueType fonts, of course, but pango will then crash later if
trying to use such a font.

Implementing the same functionality that's in pangowin32 for TrueType fonts is 
complex. I.e. coverage calculation (the TrueType code looks directly at the
TrueType font data using GetFontData()), fetching glyph indices, etc.

Once again, continuing to support Win9x (Win98 at least) also complicates
matters. Much potentially useful font-related Win32 API isn't available on
Win9x, for instance GetGlyphIndices() and GetFontUnicodeRanges().

How does one install that "Classic PS" font on Windows XP? I tried copying both
the CLR_____.PFM and CLR_____.PFB file found in the zipfile to the Windows Fonts
folder, but it didn't recognize them.

Er, scratch that. I managed to install it using Explorer's Install New Fonts.
(The catch apparently is that both the .PFM and .PFB file must get copied at the
same time.) Will check if it is any easier to support Type1 fonts than raster
fonts. I doubt it, though.

You need only to copy that font file which has an associated icon (I think it's
PFM). When you copy it to Fonts folder, other files will be copied automatically
(they must be in the same folder).

Noting that this is causeing a regression in Inkscape, which supported OpenType fonts in the 0.43, but doesn't in the latest nightly builds. (https://sourceforge.net/tracker/?func=detail&atid=604306&aid=1441767&group_id=93438 includes a test font.) 

This bug only affects the Windows GDI builds? Is it possible to make a freetype pango build for Win32?

How important is Win 98 to the pango developers now? Inkscape has stopped support for Win 98 and ME.

Considering that the very way coverage and glyph indexes are determined is by looking at the TrueType cmap tables using GetFontData(), I fail to see how this could work for such OpenType fonts that are just wrapped PostScript fonts? Oh well, I guess using the Win2k-and-later-only GetGlyphIndices() and GetFontUnicodeRanges() would work. Win9x support is totally unimportant to developers. It might be important to users, but they can't use Pango >= 1.10 anyway, as cairo uses NT-only APIs ;)

Whether TrueType or PostScript outlines are used in an OpenType font, the cmap table must be present. (so saith http://www.microsoft.com/OpenType/OTSpec/otff.htm). 

I've used a hack suggested by inkscapes cyreve: change libpangowin32-1.0-0.dll bytes 0x36fd and 0x36fe from 04 75 to 02 72) without difficulties.

Is this a *binary* patch against the DLL. Or do you have changed some source code?
If the latter a patch as unified against current cvs would be nice.

Cyreve gave a recipe for a binary patch. In the current version of libpangowin32 those bytes are actually at offset 0x377d 

I've asked him to comment here.

Yes, it's a binary patch. I've never managed to summon the patience to build Pango from source.

The equivalent source patch is to change pangowin32-fontmap.c:167 to

if (fontType < TRUETYPE_FONTTYPE)

but note that this was a trick based on the values of the #defines in order to make the patch easy. What you really want is

if (fontType != TRUETYPE_FONTTYPE && fontType != DEVICE_FONTTYPE)

however that's still not very good because it doesn't distinguish between all the various types of device fonttype. Pango can only deal with TrueType/OpenType fonts (no matter how the outlines are encoded). It cannot cope with so-called 'vector' fonts or Type1 fonts. You probably want to play with NEWTEXTMETRIC::ntmFlags to find out more.

I don't understand. You say: "Pango can only deal with TrueType/OpenType fonts". Well, isn't that then exactly what the code in pango_win32_enum_proc() does: it skips all fonts that aren't TrueType?

Or is it so that the fontType parameter to a EnumFontFamExProc actually is a bit map, should the test be if (!(fontType & TRUETYPE_FONTTYPE))? I don't have any fonts on my XP that would turn up as fontType 5 or 6, though.

No, because it seems that OpenType fonts with PostScript outlines have DEVICE_FONTTYPE. Windows abstracts how the outlines are actually encoded so Pango doesn't need to know that, but it cannot emulate GetFontData() for non-(True|Open)Type font containers. I think that OpenType can be thought of as being a bit like the .avi format, where there's a common container from which you can obtain nearly all the metadata, but once you get to the data itself it could be using any codec.

The MSDN says that the fontType parameter is a bitmask, but (in my admittedly very limited experience) I've never seen more than one value set. A quick survey says: The vast majority are TRUETYPE_FONTTYPE (comprising OpenType/TT and TT/TT), OpenType/PS are DEVICE_FONTTYPE, Type1/PS are DEVICE_FONTTYPE, the bitmap fonts are RASTER_FONTTYPE, and the vector fonts are 0.

Hmm, I am a bit confused now. The subject of this bug is that Pango on Windows doesn't work for non-TrueType fonts. The reason why this is so is because the code looks into the TrueType font data using GetFontData(). So, to avoid looking at non-TrueType fonts, they are filtered out early with the if statement described above. Just changing that if statement to let non-TrueType fonts through isn't magically going to make the rest of the code able to handle them.

No, it isn't. But it appears that it is unfairly filtering some fonts that do actually work, because of bizarreness in Microsoft's APIs.

*** Bug 350730 has been marked as a duplicate of this bug. ***

cairo is using GetFontData (hdc, OPENTYPE_CFF_TAG, 0, NULL, 0) != GDI_ERROR to test for Postscript OpenType fonts.

http://gitweb.freedesktop.org/?p=cairo;a=commitdiff;h=da1019c9138695cb838a54f8b871bbfd0e8996d7

Perhaps this test can be used in pango_win32_enum_proc.
Checking fontType == DEVICE_FONTTYPE first may be an optimization if performance is an issue.

Do all Postscript OpenType fonts have the OPENTYPE_CFF_TAG table?  If not, testing any table expected in OpenType fonts, such as HEAD, may be fine.

I'm not sure how NEWTEXTMETRIC::ntmFlags is meant to be used.  Although there is a NTM_PS_OPENTYPE flag it seems the NEWTEXTMETRIC is only available for TrueType fonts.

http://msdn2.microsoft.com/en-us/library/ms534234(VS.85).aspx
"The function uses the NEWTEXTMETRICEX structure for TrueType fonts; and the TEXTMETRIC structure for other fonts."

I only way I can see to detect whether the NEWTEXTMETRICEX structure is available is to check FontType == TRUETYPE_FONTTYPE.

(In reply to comment #24)
> I'm not sure how NEWTEXTMETRIC::ntmFlags is meant to be used.  Although there
> is a NTM_PS_OPENTYPE flag it seems the NEWTEXTMETRIC is only available for
> TrueType fonts.

NEWTEXTMETRIC::ntmFlags is available on Windows 2000 and XP (but not NT?):

http://msdn2.microsoft.com/en-us/library/ms533913(VS.85).aspx
"...EnumFontFamiliesEx functions have been modified to return pointers to the ENUMTEXTMETRIC..."

Created attachment 102397 [details] [review]
Support OpenType/PS fonts

The attached patch adds support of OpenType fonts PostScript outlines.

OpenType fonts are a superset of TrueType fonts. All tables that are
mandatory in TrueType (including cmap) are also mandatory in
OpenType. The one exception is that the 'glyf' table containing the
TrueType outlines may be substituted with a 'CFF ' table containing
Compact Font Format encoded Type 1 or Type 2 outlines.

Windows 2000 and later supports OpenType fonts with either TrueType or
PS outlines. All of the Windows API that works with TrueType fonts
also works with OpenType/PS fonts.

Fixing Pango to work with OpenType/PS fonts only requires modifying
the test in pango_win32_enum_proc that filters out unsupported fonts.
The TRUETYPE_FONTTYPE bit of the fontType parameter matches both
TrueType fonts and OpenType fonts with TrueType outlines. However the
fontType parameter does not provide anyway to test for OpenType/PS
fonts.

The way to test for OpenType/PS fonts is with the second parameter of
pango_win32_enum_proc that contains a NEWTEXTMETRIC structure. The
ntmFlags field of this structure includes bits for testing for
OpenType/TT, OpenType/PS and Type1 fonts. Note that the OpenType/TT
flags is only set for OpenType fonts with TrueType outlines, not for
TrueType fonts so the TRUETYPE_FONTTYPE bit fontType parameter still
needs to be checked.

The EnumFontFamExProx documentation contains the statement "The
function uses the NEWTEXTMETRICEX structure for TrueType fonts; and
the TEXTMETRIC structure for other fonts." However this does not make
sense. EnumFontFamExProx is the only Windows function I can find that
provides the NEWTEXTMETRIC structure. If it is only provided for
TrueType fonts the additional bits for identifying OpenType/PS and
Type 1 fonts would serve no purpose.

I've tested the EnumFontFamExProc on Windows XP and it is providing
the NEWTEXTMETRIC struct and ntmFlags contains the correct font types
for all the font types I tested (TrueType, OpenType/TT, OpenType/PS,
Type1, Stroke, and Bitmap).

OpenOffice uses the ntmFlags in the NEWTEXTMETRIC provided by
EnumFontFamExProc without using any checks for the Windows version. As
OpenOffice runs on Win98 and later it is clear that EnumFontFamExProc
always provides this struct on at least Win98 and later.

The Windows documentation also mentions that the additional NTM flags
such as NTM_PS_OPENTYPE are only supported on Win 2000 or later. This
is because earlier version of Windows did not support OpenType and
Type 1 fonts. These flags are probably set to zero on earlier versions
of Windows as the OpenOffice code does not contain any version checks
when using them. However I included a >= Win2000 check in my patch
since pango already has that information.

The attached patch works for me on WinXP. Cairo >= 1.4.0 supports
OpenType/CFF fonts on Windows however the current snapshot 1.5.4 has a
bug that breaks OpenType/CFF fonts on Windows. I have fixed this in
the git repository.

My version of MingW does not have the #defines for NTM_PS_OPENTYPE so
the patch defines this if it is not already defined.

Created attachment 102398 [details] [review]
Add support for Type 1 fonts

While working on fixing the broken handling of Type 1 glyph indices in
cairo I did some testing with pango to see how it handles Type1 on
Windows only to find it doesn't.

To be able to test Type 1 fonts with cairo on Windows I created this
patch. It enable Type 1 fonts in pango_win32_enum_proc. In
pangowin32.c, if the cmap lookup fails it falls back to using
GetGlyphIndices in the pango_win32_font_get_glyph_index and
GetFontUnicodeRanges in pango_win32_font_calc_coverage. These two
functions are Win 2000 or later API however only Windows 2000 and
later have Type 1 font support.

There are also two areas where GetFontData lookups of the 'name' table
need to have suitable fallbacks. One name lookup is in get_family_name
in pangowin32-fontmap.c. However this does not lookup the 'name' table
if the font name is in ascii which I think all Type 1 fonts are. There
is also a name lookup in pango_win32_font_calc_coverage related to
CJKV fonts. I don't know what that is for but failing the lookup did
not cause a problem for my testing.

This only works with the git version of cairo as I have just pushed
out a fix for the Windows Type 1 font problems.

The patch was good enough to do my testing with cairo. I do not know
enough about pango to be able fix the remaining limitations of the
patch.

Tor, any review here?  These would be nice to have in 1.20.

Thanks.

Tor, ping?  Getting late.

I committed the first patch, thanks!

I am still looking at the second one. Adrian, you say that this only works with a git version of cairo, what happens if one uses it with cairo 1.4.14? (Yeah, I should go looking for some sample Type 1 font and check myself...)

Adrian, do you think it would be a good idea to use GetGlyphIndices() and GetFontUnicodeRanges() for *all* font types, not just Type 1? If I recall correctly, the reason why pangowin32 hasn't used them is because they were not in Win9x. But Pango has explicitly dropped Win9x support already in 1.16(.1). Then the code to read the cmap tables could be dropped, couldn't it?

Hmm, one reason not to use GetFontUnicodeRanges() (and maybe GetGlyphIndices, too) is its lack of support for characters outside the BMP. The code in Pango that reads cmap tables handles format 12 tables which cover them. 

(To be complete, Pango would need to handle the other cmap formats, too. See http://www.microsoft.com/OpenType/OTSpec/cmap.htm . I wonder whether these much exist in the wild, though?)

I don't see how GetFontUnicodeRanges() could support characters outside the BMP, though, as WCRANGE uses a WCHAR, which is 16 bit. Maybe they use a trick with two WCRANGEs (representing a surrogate pair) to indicate a single range outside the BMP? Will have to do some experimentation... wonder if I have any font with coverage outside the BMP.

Nope, GetFontUnicodeRanges() does not try any tricks to return coverage outside the BMP. And anyway, some writing on the net seem to say that GetFontUnicodeRanges() isn't entirely reliable otherwise either. So it seems to be best to continue parsing the cmap tables ourselves, and rely on GetFontUnicodeRanges() only when no format 4 or 12 cmap is present.

(In reply to comment #30)
> I committed the first patch, thanks!
> 
> I am still looking at the second one. Adrian, you say that this only works with
> a git version of cairo, what happens if one uses it with cairo 1.4.14? (Yeah, I
> should go looking for some sample Type 1 font and check myself...)
> 

Using cairo 1.4.14 to display Type 1 glyphs with cairo_win32_surface should work. The bug is in cairo-win32-font.c which affects printing and font metrics. The problem is inconsistent and incorrect use of ETO_GLYPH_INDEX. As a result depending on which combination of cairo_show_glyphs()/cairo_show_text() and display/printing you are using you may get the wrong glyphs.

The problems was due to a bug in the GetCharacterPlacement() call used by cairo_show_text(). It was returning unicode chars instead of glyph indices for Type 1 fonts. At the time it was assumed that the problem was with Type 1 fonts rather that with GetCharacterPlacement(). As a result ETO_GLYPH_INDEX was avoided for Type 1 fonts in cairo-win32-font.c however cairo-win32-surface still used ETO_GLYPH_INDEX for all fonts.

Now that I know GetCharacterPlacement() is the problem and GetGlyphIndices() can be used to get Type 1 glyphs indices, ETO_GLYPH_INDEX is used for all fonts and a workaround for cairo_show_text() has been implemented that uses GetGlyphIndices() for Type 1 fonts. This should all be working correctly in the cairo 1.5.8 snapshot.


If you have Ghostscript installed the Ghostscript fonts directory is a good source of Type 1 fonts.

Any ideas on how we can ensure the GetFontData() lookups of the 'name' table are avoided for Type 1 fonts?

The 'name' lookup in pangowin32-fontmap.c is not done if the font name is all ASCII. Could we add some code to pango_win32_enum_proc() to exclude Type 1 fonts with non ASCII names (if there is such a thing). Or would this code need to go in pango_win32_inner_enum_proc()?

The other call to GetFontData() is in pangowin32.c. This name lookup seems to be related to CJKV fonts however I don't understand what that code is doing.

> Any ideas on how we can ensure the GetFontData() lookups of the 'name' 
> table are avoided for Type 1 fonts?

I guess one chould save the type of the font in the, uh, PangoWin32Face struct?

> The other call to GetFontData() is in pangowin32.c. This name lookup seems to
> be related to CJKV fonts however I don't understand what that code is doing.

Because the Han characters are unified, it's hard to say just from the coverage whether a font is designed for Traditional Chinese, Simplified Chinese, Japanese or Korean (or Vietnamese even, but I don't know if they actually use different style of glyphs). So when calculating language-specific coverage one cannot assume that a font that contains unified Han glyphs suits for any of the CJKV languages. 

The font_has_name_in() function checks if the font has a name in the CJKV language passed as a parameter. For example, if a font has a name record that is in Traditional Chinese, it is assumed that the glyphs too then are of the traditional variant, etc. Apparently this works fine, at least I don't think anybody has complained.

After spending some more time studying the code that calls GetFontData() to lookup the 'name' table I have come to the conclusion that the changes to pangowin32-fontmap.c in my Type 1 font patch are correct.

The 'name' table lookups in pangowin32-fontmap.c are called from 
pango_win32_font_description_from_logfont() and pango_win32_font_description_from_logfontw(). Both functions have a fallback path in the event that the GetFontData() call fails. It is also possible to call these functions with a Type 1 font specified by the LOGFONT without my patch.

I'm not 100% certain about the correctness of the changes to pangowin32.c. The existing code sets PANGO_COVERAGE_APPROXIMATE on Han characters when the font name language does not match the coverage language. As it is not possible to obtain a language from a Type 1 font name, should I always set PANGO_COVERAGE_APPROXIMATE for Han characters in Type 1 fonts?

It is unlikely that anyone is using Type 1 fonts for CJK languages due to the limitations of the 8-bit encoding. I expect all Type 1 fonts for CJK languages are of the CID type which I don't believe Windows supports.

Created attachment 105563 [details] [review]
Add support for Type 1 fonts

I've updated the Type1 patch to set the coverage of Han characters to PANGO_COVERAGE_APPROXIMATE for Type 1 fonts because we can not determine the language of a Type 1 font name. I've also moved the CJKV language test in pango_win32_font_calc_coverage() to after the call to font_get_cmap() so that the GetFontData 'name' table lookup is not called for Type 1 fonts.

This patch should now be complete with no known problems remaining.

Thanks. Committed the last patch, too, to trunk. Have you had any fresh thoughts about this in the meantime, or should the non-TrueType font support be fine now? (Reopen this bug if necessary.)

ChangeLog entry:

2008-02-24  Tor Lillqvist  <tml@novell.com>

	Bug 515484 – Pango on Windows is missing Type 1
	font support
	Patch from Adrian Johnson.
	
	* pango/pangowin32-private.h (PangoWin32Face): Add has_cmap field
	that tells whether the font has a cmap or not. A Type 1 font
	doesn't.

	* pango/pangowin32.c (pango_win32_font_get_type1_glyph_index): New
	static function. Uses GetGlyphIndicesW() to get the glyph indices
	for Type 1 fonts. Possibly also TrueType fonts that for some
	reason lack the cmap formats we understand.
	(pango_win32_font_calc_type1_coverage): New static function. Uses
	GetFontUnicodeRanges() to get the coverage for Type 1 fonts, and
	possibly TrueType fonts that lack the cmap formats we understand.
	(pango_win32_font_get_glyph_index): Set has_cmap to false if the
	font doesn't have a cmap. Call
	pango_win32_font_get_type1_glyph_index() in that case.
	(pango_win32_font_calc_coverage): Set has_cmap to false if the
	font doesn't have a cmap. Call
	pango_win32_font_calc_type1_coverage() in that case.
	
	* pango/pangowin32-fontmap.c (pango_win32_enum_proc): Accept also
	Type 1 fonts.
	(pango_win32_insert_font): Initialise has_cmap tentativaly to True.

(In reply to comment #38)
> Thanks. Committed the last patch, too, to trunk. Have you had any fresh
> thoughts about this in the meantime, or should the non-TrueType font support be
> fine now?

Should be all fine now.