GNOME Bugzilla – Bug 611943
CoreText backend is needed
Last modified: 2011-04-16 20:06:55 UTC
ATSUI has been deprecated since Mac OS 10.5 and is only partially supported in the 64-bit versions of Mac OS 10.6. The CoreText framework is the replacement for ATSUI. In order to provide support for Pango on Mac OS X in the future, a backend implemented using CoreText is needed. I have written a CoreText backend for Pango over the last few days. It is almost at the same level as the old ATSUI backend (with respect to character coverage support it is even better) and thus almost ready for review. For reviewal I will most likely push the git branch somewhere. I will notify in this bug when that has been done. This new backend does not use any objective-C API and is fully written in pure C. Once the new backend has been accepted for inclusion, I would propose we create a new coretext component to track bugs.
Thanks Kris. You can push the branch to the pango git repo.
I am still working on getting font fallbacks properly working. It took some time to figure out how to do this, especially as we do not have access to the font cascading lists on OS X in the same detail as we have with fontconfig. I do have a plan to get font fallbacks properly working now and this involves writing an implementation of a PangoCoreTextFontset, not using the simple fontset as the old ATSUI backend is doing. This will introduce quite some changes to the backend, so I plan to get this implemented and working first and will then publish the branch for testing and review.
That sounds like the right thing to do. Thanks Kris for working on this.
I am running into some problems while implementing fontset support in the CoreText backend. This fontset code is largely based on the PangoFcFontset. I am hoping that Behdad can help out a bit here, since he knows much more FontConfig details than I do. I might be overlooking things as well. Apologies in advance for the somewhat long comment. Ideally when a fontset is loaded, I wanted to translate the given PangoFontDescription and language into a CTFontDescriptor and then use the CoreText infrastructure to supply the best matching fonts. It looks like CoreText does very strict matching of the given attributes in a CTFontDescriptor, in the sense that it will very quickly fall back to the default fallback font ("Lucida Grande"). One problem here is that weight is expressed as a double between -1.0 and 1.0, with 0.0 being "regular". This does not map nicely to PangoWeight, you'll get rounding errors, etc. If a value of 0.4 is mapped onto a PangoWeight value and then mapped back it might end up being 0.5, in this case CoreText will not find a match. How would FontConfig handle this? For loading appropriate fonts, I see three options: i) Translate PangoFontDescription into CTFontDescriptor. Create a font collection that will consist of all fonts that match this font descriptor. This will only include fonts that fully match the properties in the descriptor, so potentially none. There is no "this font comes closest" match. ii) Same translation of a PangoFontDescription into a CTFontDescriptor. But this time we use "CTFontCreateWithFontDescriptor" using the font descriptor to get the best matching font. As far as I can see, this seems to just return the default fallback font very quickly. For example, requesting Helvetica with a weight of 0.5 (which does not exist) will give you the default fallback font, instead of Helvetica with a weight of 0.4 that does exist. iii) Translate the PangoFontDescription into a PangoFontFace by iterating over families and faces using pango_font_description_better_match(). This is essentially what the ATSUI backend is doing (see find_best_match() in pangoatsui-fontmap.c). This will give you a PangoFontFace that can yield an exact translation into a CTFontDescriptor (in the worst case by caching the CTFontDescriptors). Given the weight mapping problem outlined above and also the fact that it looks like that some fonts can only be distinguished by using their style name or postscript name (that cannot be stored in a PangoFontDescription, unless it is added to the family name somehow), using a combination of options i and iii might be the best one. Then there is the issue of font cascading/substitution. The main issue here is that we do not have as extensive access to the sorted/ordered font tables as we have with FontConfig. There is API advertised to obtain the cascade list from a font via font attributes. Unfortunately, this attribute value seems to be always unset. In such a case the "global cascade list" is used, which, as far as I can see, is not accessible using public API. I have found a function called "CTFontCopyDefaultCascadeList" that is exported by the CoreText library, but not defined in the header file nor mentioned in the API documentation. I am very wary if we should use this function. For loading fontsets, my current best attempt is to do this by loading a font from a font descriptor that requires support for the language specified in the load_fontset call. If this does not yield a font, then unset the family and try again so that we do get a font that can actually render the language requested. There actually is a function called "CTFontCreateUIFontForLanguage" which does either not do what we want/expect or does not work as advertised. What does work really well is "CTFontCreateForString". Given a font and a string, it will select a font that can best map the characters in the string based on the given font. It uses the cascade list internally. Although this function works very well, it is very hard if not impossible to use this in a Pango backend. (Unless we want to do things like scanning the entire character set or translating a language into a representative character for that language). The "unset family trick" works okayish, but only if the language has been guessed correctly by Pango, which is not always the case. In order to get full coverage we do need to either use "CTFontCreateForString", or the "private" function for getting the default cascade list. We would be appending this default cascade list to the font matching the font descriptor in each fontset.
Does CoreText backend could works on 10.6 now ?
Created attachment 183997 [details] [review] Patch to replace ATSUI with CT when using a 64bit compiler Minimal patch to get rid of ATSUI and replace essential functions by CoreText equivalent ones.
More comments on my patch: I needed to get rid of the ATSUI calls mainly for performance reasons (I'm working on a code editor, and textarea performance is a priority). So I made a very simple translation to CoreText that is present in the above patch. For compatibility reasons with older version of OSX, the coretext API is only activated with 64-bit compilers, so that the 'atsui' module remains compatible with Tiger. From what I see above, my contribution might be somewhat of a limited interest, but if anything is useful, please feel free to use it !
What your patch does is basically included in the code I was talking about in the opening comment. Though I completely forgot about it and never published the branch somewhere (bad me). I will actually work on getting the branch public now, it introduces a whole new "coretext" module without touching the "atsui" module, so we can avoid using a lot of #ifdef in the code.
I just pushed the first part of my coretext branch (rebased onto latest master): http://git.gnome.org/browse/pango/log/?h=coretext It is pretty much a copy from the ATSUI code, but then entirely transformed to CoreText, so it does not depend on ATSUI at all. It works pretty well actually. Apologies for letting this rot on my disk for a year ... Why did I let this rot on my disk? I was looking into properly implementing font sets, so we could do font fallbacks and cascading properly. Turns out this is pretty hard to do with CoreText, as these are all tasks that are actually handled by CoreText itself. Anyway, see comment 4 for the gory details. So for now the CoreText backend does work, we could even release it for a start. I am halfway through implementing the proper fontset support in this backend, but that code will remain on my disk only until it is finished. I hope to this give another shot in the near future.
Created attachment 185686 [details] [review] Patch on configure.in, applied to the coretext branch I just tried the coretext branch and saw issues with the coretext module not being loaded correctly. After some investigation, it seems there is an issue in configure.in (copy/paste problem) that made the coretext module not being loaded correctly if the atsui module is not activated. Attached is the patch correcting the issue
Kris, lets merge this into master.
(In reply to comment #10) > Created an attachment (id=185686) [details] [review] > Patch on configure.in, applied to the coretext branch > > I just tried the coretext branch and saw issues with the coretext module not > being loaded correctly. After some investigation, it seems there is an issue in > configure.in (copy/paste problem) that made the coretext module not being > loaded correctly if the atsui module is not activated. > > Attached is the patch correcting the issue Good catch! I will put this change on the branch... (In reply to comment #11) > Kris, lets merge this into master. ... and merge the branch into master. Once I get the time to do the further work, i.e. doing fontsets properly, I will work on a separate branch again and merge that only after review. I will try to merge tonight or otherwise tomorrow.
It's in master now. I will open a new bug for the fontset/cascading/coverage features that I still need to finish implementing.
We have added a coretext component to the Pango product for tracking CoreText-specific bugs. The bug for the font fallback enhancement has been filed as bug 647969.