Automatic hinting in Fontforge will result in poorly hinted glyphs with varying x-heights at various sizes. (d, b, 0 in particular).

Created attachment 269040 [details] [review]
autohint Cantarell

- introduces numerous issues with varying x-height
  for numerous glyphs.

Created attachment 269041 [details]
example of poorly hinted b and d (strong hinting).

For now, would shipping a fontconfig file with Cantarell that will tell the renderer to only use slight hinting and not full hinting be an appropriate workaround/fix?

(In reply to comment #4)
> For now, would shipping a fontconfig file with Cantarell that will tell the
> renderer to only use slight hinting and not full hinting be an appropriate
> workaround/fix?

While the provided PS hints are not right in Cantarell, the major regression with the move to Freetype 2.5 is the increated weight/gamma/darkening. Only slight hinting retains the original darkness of the fonts, so I'd say it's an appropriate workaround for now.

Created attachment 275033 [details] [review]
[PATCH] fontconfig: try to force slight hinting

Here's a patch.

Unfortunately, it doesn't work.

It seems that either the desktop hinting setting will always override fontconfig, or I'm doing it wrong.

I think the reason the fontconfig snippet does not work might be this bug: https://bugs.freedesktop.org/show_bug.cgi?id=11838

(I actually want slight hinting for TTF and medium/full for CFF but this does not work with GTK/Cairo due to above bug; other applications not using Cairo are fine.)

A bug report with a detailed description of the issues and comparison screen shots can be found here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=798805

The solution the bug reporter seems to look for is documented in http://freetype.org/freetype2/docs/reference/ft2-cff_driver.html#no-stem-darkening. An alternative way is to patch FreeType's CFF font driver to disable the functionality by default.

Stem darkening is a desirable feature that ***REQUIRES*** the rendering library (e.g. cairo, Skia, whatever Qt5's is called) to apply gamma correction when putting glyphs on a surface. It is on by default because it is the right thing to do.. when rendering libraries play along. Which none on Linux do by default, even when e.g. Skia (-> Chrome) and Qt5 have the capability, it's just disabled by default.

The problems with Cantarell the font are a separate issue which I'm going to open a bug report on next.

Is it possible to set this property for a specific font by means of fontconfig?

If fontconfig has added access to the property recently, yes. Otherwise, not that I know of. You could use the .ttf version of the font, neither the TrueType driver nor the autohinter currently do stem-darkening (I'm working on the autohinter right now to do just that, but it's probably not hitting mailine FreeType until at least 2.7). If you go the .ttf route, check if the bytecode (hinfull) looks good, otherwise use the autohinter (hintslight).

If you want to keep shipping CFF fonts (yay! Go CFF!): Does Debian ship with FT_CONFIG_OPTION_SUBPIXEL_RENDERING enabled? If so, disabling stem darkening in general for the CFF renderer is just one more patch away. In the the freetype2 source:

src/cff/cffobjs.c
1055:    driver->no_stem_darkening = FALSE;

Change to TRUE. This should do the trick, but I haven't tested it here.

Please keep in mind that this is a temporary hack until the font rendering ecosystem works together to do the right thing... at least before the coming of  glorious 200dpi screens where tricks like this become unnecessary.

Now, back to fixing up hinting on Cantarell...

I've known about the stem darkening flag, but disabling it within CFF is not a solution. The font looks the same regardless of the no_stem_darkening boolean value.

Then you mean the inconsistent letter forms? Cantarell had wrong blue zones and is riddled with off-by-ones-or-twos. Have a look at bug 642395, I'm on it.

Fix is in master.

While the fixes in 0.0.17 improve the font dramatically it is not completely fixed.

- Full/Medium hinting causes the font to become heavy, almost like it is BOLD. This was occurring before 0.0.17.
- Slight hinting makes the font almost identical to freetype < 2.5, but slight hinting causes monotype fonts to become BOLD. Terminal emulators look dramatically different. I'll have to carry a fontconfig file from comment 6.
- D, h, I, m, R, t, T vertical hints are too strong causing blurring of what should be straight lines into smudges.

Examples (Medium hinting)
Freetype 2.4.11 / Cantarell 0.0.16: http://i.imgur.com/s3KTAKb.png
Freetype 2.4.11 / Cantarell 0.0.17: http://i.imgur.com/w9JOEbn.png
Example (Slight hinting)
Freetype 2.5.5  / Cantarell 0.0.17: http://i.imgur.com/EAAnf84.png

These examples used fontforge to build the font.

Generally, the examples look more or less fine to me (attention! I'm accustomed to fonts looking similar to your screenshots!). I think there are a few things going on here.

1. Are you generating .ttfs from the .sfds? Because your medium/full examples look like glyphs are snapped on the Y *and* X axis (slight hinting: Y axis only). How do you hint them? FreeType's ttfautohint?

2. All the hinting fixes (outside outline adjustments) I made target the .otf file format and do nothing for .ttfs. What you do to hint them determines the end result. I recommend you use the .otfs provided by the repo together with FreeType 2.5+, that's what they're made for. Use hintfull or hintmedium (the same thing for OpenType/CFF fonts) for best results, hintslight will also do.

3. You're comparing whatever automatic hinting happened on Cantarell to the almost overdone hinting on DejaVu Sans Mono (default font on the console). The DejaVu font family was hinted extensively to look like fonts on Windows XP, just with more antialiasing. This requires *massive* amounts of time to do manually. If automatic tools like ttfautohint won't do, you are probably out of luck. Basically, if you want all your fonts to look the way DejaVus do, you need to exclusively use extensively hinted fonts like those shipped with Windows. But even that might not save you -- the modern fonts it ships rely mostly on ClearType to look good and FreeType can't handle ClearType fully yet afaik.

4. Hintslight in general and hintmedium/full for .otfs snap glyphs to the Y axis only. This preserves much of the original design while reducing blur on the low-DPI screens we are currently stuck with. ClearType tries the same, but is not yet fully handled by FreeType.

5. What I want to say with 4. is, hintslight does not make DejaVu Sans Mono become bold -- it *is* bolder (by design) than what your brain is accustomed to seeing, hintfull/medium just (by design) brutally beats the outlines into the grid. .otfs with hintfull will render even bolder than designed because the OpenType/CFF engine emboldens glpyhs slightly when rendering at small sizes to enhance readability.

6. The smudges you see are the result of not fitting stems exactly to the grid *and* missing gamma correction. See my comment 9. In a perfect font rendering world, Cantarell looks like here (pre 0.0.17): https://bug642395.bugzilla-attachments.gnome.org/attachment.cgi?id=312047



So, long story short, what can you do?
1. Try the .otfs provided in the repo
2. Try to get used to fonts not DejaVu looking bold in comparison
3. Try a different console font, I recommend Source Code Pro.
4. Font rendering on Linux is still a messy affair and there are no easy solutions on the horizon. Your best bet, imho, is to use this /etc/fonts/conf.d/10-autohint.conf file:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
<match target="font">
  <edit mode="assign" name="hinting">
    <bool>true</bool>
  </edit>
  <edit mode="assign" name="autohint">
    <bool>true</bool>
  </edit>
  <edit mode="assign" name="hintstyle">
    <const>hintslight</const>
  </edit>
</match>
<match target="font">
  <test name="fontformat">
    <string>CFF</string>
  </test>
  <edit mode="assign" name="autohint">
    <bool>false</bool>
  </edit>
  <edit mode="assign" name="hinting">
    <bool>true</bool>
  </edit>
  <edit mode="assign" name="hintstyle">
    <const>hintfull</const>
  </edit>
</match>
</fontconfig>

and get used to the look. It enables slight hinting globally and full hinting only for .otfs (GNOME ignores it by default and will use what you set in gnome-tweak-tool). This gets you the highest-fidelity rendering at the expense of sharpness. I got used to it long ago and like it. .otfs will render bolder, in a future FreeType release, slightly hinted fonts will, too, just so you know. The missing piece then will be gamma correction.

Oh, and maybe rebuild FreeType with #define FT_CONFIG_OPTION_SUBPIXEL_RENDERING in include/freetype/config/ftoption.h. Glyphs will look less smudgy.

> Does Debian ship with FT_CONFIG_OPTION_SUBPIXEL_RENDERING enabled?

Yes.

> If so, disabling stem darkening in general for the CFF renderer is just one more patch away. 

It's a shame it is not possible to switch these features at runtime. :(

I'm using Cantarell 0.0.17 on Debian with current Freetype, and I'm still seeing blurry rendering, with lines smeared across pixels rather than snapped to pixel edges.  I'll attach a screenshot; look at the title bars and the gnome-shell UI.

Created attachment 312832 [details]
Cantarell 0.0.17 showing blurry rendering, with lines smeared across pixels

@Josh Triplett: The "blurry" you're seeing is expected rendering :) See my comment 16. Never expect DejaVu-like ultra-sharp rendering with anything but extensively hinted .ttfs and hintfull. However, demand gamma correction from the rendering libraries you use so that stuff looks like https://bug642395.bugzilla-attachments.gnome.org/attachment.cgi?id=312047.

Yes, I know having DevaVu-like rendering and fuzzy Cantarell in the same UI is jarring. My recommendation is to switch to slight hinting globally and full hinting only for .otfs and get used to it.

Oh, and your Cantarell-Bold seems to use a different rendering method? Are you using both .otfs in 0.0.17? It should be just as "fuzzy" as the regular face.

(In reply to madigens from comment #21)
> @Josh Triplett: The "blurry" you're seeing is expected rendering :) See my
> comment 16. Never expect DejaVu-like ultra-sharp rendering with anything but
> extensively hinted .ttfs and hintfull. However, demand gamma correction from
> the rendering libraries you use so that stuff looks like
> https://bug642395.bugzilla-attachments.gnome.org/attachment.cgi?id=312047.

While that's an improvement, it's still not something I could manage to look at for any significant amount of time without getting a headache.  I definitely don't want to make my fonts look that blurry.  I don't care as much fonts looking "too bold"; I'll worry about that after I figure out how to stop stems from smearing across pixels.

> Yes, I know having DevaVu-like rendering and fuzzy Cantarell in the same UI
> is jarring. My recommendation is to switch to slight hinting globally and
> full hinting only for .otfs and get used to it.

Making my crisp fonts blurry so they match?  No thank you.  But if you can suggest a way to make my blurry fonts crisp, I'd love to hear it. :)

Switching to slight hinting does actually improve the rendering of Cantarell somewhat (not perfectly, but somewhat), but makes DejaVu blurry.  Switching to the autohinter similarly improves Cantarell rendering.  Note that I use the Cantarell .otf files, as packaged in Debian.

As far as I can tell, this is the standard Mac versus Windows font-rendering debate (designer-intended shapes versus crisp rendering), to which my answer is that I'll happily sacrifice "intended" font shape for rendering that recognizes that displays have pixels and don't have infinite resolution.  When all the screens I use hit 300+ DPI (note that 200 doesn't suffice), I'll stop wanting stems snapped to pixel boundaries.

I would be tempted to just switch from Cantarell to DejaVu Sans, except that as far as I can tell, several bits of the GNOME UI rely on the application font to provide significant interline spacing, and switching to DejaVu removes that interline spacing, making the UI feel more crowded.

> Oh, and your Cantarell-Bold seems to use a different rendering method? Are
> you using both .otfs in 0.0.17? It should be just as "fuzzy" as the regular
> face.

Which part are you referring to?  All the fonts in the screenshot I posted in #20 are blurry.

(In reply to Josh Triplett from comment #22)
> I would be tempted to just switch from Cantarell to DejaVu Sans, except that
> as far as I can tell, several bits of the GNOME UI rely on the application
> font to provide significant interline spacing, and switching to DejaVu
> removes that interline spacing, making the UI feel more crowded.

Also, I actually *prefer* the shape and aesthetic of Cantarell; I think it's an improvement.  I'd just like to see it rendered more crisply, as it used to. :)

> While that's an improvement, it's still not something I could manage to look
> at for any significant amount of time without getting a headache.  I
> definitely don't want to make my fonts look that blurry.  I don't care as
> much fonts looking "too bold"; I'll worry about that after I figure out how
> to stop stems from smearing across pixels.

I'm sure you'd get used to it eventually. Like people who switch between Windows and Mac OS X and first find the respective font rendering weird but get used to it over time.
 
> Making my crisp fonts blurry so they match?  No thank you.  But if you can
> suggest a way to make my blurry fonts crisp, I'd love to hear it. :)

That's the problem I tried to explain in comment 16. If you want a razor-sharp Cantarell, *someone* would need to hint it like the DejaVu family. Don't count on me, that would require weeks or months of full time work... Maybe you'll get lucky with ttfautohint?

> Switching to slight hinting does actually improve the rendering of Cantarell
> somewhat (not perfectly, but somewhat), but makes DejaVu blurry.  Switching
> to the autohinter similarly improves Cantarell rendering.  Note that I use
> the Cantarell .otf files, as packaged in Debian.

The autohinter doesn't yet darken glyphs. Until I land my changes in FreeType. *evil laughter*. Ahem, you can switch off stem-darkening for .otfs with full hinting when compiling your own FreeType or waiting for fontconfig to throw up a switch for it.

> As far as I can tell, this is the standard Mac versus Windows font-rendering
> debate (designer-intended shapes versus crisp rendering), to which my answer
> is that I'll happily sacrifice "intended" font shape for rendering that
> recognizes that displays have pixels and don't have infinite resolution. 
> When all the screens I use hit 300+ DPI (note that 200 doesn't suffice),
> I'll stop wanting stems snapped to pixel boundaries.

Well, both autohinter and Adobe CFF engine snap to pixels, but on the y-axis only, just like ClearType on Windows ;) Oh, and 200dpi is very nice to look at judging from the 27"-Retina-iMacs.

But I get your point. The problem is, I don't have the resources to make an aggressively hinted .ttf. I was going to suggest that you switch to DejaVu Sans for the UI but I see you tried that already. If you ever feel adventurous, have a look at its' OS/2 table and change the heights to get more spacing.

If not, setting hintslight for Cantarell only via fontconfig is the only option I can think of now that would moderately please you. Or, well, it's complicated. GTK/cairo/GNOME-whatever has its' own opinion about what hinting is to be used, so you'd have to set hintslight in gnome-tweak-tool and hintfull for DejaVu in fontconfig. My brain hurts.

> Which part are you referring to?  All the fonts in the screenshot I posted
> in #20 are blurry.

I meant the fonts in the black bar. They look slightly hinted. But never mind, doesn't matter.

(In reply to madigens from comment #24)
> > While that's an improvement, it's still not something I could manage to look
> > at for any significant amount of time without getting a headache.  I
> > definitely don't want to make my fonts look that blurry.  I don't care as
> > much fonts looking "too bold"; I'll worry about that after I figure out how
> > to stop stems from smearing across pixels.
> 
> I'm sure you'd get used to it eventually. Like people who switch between
> Windows and Mac OS X and first find the respective font rendering weird but
> get used to it over time.

I've tried.  Blurry rendering literally (not figuratively) gives me a headache due to eyestrain.  So it's not something I could get used to. :)

> > Making my crisp fonts blurry so they match?  No thank you.  But if you can
> > suggest a way to make my blurry fonts crisp, I'd love to hear it. :)
> 
> That's the problem I tried to explain in comment 16. If you want a
> razor-sharp Cantarell, *someone* would need to hint it like the DejaVu
> family. Don't count on me, that would require weeks or months of full time
> work... Maybe you'll get lucky with ttfautohint?

Autohint does improve rendering.  With autohint enabled, Cantarell 0.0.17 has gone back to a degree of crispness I can live with, comparable to the pre-CFF rendering.

FWIW, I do appreciate the noticeable improvements in 0.0.17.

> > Switching to slight hinting does actually improve the rendering of Cantarell
> > somewhat (not perfectly, but somewhat), but makes DejaVu blurry.  Switching
> > to the autohinter similarly improves Cantarell rendering.  Note that I use
> > the Cantarell .otf files, as packaged in Debian.
> 
> The autohinter doesn't yet darken glyphs. Until I land my changes in
> FreeType. *evil laughter*. Ahem, you can switch off stem-darkening for .otfs
> with full hinting when compiling your own FreeType or waiting for fontconfig
> to throw up a switch for it.

Thanks to Fabian Greffrath, I've now tried a FreeType without stem darkening, and that does improve things, but not as much as switching to autohint.

> > As far as I can tell, this is the standard Mac versus Windows font-rendering
> > debate (designer-intended shapes versus crisp rendering), to which my answer
> > is that I'll happily sacrifice "intended" font shape for rendering that
> > recognizes that displays have pixels and don't have infinite resolution. 
> > When all the screens I use hit 300+ DPI (note that 200 doesn't suffice),
> > I'll stop wanting stems snapped to pixel boundaries.
> 
> Well, both autohinter and Adobe CFF engine snap to pixels, but on the y-axis
> only, just like ClearType on Windows ;) Oh, and 200dpi is very nice to look
> at judging from the 27"-Retina-iMacs.

I have a 200 DPI screen at home, and Cantarell with hintfull (with or without stem darkening) still looks blurry, though somewhat less so.

> But I get your point. The problem is, I don't have the resources to make an
> aggressively hinted .ttf. I was going to suggest that you switch to DejaVu
> Sans for the UI but I see you tried that already. If you ever feel
> adventurous, have a look at its' OS/2 table and change the heights to get
> more spacing.

Is there any way to adapt GTK to get enough line spacing (e.g. the equivalent of setting CSS line-height) in a font-independent way, rather than artificially padding the font's line height?

> If not, setting hintslight for Cantarell only via fontconfig is the only
> option I can think of now that would moderately please you. Or, well, it's
> complicated. GTK/cairo/GNOME-whatever has its' own opinion about what
> hinting is to be used, so you'd have to set hintslight in gnome-tweak-tool
> and hintfull for DejaVu in fontconfig. My brain hurts.

Enabling autohint for Cantarell works for now; I can do so via fontconfig configuration.  I plan to encourage the Debian fonts-cantarell package to do so by default, but in any case I can do so in my personal fontconfig configuration.

> > Which part are you referring to?  All the fonts in the screenshot I posted
> > in #20 are blurry.
> 
> I meant the fonts in the black bar. They look slightly hinted. But never
> mind, doesn't matter.

If I recall correctly, I think both gnome-shell and the browser rendered with hintmedium in that screenshot.

> Autohint does improve rendering.  With autohint enabled, Cantarell 0.0.17
> has gone back to a degree of crispness I can live with, comparable to the
> pre-CFF rendering.

Happy to hear you found a good compromise :)
 
> Thanks to Fabian Greffrath, I've now tried a FreeType without stem
> darkening, and that does improve things, but not as much as switching to
> autohint.

Hm? Both autohinter and hintfull for .otfs use the same method of snapping points to the y-axis and leaving the x-axis alone (-> blur). They go about it differently so output may vary.

> I have a 200 DPI screen at home, and Cantarell with hintfull (with or
> without stem darkening) still looks blurry, though somewhat less so.

At the same pt-size? Yeah, but it's similar to 5" smartphones with HD vs. FullHD screens. You can see the difference but it is much less jarring compared to low-DPI screens.

> Is there any way to adapt GTK to get enough line spacing (e.g. the
> equivalent of setting CSS line-height) in a font-independent way, rather
> than artificially padding the font's line height?

I unfortunately know nothing about GTK theming and stuff.

(In reply to madigens from comment #26)
> > Thanks to Fabian Greffrath, I've now tried a FreeType without stem
> > darkening, and that does improve things, but not as much as switching to
> > autohint.
> 
> Hm? Both autohinter and hintfull for .otfs use the same method of snapping
> points to the y-axis and leaving the x-axis alone (-> blur). They go about
> it differently so output may vary.

The output varies quite a bit; see the screenshots in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799345#83 .

> > I have a 200 DPI screen at home, and Cantarell with hintfull (with or
> > without stem darkening) still looks blurry, though somewhat less so.
> 
> At the same pt-size?

No, 1.5x the size.  11pt with 1.5x font size magnification set.

> Yeah, but it's similar to 5" smartphones with HD vs.
> FullHD screens. You can see the difference but it is much less jarring
> compared to low-DPI screens.

Granted, but it's still visibly blurry.

> The output varies quite a bit; see the screenshots in
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799345#83 .

Mmmh, if you disable stem-darkening which is a separate feature, not much actually. Quickly switch between screenshot-nostemdarken-autohint and screenshot-nostemdarken-hintfull. The slightly smaller capital letters with the autohinter are an algorithm quirk. BTW, did you get your fontconfig right for the screenshots? The respective hintslight/full pics are the same. Hintslight should force the autohinter.

> No, 1.5x the size.  11pt with 1.5x font size magnification set.

Well duh ;) Is it a 4k screen? Have you tried dpi x 2 scaling in gnome-tweak-tools?

(In reply to madigens from comment #28)
> > The output varies quite a bit; see the screenshots in
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799345#83 .
> 
> Mmmh, if you disable stem-darkening which is a separate feature, not much
> actually. Quickly switch between screenshot-nostemdarken-autohint and
> screenshot-nostemdarken-hintfull. The slightly smaller capital letters with
> the autohinter are an algorithm quirk.

Sizes aside, the difference in blurriness seems quite apparent to me. The autohinter still smears stems across pixels, but less so, with the adjacent grayish line being much more subtle.  Look at the vertical stems in the N and T, and compare between the two.  Ideally there shouldn't be an adjacent grayish line at all, but the autohint version seems like an improvement.

Antialiasing should make curves less jagged, but should not attempt to make straight lines a non-integer number of pixels wide.

> BTW, did you get your fontconfig
> right for the screenshots? The respective hintslight/full pics are the same.
> Hintslight should force the autohinter.

I assigned "hintslight" to "hintstyle" in a match block for Cantarell.  If I dropped the Cantarell match, I could easily see the effect on DejaVu.

> > No, 1.5x the size.  11pt with 1.5x font size magnification set.
> 
> Well duh ;) Is it a 4k screen? Have you tried dpi x 2 scaling in
> gnome-tweak-tools?

It's 2560x1440 at 14" (X1 Carbon), and 2x DPI scaling is far too large, hence the use of font scaling instead, which supports fractions.

> Sizes aside, the difference in blurriness seems quite apparent to me. The
> autohinter still smears stems across pixels, but less so, with the adjacent
> grayish line being much more subtle.  Look at the vertical stems in the N
> and T, and compare between the two.  Ideally there shouldn't be an adjacent
> grayish line at all, but the autohint version seems like an improvement.

Careful! The difference in size is important here, as the "slight smear" scales with size and can become "stronger" ;) By the way, have you tried the light LCD filter yet?

$ sudo rm /etc/fonts/conf.d/11-lcdfilter*
$ sudo ln -s /usr/share/fontconfig/conf.avail/11-lcdfilter-light.conf /etc/fonts/conf.d/

The default LCD filter smears a bit by design because of missing gamma correction in most rendering libs that can increase color fringing with the light filter.

> Antialiasing should make curves less jagged, but should not attempt to make
> straight lines a non-integer number of pixels wide.

Antialiasing is working fine, it's the way the autohinter, the CFF engine and modern ClearType on Windows works: only grid-fit outlines to the y-axis and use subpixel rendering ("rgba") to increase sharpness/contrast (== paper over blurriness) on the x-axis. I say, this method produces a good compromise between outline fidelity and readability. The stem darkening increases readability further, but can only shine with gamma correction. :)

> I assigned "hintslight" to "hintstyle" in a match block for Cantarell.  If I
> dropped the Cantarell match, I could easily see the effect on DejaVu.

Hum. Could you paste the exact blocks you used for each no-stem-darkening pic? I'm curious.

> It's 2560x1440 at 14" (X1 Carbon), and 2x DPI scaling is far too large,
> hence the use of font scaling instead, which supports fractions.

Ah, okay :) Still a nice screen ;)

Enabling autohinter and full hinting does return the font to its crisp form as it was pre-CFF. Caveats: Some glyphs are hinted incorrectly and shifts pieces of them to cause a "T" for example to become asymmetrical. On high-dpi screens character spacing becomes incorrect and something like "Home" becomes "Ho me". There doesn't seem to be an easy solution.

Oh high-dpi screens (take my 13" 2560x1440 ASUS UX301LAA) the font looks fine, regardless of CFF or not. Fully hinted or medium look as they do on my standard dpi screens. I'm happy on my laptop. Unfortunately I don't have high-dpi screens on all of my machines I use, yet.