GNOME Bugzilla – Bug 152593
better antialiasing for shallow angles in paths
Last modified: 2004-12-22 21:47:04 UTC
summary: Angles approaching 0/90 are poorly antialiased. steps to reproduce: 1. make a new document (filled with white) 2. select a fairly large triangle using the path tool. include one shallow angle. 3. fill the selection with black. on the shallow edge the antialiasing degrades a lot. temporary, fiddly workaround: 1. enter qmask mode after making the selection 2. select the rectangle containing the shallow angle. 3. perform a gaussian blur. the axis that the angle traverses quickest should be blurred proportional to the speed of it's traversal. the other should equal 1. for my testcase i found a radius of 2 was sufficient. 4. exit qmask mode. the amount of antialiasing needs to be exaggerated by about *0.5 in both directions -- ie the antialiasing should extend more outward and more inward, producing a smoothed effect.( outward == away from the line segment, inward == into it.). the current way coverage is estimated only works 'correctly' if the ramp from color A -> color B actually is linear (that is, for example (A+B)/2 resulting in a color that is perceptually half-way between A and B). this is rarely the case, and this strategy makes the error less obvious by diffusing it. now to figure out how to implement it :)
We should probably reassign this report to libart but I will leave it up to Simon to decide what to do. On the other hand, the comments at the bottom make me believe that the problem is solely a wrong gamma setting on David's display.
I have trouble understanding this report. David, could you please attach a screenshot/sample image of the bad antialiasing? I too suspect that it is a problem with Gamma. IIRC libart assumes a gamma value of 1.0 and the results might look not optimal on different settings. I have no idea if it is possible to change that in libart.
The way it works now looks pretty good to me. It would be helpful if you (David) could attach a small image, showing an example of (a) how GIMP does it normally, and (b) how it looks after your correction, so that we can get a more concrete sense of what you are proposing.
Created attachment 31568 [details] example image gimp-version on the left; corrected version on the right. this could be considered a gamma issue; however gamma-correction a) is insufficient; it improves the antialiasing somewhat but still leaves obvious steps. b) will make the image look worse on machines with different gamma settings ( a linear intensity ramp has best 'portability' in this sense, not a curved one) the right triangle was created by gaussian-blurring the part of the selection mask containing the shallow angle with horizontal blur = 1, vertical blur = 0.. multiple times.
Maybe its just me, but the lower edge of the triangle looks blurry to me. If you look at the image in magnification you'll see that there are two rows of grey pixels below the black pixels, this is more than I'd expect from antialiasing. You can improve the visual appearance of the lower edge by using the levels tool with a gamma value of 2.2. However, since we now don't assume any color space in the image I don't know if this is the right thing to do (And I have no idea if this is easily possible in libart). I suggest to mark this bug as WONTFIX and look at the issue again when we have GEGL.
IMHO it makes sense to take a closer look at the libart API. There seems to be a way to specify a gamma ramp for the alpha values for art_rgb_svp_aa() but at the moment I fail to see how such a correction could be achieved with art_gray_svp_aa(), which is what we are using.
Changing status because info has been provided, leaving as UNCONFIRMED because the comments are ambiguous.
Simon had a look at this and the output can indeed be improved by applying a gamma correction. This correction must however be applied to the image display, not to the selection simply because we don't know if the selection will turn out dark on a light background or vice versa. I suggest to close as NOTABUG and focus on proper display gamma correction instead.
Problem seems solved in 2.2 - I just tried it in 2.2 pre-1 and shallow angles look just fine.
We didn't change anything though. But I think we have discussed this to the point where we can agree that there isn't anything we can do to improve the output. So it doesn't make sense to keep this report open any longer.