GNOME Bugzilla – Bug 737925
Some filters / operations provide float values superior to 1.0
Last modified: 2018-05-24 14:45:45 UTC
We can observe this bug using some filters / operations on a layer mask : 1. create an image (i use template 640x480, 8-bit gamma integer precision) 2. fill the background layer with pure red 3. create a new white layer above the background layer 4. add some content to this layer (i use Filters > Render > Noise > Solid Noise... with default parameters) and create a layer mask on it using grayscale copy of layer 5. with the layer mask selected, use Colors > Brightness-Contrast... and increase the Contrast slider Cyan color appears on the canvas preview ; clicking "OK" to apply the operation give the expected result : cyan color disappears, certainly due to the float to integer conversion. But if you repeat the exact same procedure with a floating point precision image, applying the operation changes nothing : cyan color stays visible (no float to integer conversion, layer mask keeps values superior to 1.0) I was able to reproduce this behavior with : - Colors > Levels - Colors > Exposure - Filters > Enhance > Unsharp Mask
Is this bug about: 1. the fact that some filters produce > 1.0 values 2. the fact that the preview is different from the result for 8-bit images
Sorry for this unclear explanation... This bug is about the fact that some filters produce > 1.0 values and even < 0.0 values (some printf in the operations code confirm that). Applying these filters with the procedure descrived above give unexpected results.
Created attachment 289991 [details] [review] Ensure britghtness-contrast operation returns values in range 0.0, 1.0
Created attachment 289992 [details] [review] Ensure levels operation returns values in range 0.0, 1.0
See for instance bug 728603 which asks for the opposite in other places. We need to look into this, but the solution might be more complicated than adding clamping inside the ops. Values outside (0.0...1.0) can be correct results, but probably not when we work destructively on 8-bit images. In any case the preview should match the result.
Ok, I understand that operations producing values outside (0.0, 1.0) range is just a correct behavior. Putting aside "the fact that the preview is different from the result for 8-bit images", what about applying this kind of operations on a layer Mask ? It appears that having layer mask values outside (0.0, 1.0) range provides UNEXPECTED result (this can be easily reproduced with the procedure I describe above with a float point precision image) : current layers'blending modes implementation do not only affect the transparency for the related layers when associated masks contains values outside the (0.0, 1.0). Maybe I'm wrong, but as a Gimp user, when a layer mask is pure white or pure black as I can see it in the layers' stack, I think, ok, my layer is totally opaque (or totally transparent) and I do not expect to see magical colors appearing in the final composition.
Painting on a mask, and finding that the mask was carrying values outside the range 0-1, really is disconcerting, but I'm not sure it's always a bad thing (still experimenting). It's also disconcerting to find that when soft proofing to an output profile, the soft proofed image isn't clipped to the color gamut of the intended output profile. This probably really is always a bad thing. However, the ability to keep and work with out of gamut RGB values is useful for some operations (and critical for GIMP as a future HDR image editor). For example, when working with an image with a bright sky and a dark ground, use Levels to add four stops so the ground isn't dark, then paint on a mask to bring the sky back into gamut. In image editors that clip, you've lost the highlights and you have to use a very different approach. In GIMP, it's like magic, the sky information is still there and can be painted back in. For another example, when converting a saturated image to a smaller color gamut at floating point, some users aren't going to expect or know what to do with the resulting out of gamut RGB values. But some users will quickly figure out what's happening and be able to deal creatively with the resulting RGB values (which for example provide the opportuniaty to use Levels followed by Curves to bring all values into gamut without the clipping that happens from a normal "clipped" ICC profile conversion). On the other hand, operations that involve multiply or divide will often produce completely wrong results when done on out of gamut channel values. For example, consider multiplying a layer by itself: people expect that multiplying a layer by itself will produce a darker image, not a brighter image, but multiplying values that are out of gamut produces brighter values. So making good use of the out of gamut values does require a certain amount of knowledge on the part of the user. I expect many GIMP users will not like dealing with out of gamut values. And many will discover that once they get used to the idea, being able to carry along out of gamut values is a useful tool. A "clip all" switch in Preferences would be very helpful for people who do want to work at floating point but don't want to deal with out of gamut channel values. Also it would be nice to have the user option to clip a particular layer, depending on what the user wants to do next. My workaround is to do a null Curves operation on the layer that I want clipped, as Curves does clip (though with parametric curves, a future possibility? that might change).
Created attachment 309890 [details] Example UI for allowing the user to clip out of gamut RGB values The attached screenshot shows a UI for allowing the user to clip out of gamut RGB values when performing a Levels operation. The screenshot was sent to me by a VFX person who's been testing a modified version of GIMP 2.9 (modified to eliminate the babl flips). The screenshot is posted here with permission, to provide an example of a per-operation option to clip out of gamut RGB values. I don't know what software it's from.
(In reply to Elle Stone from comment #8) > Created attachment 309890 [details] > Example UI for allowing the user to clip out of gamut RGB values > > The attached screenshot shows a UI for allowing the user to clip out of > gamut RGB values when performing a Levels operation. The VFX person sent me a note that said that the dropdown options are On, Off, Off for 32 bpc Color, and then added: > Nice thing about using a tool that clips values from 0 to 1 (like > Curves) is that we clip negative values, the problem is that we clip > also positive values above 1 . . . In the 3D package we have a Limiter > node that allows us to clip negative or positive values to any arbitrary > value. A tool like that is better, I think. Based on my own testing/using GIMP 2.9, sometimes interpolated camera raw files imported from darktable (which allows to export 32f images) into GIMP have noise in the shadows that results in pixels with negative Luminance values. After dealing with these pixels, some GIMP operations can still cause additional shadow pixels with negative Luminance values. I usually don't want to clip any >1.0 pixel channel values until I'm ready to output the image. These values represent real data that can be pulled below 1.0 at a later point. But depending on the source of the negative channel values, usually I do want to clip the negative channel values that crop up, but only after initial processing to deal with noise and bring colors into gamut. It might be nice to have an operation in GIMP that provides the user with two options: 1. "Clip all RGB values in a layer or mask to the range X to Y" 2. "Clip RGB values below X" with X defaulting to 0.0 and Y defaulting to 1.0, but X and Y being user-settable to other values.
(In reply to Michael Natterer from comment #5) > In any case the preview should match the result. With integer image precision, preview does not always match the result. For example : 1. create an image with integer precision 2. fill the layer with mid grey #808080 3. duplicate the layer and set the top layer blend mode to multiply 4. activate the levels tool on the top layer and set the high input level to 50 At this step, we can see a bad canvas preview (white color) ; multiply operation is computed with HDR values produced by the levels operation. As soon as you commit the levels operation by clicking Ok, we obtain a different but correct canvas result (mid grey) ; multiply operation is computed with clipped values since levels'HDR values are removed when stored to the integer layer buffer.
That really is disconcerting. To see the effect, you have to apply the levels operation twice, or just make a white layer and then apply the levels operation, and watch the gray progressively approach white as the slider is moved to the midpoint, only to turn gray again when "OK" is clicked. At integer precision can the "uncommitted" results be clipped before they are sent to the screen?
(In reply to Elle Stone from comment #11) > That really is disconcerting. To see the effect, you have to apply the > levels operation twice, or just make a white layer and then apply the levels > operation, and watch the gray progressively approach white as the slider is > moved to the midpoint, only to turn gray again when "OK" is clicked. > > At integer precision can the "uncommitted" results be clipped before they > are sent to the screen? If I'm not mistaken enabling the gamma hack already produces a preview equal to the output, so it should be possible to use a "gegl:convert-format" as the cast_after operation here: https://git.gnome.org/browse/gimp/tree/app/core/gimpimagemap.c#n545 instead of a nop, in the place where the gamma hack code is using a "gegl:cast-format" https://git.gnome.org/browse/gimp/tree/app/core/gimpimagemap.c#n533
(In reply to Massimo from comment #12) > If I'm not mistaken enabling the gamma hack already produces > a preview equal to the output, so it should be possible to > use a "gegl:convert-format" as the cast_after operation [...] Indeed, but in a performance point of view, converting the output format will obliged the next operation in the gegl graph to proceed to an additional integer to float conversion for its input buffer. IMO, a "clip" operation would provide the expected result, without involving additional format conversions.
(In reply to Thomas Manni from comment #13) > (In reply to Massimo from comment #12) > > If I'm not mistaken enabling the gamma hack already produces > > a preview equal to the output, so it should be possible to > > use a "gegl:convert-format" as the cast_after operation [...] > > Indeed, but in a performance point of view, converting the output format > will obliged the next operation in the gegl graph to proceed to an > additional integer to float conversion for its input buffer. > > IMO, a "clip" operation would provide the expected result, without involving > additional format conversions. Do you mean a new operation that outputs float in [0 1]? I thought about that, but then someone might notice the difference between rounding to the nearest u{8,16,32} that is done when storing the result of the filter to the drawable and the preview that would not round. Performance-wise if a conversion model float -> model u{8,16,32} -> model float is missing then it should be added.
For operations like Levels, that do support unclipped results at floating point precision, but put floating point output to the screen before the operation is actually committed by clicking "OK", what about using an "IF" line that checks to see if the image is at integer precision, and if so, use the CLAMP function to confine the result to the range 0f to 1f? This would allow the option to expand the "IF" line of code to provide clamping even when at floating point, should GIMP eventually provide users with clamping options, which seems to be a very useful thing to eventually make possible, if UI consirations could be worked out. Currently to clip floating point results to the range 0-1f, one approach is to duplicate the layer and use a NULL curves operation. And if you only want to clip the negative channel values, set the duplicated clipped layer's blend mode to Lighten only.
(In reply to Elle Stone from comment #15) > For operations like Levels, [...] Levels is only an example ; any gegl operation can potentially produce result outside the [0;1] range. As mitch said on comment #5 > the solution might be more complicated than adding clamping inside the ops Massimo's suggestion is one solution, at least for the integer precision image issue. A "rgb-clip" operation is another one. Anyway, imo, rgb-clip operation will have its utility, sooner or later, so I will push a first version in gegl workshop directory.
That rgb-clip operation works perfectly. Thanks!
(In reply to Elle Stone from comment #7) > Painting on a mask, and finding that the mask was carrying values outside > the range 0-1, really is disconcerting, but I'm not sure it's always a bad > thing (still experimenting). See this tutorial for an example of the usefulness of *not* summarily clipping out of gamut Y values on grayscale masks: Tone mapping and shadow recovery using GIMP Levels http://ninedegreesbelow.com/photography/gimp-tone-map-with-levels.html The tutorial mentions using the rgb-clip operation for various tasks. It would be nice to have the rgb-clip operation in the main part of the GIMP UI somewhere very handy rather than buried in the workshop.
*** Bug 785977 has been marked as a duplicate of this bug. ***
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gimp/issues/595.