Created attachment 203884 [details]
Example ›area‹ screenshot with alpha channel, smooth borders and no desktop background.

The effect on quality is much easier to see clicking the image under this link:

http://screenshots.debian.net/package/octave3.2

This broke when we made mutter have proper anti-aliased corners, since we no longer set an XShape on the window that gnome-screenshot would carve out again. Unfortunately, since X doesn't have an 8-bit mask for windows, we can't make gnome-screenshot a separate tool without knowledge of the compositor's internals.

This will be fixed when we port gnome-screenshot to use the DBus API that gnome-shell provides, as it can access the actual window texture in memory, before compositing.

Maybe this should be reassigned to gnome-shell then, since gnme-screenshot is supposed to work for old-fashioned fallback mode as well.

Created attachment 203944 [details]
Screenshot taken using gnome-shell's internal window_screenshot method

Actually, this does not seem to only be related to gnome-screenshot. My gnome-shell extensions uses the internal window_screenshot() method of gnome-shell,

And while it catches the transparency of e.g. the terminal window itself, it completly screws up around the window border.

Created attachment 203995 [details] [review]
shell-global: Correct screenshot_window() after mutter changes

This requires the latest mutter patches in bug #660941.

Sounds good. I failed juggling with all those patches and will wait with testing until all this comes naturally down the ricotz PPA pipe.

I’m curious, would it be feasible to do this with multiple windows at the same time while preserving their spatial relations (i.e., enhance screenshot_area() with an include_background flag that doesn’t just use cogl_read_pixels)?

Created attachment 205783 [details]
Comparison of terminal before and after (with new patches)

I finally got to try this out. That was painful to get to work.

The borders look fine now with screenshot_window(), but for some reason the subpixels of the fonts are messed up in the (perhaps some gamma issue in the alpha channel?). Also the shadows are not included, which would be nice to have.

Also interacting with GTK2 apps makes gnome-shell segfault at random points frequently.

(In reply to comment #8)
> Created an attachment (id=205783) [details]
> Comparison of terminal before and after (with new patches)

There are three images here. Can you list the three scenarios?

> I finally got to try this out. That was painful to get to work.
> 
> The borders look fine now with screenshot_window(), but for some reason the
> subpixels of the fonts are messed up in the (perhaps some gamma issue in the
> alpha channel?). Also the shadows are not included, which would be nice to
> have.

Does it happen with an opaque gnome-terminal?

> Also interacting with GTK2 apps makes gnome-shell segfault at random points
> frequently.

That shouldn't be happening. File a new bug?

Created attachment 205785 [details]
Manipulated image: Fake transparency with background tricks and gimp

> There are three images here. Can you list the three scenarios?

1. just a rectangle area screenshot to see how the text looks normally
2. before these patches
3. using these patches:

00_MetaShapedTexture:_Junk_old_"notify"_hack.patch
03_MetaShapedTexture:_Make_public_and_into_a_ClutterActor.patch
04_MetaShapedTexture:_Add_a_new_method_to_flatten_the_shaped.patch
and
shell-global:_Correct_screenshot_window()_after_mutter_changes.patch

> Does it happen with an opaque gnome-terminal?

No, the other 100% opaque windows are fine.

I attached a model, where I artificially created the translucency in gimp by making two conventional screenshots against white and against black desktops, then subtracting the two and copying the result into the alpha channel.

This is evindence that a single alpha channel is suffient. I’ve read somewhere that for subpixel filtering different gammas are needed, but I’m not sure anymore what that was about exactly.

See https://bugs.freedesktop.org/show_bug.cgi?id=3307#c25
I think this is the place where I picked this up. But he is talking about cairo and rendering to a surface. So it may or may not be relevant at all.

I tested this with cairo-clock, one of the few programs with free-form windows, and those screenshots do look quite nice. 

See here: http://www.alice-dsl.net/~towolf/clock-screenshots/

The only apparent problem is the issue with subpixel text from above. I would screenshot gnome-shell itself because it has similar effects (overlay, menus, clutter dialogs), but I can’t because of the Xinput grab issue mentioned elsewhere.

As highlighted by Jasper, this needs fixing in mutter/gnome-shell.

Created attachment 206714 [details] [review]
shell-global: Correct screenshot_window() after mutter changes

Recent mutter changes made MetaShapedTexture not a ClutterTexture,
but instead a special clutter actor that implemented the texture-y bits
itself. Use recently introduced API in MetaShapedTexture so that we can
get the raw texture data and spit it out as a PNG.

Use the new meta_shaped_texture_as_flattened_data() to get a window's
texture data, including any output shape or X SHAPE region it has,
allowing for a proper alpha channel on the screenshot.



Rebased and updated to both changes on the mutter side and the shell side.

-> gnome-shell

Reassigning component

Created attachment 206734 [details] [review]
shell-global: Correct screenshot_window() after mutter changes

Recent mutter changes made MetaShapedTexture not a ClutterTexture,
but instead a special clutter actor that implemented the texture-y bits
itself. Use recently introduced API in MetaShapedTexture so that we can
get the raw texture data and spit it out as a PNG.

Use the new meta_shaped_texture_get_image() to get a window's texture
data, including any output shape or X SHAPE region it has, allowing
for a proper alpha channel on the screenshot.



Use the new name for the method: meta_shaped_texture_get_image()

Review of attachment 206734 [details] [review]:

> Use the new meta_shaped_texture_get_image() to get a window's texture
> data, including any output shape or X SHAPE region it has.

It does not return any shape (just the texture data after doing its "magic").

Otherwise looks good, just one style issue.

::: src/shell-global.c
@@ +2129,3 @@
   ClutterActor *window_actor;
+  MetaShapedTexture *stex; 
+ gfloat actor_x, actor_y;

Weird indentation.

Created attachment 206740 [details] [review]
shell-global: Correct screenshot_window() after mutter changes

Recent mutter changes made MetaShapedTexture not a ClutterTexture,
but instead a special ClutterActor subclass that implemented the texture-y
bits itself. Use recently introduced API in MetaShapedTexture so that we can
get the raw texture data and spit it out as a PNG.

Use the new meta_shaped_texture_get_image() to get a window's texture data.
meta_shaped_texture_get_image() flattens the image against any mask it may
have, so a screenshot of it should look exactly as it does on the display.



Whoops, didn't realize I was screwing up the indentation there.

Is this commit message any better?

Review of attachment 206740 [details] [review]:

Yeah that's better.

Attachment 206740 [details] pushed as d254e2e - shell-global: Correct screenshot_window() after mutter changes

Created attachment 206763 [details]
Overview of start situation

Really nice work, Jasper.

Can I add a follow-up comment. Will open new bugs if necessary.

this image serves as a baseline

Created attachment 206764 [details]
screenshot_area() misses

Problem 1:

  The area screenshot now shoots with an offset from the given rectangle, i.e., it misses the rect selection given with <x, y, width, height> and shifts to the lower left. Could this be related to cosimoc’s commit 9147dee0 "screenshot: use the correct offsets when calculating the window area". 

This screenshot came out when it was given the rectangle seen in the previous image. This worked in December.

Created attachment 206765 [details]
screenshot_window() alpha creates artifacts on RGBA antialiased text

Problem 2:

  What we discussed above. The RGB decimated text is artifacted. I’m not sure how to handle this.

Finally, could you comment on the shadows, i.e., is this out-of-scope, or perhaps technically infeasible, or not just now, etc? With real shadows we would match the quality of other operating systems (cf. Wikipedia).

(In reply to comment #22)
> Created an attachment (id=206764) [details]
> screenshot_area() misses
> 
> Problem 1:
> 
>   The area screenshot now shoots with an offset from the given rectangle, i.e.,
> it misses the rect selection given with <x, y, width, height> and shifts to the
> lower left. Could this be related to cosimoc’s commit 9147dee0 "screenshot: use
> the correct offsets when calculating the window area". 
> 
> This screenshot came out when it was given the rectangle seen in the previous
> image. This worked in December.

File a new bug for this.

(In reply to comment #23)
> Created an attachment (id=206765) [details]
> screenshot_window() alpha creates artifacts on RGBA antialiased text
> 
> Problem 2:
> 
>   What we discussed above. The RGB decimated text is artifacted. I’m not sure
> how to handle this.

This should be fixed as I'm now using cairo.

(In reply to comment #24)
> Finally, could you comment on the shadows, i.e., is this out-of-scope, or
> perhaps technically infeasible, or not just now, etc? With real shadows we
> would match the quality of other operating systems (cf. Wikipedia).

gnome-screenshot should add shadows to the output image manually. It's not that hard to do.

(In reply to comment #25)

> gnome-screenshot should add shadows to the output image manually. It's not that
> hard to do.

Well there is no point in doing so really. (Adding shadows to screenshots).

(In reply to comment #26)
> (In reply to comment #25)
> 
> > gnome-screenshot should add shadows to the output image manually. It's not that
> > hard to do.
> 
> Well there is no point in doing so really. (Adding shadows to screenshots).

But wouldn’t you agree that [0] looks better than [1] if you want to demonstrate a program? If decorations can be toggled on-and-off why not shadows?

[0] https://en.wikipedia.org/wiki/File:Audacity-macosx.png
[1] https://en.wikipedia.org/wiki/File:Audacity_Ubuntu.png
[2] https://en.wikipedia.org/wiki/File:Windows_Internet_Explorer_9.png

(In reply to comment #25)
> (In reply to comment #22)
> > Created an attachment (id=206764) [details] [details]
> > screenshot_area() misses
> > 
> > Problem 1:
> > 
> >   The area screenshot now shoots with an offset from the given rectangle, i.e.,
> > it misses the rect selection given with <x, y, width, height> and shifts to the
> > lower left. Could this be related to cosimoc’s commit 9147dee0 "screenshot: use
> > the correct offsets when calculating the window area". 
> > 
> > This screenshot came out when it was given the rectangle seen in the previous
> > image. This worked in December.
> 
> File a new bug for this.

Filed as Bug 669366

> > Problem 2:
> > 
> >   What we discussed above. The RGB decimated text is artifacted. I’m not sure
> > how to handle this.
> 
> This should be fixed as I'm now using cairo.

It’s still there. I’m using the git head from last night.

(In reply to comment #26)
> (In reply to comment #25)
> 
> > gnome-screenshot should add shadows to the output image manually. It's not that
> > hard to do.
> 
> Well there is no point in doing so really. (Adding shadows to screenshots).

Well it's not about adding shadows to screenshots in general, but keeping the existing shadow-decoration of a window when making a window-screenshot.

(In reply to comment #29)
> > > Problem 2:
> > > 
> > >   What we discussed above. The RGB decimated text is artifacted. I’m not sure
> > > how to handle this.
> > 
> > This should be fixed as I'm now using cairo.
> 
> It’s still there. I’m using the git head from last night.

I can't reproduce. Can you fiddle with both the font and hinting options to see if that affects it? The screenshot also appears to be opaque, not transparent, so I don't think you have the patch installed correctly.

Well, if you use a distro like Fedora or Suse, you cannot repro because they disable LCD filtering (subpixel rendering) in libfreetype. My font rendering is the Ubuntu default (rgba, hintslight, lcddefault).

If you load my image from comment #23 in eog and set the background to checkered you will see that it is semi-translucent.

Also, Ben’s example from before your changes exhibits the same.

Enlarged, it looks like this:

http://www.alice-dsl.net/~towolf/clock-screenshots#lcd-fonts

Is the problem on(In reply to comment #33)
> Enlarged, it looks like this:
> 
> http://www.alice-dsl.net/~towolf/clock-screenshots#lcd-fonts

Did the alpha channel get lost in the screenshot? does the problem only occur for alpha-transparent terminals?

No, the alpha channel is there. If you look at my silly little webpage, "fig : screenshot_window() after" has an alpha channel (when you hover it switches from checkerboard to table cloth behind it)

The only problem is that the RGB decimated LCD filtered fonts have artifacts.

I wish I were more knowledgable in these matters. It could be a gamma issue, it could also be that a single alpha channel is not enough to composite these onto a generic background. At least i’ve read something to that effect (s. above, Comment #11).

Created attachment 206798 [details]
Gnome Shell itself is translucent

(In reply to comment #34)
>                                            does the problem only occur
> for alpha-transparent terminals?

Well, g-t and gnome-shell itself. But I cannot screenshot_window the parts of g-s that are translucent, because the keyboard shortuts are grabbed by them.

Attachment shows effect. Once Xinput GrabOverride is implemented this will crop up for the Overview. Rigth now only gnome-terminal.

(In reply to comment #36)
> Created an attachment (id=206798) [details]
> Gnome Shell itself is translucent
> 
> (In reply to comment #34)
> >                                            does the problem only occur
> > for alpha-transparent terminals?
> 
> Well, g-t and gnome-shell itself. But I cannot screenshot_window the parts of
> g-s that are translucent, because the keyboard shortuts are grabbed by them.
> 
> Attachment shows effect. Once Xinput GrabOverride is implemented this will crop
> up for the Overview. Rigth now only gnome-terminal.

The fact that ARGB text doesn't work quite correctly on ARGB windows is a limitation of the fact that ARGB windows only have 4 channels not 6 channels. But the breakage should only be slight for windows that are only slightly transparent.

Your screenshot looks like a ARGB window that lost it's transparency - like maybe the alpha channel was just filled with 0xff. That will give further color distortion. Is that right? Did that happen when gnome-shell took the screenshot?

- Owen

Created attachment 206799 [details]
8-bit alpha channel of the artifacted image

(In reply to comment #37)
> The fact that ARGB text doesn't work quite correctly on ARGB windows is a
> limitation of the fact that ARGB windows only have 4 channels not 6 channels.
> But the breakage should only be slight for windows that are only slightly
> transparent.
> 
> Your screenshot looks like a ARGB window that lost it's transparency - like
> maybe the alpha channel was just filled with 0xff. That will give further color
> distortion. Is that right? Did that happen when gnome-shell took the
> screenshot?

The attachment [0] comes straight from the screenshot_window() function of gnome-shell after the commits by Jasper. When you look at it you can see that the text’s pixels have alpha avalues out of [0 .. 1]. It’s not a binary mask.  

I’ll attach the extracted alpha channel to this comment.

[0] https://bugzilla.gnome.org/attachment.cgi?id=206765

(In reply to comment #32)
> Well, if you use a distro like Fedora or Suse, you cannot repro because they
> disable LCD filtering (subpixel rendering) in libfreetype. My font rendering is
> the Ubuntu default (rgba, hintslight, lcddefault).

Not if you install freetype-freeworld from rpmfusion (for Fedora, at least), which is what I have done. Which key has the "lcddefault" value? Grepping through g-s-d's settings plugin I couldn't find anything like that.

I tried setting my hinting to default, and I still couldn't reproduce.

Created attachment 206819 [details]
Correct screenshot of a partially transparent terminal

Jasper, foo.png has gray antialias.


(In reply to comment #39)
> (In reply to comment #32)
> > Well, if you use a distro like Fedora or Suse, you cannot repro because they
> > disable LCD filtering (subpixel rendering) in libfreetype. My font rendering is
> > the Ubuntu default (rgba, hintslight, lcddefault).
> 
> Not if you install freetype-freeworld from rpmfusion (for Fedora, at least),
> which is what I have done. Which key has the "lcddefault" value? Grepping
> through g-s-d's settings plugin I couldn't find anything like that.
> 
> I tried setting my hinting to default, and I still couldn't reproduce.

I have this in g-s-d:

org.gnome.settings-daemon.plugins.xsettings antialiasing 'rgba'
org.gnome.settings-daemon.plugins.xsettings hinting 'slight'
org.gnome.settings-daemon.plugins.xsettings rgba-order 'rgb'


And if that doesn’t work because it’s somehow overridden could you try this ~/.fonts.conf snippet?


<?xml version='1.0'?>
<!DOCTYPE fontconfig SYSTEM 'fonts.dtd'>
<fontconfig>
  <match target="font" >
    <edit mode="assign" name="rgba" >
      <const>rgb</const>
    </edit>
    <edit mode="assign" name="hinting" >
      <bool>true</bool>
    </edit>
    <edit mode="assign" name="hintstyle" >
      <const>hintslight</const>
    </edit>
    <edit mode="assign" name="antialias" >
      <bool>true</bool>
    </edit>
    <edit mode="assign" name="lcdfilter" >
      <const>lcddefault</const>
    </edit>
    <edit mode="assign" name="autohint" >
      <bool>true</bool>
    </edit>
    <edit mode="assign" name="embeddedbitmap" >
      <bool>false</bool>
    </edit>
  </match>
</fontconfig>

I’ve spent some time trying to figure out what kind of operation might lead to the artifacts. Not understanding the code, I did some experiments.

• The alpha channel in the result is fine and correct. 
• The artifacting occurs in the R and B channels themselves

I verified this by transplanting the alpha channel to an opaque screenshot. It’s also equal to the empirically determined alpha channel. (if you’re bored watch the show-and-tell [0]). 

I couldn’t find out what kind of pixel color twiddling on the RGB takes place, however. Which code would I have to look at? Any relation to pre-multiplied alpha and non-premultiplied?

[0] http://www.alice-dsl.net/~towolf/alpha-transplant.mp4 
    (use totem, it’s 4:4:4)

GNOME is going to shut down bugzilla.gnome.org in favor of  gitlab.gnome.org.
As part of that, we are mass-closing older open tickets in bugzilla.gnome.org
which have not seen updates for a longer time (resources are unfortunately
quite limited so not every ticket can get handled).

If you can still reproduce the situation described in this ticket in a recent
and supported software version, then please follow
  https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines
and create a new ticket at
  https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/

Thank you for your understanding and your help.