Yes, I can reproduce as well *sometimes* - The back area is actually the shadows mutter adds to server-side decorations with the X11 backend.

 - It seems fairly random - Moving the window might make the issue go away.
 - It seems to affect some apps more than others (FF, TB whereas I cannot reproduce with xterm for example)
 - It doesn't seem to be HW/driver related

(In reply to Olivier Fourdan from comment #1)
>  - It seems to affect some apps more than others (FF, TB whereas I cannot

Actually it seems to affect gtk2 and Qt based apps and I think this could be related to the use of the _NET_WM_SYNC protocol v1 by these toolkits.

As a test, if I disable support for _NET_WM_SYNC in mutter, I cannot reproduce the problem anymore (but obviously resizes are a lot less smooth)

(In reply to Olivier Fourdan from comment #2)
> Actually it seems to affect gtk2 and Qt based apps and I think this could be
> related to the use of the _NET_WM_SYNC protocol v1 by these toolkits.

Well, not just v1 actually, forcing X11 backend on gtk3-demo and resizing service-side decorated windows can exhibit the problem as well.

And I cannot reproduce with _NET_WM_SYNC disabled ("window->disable_sync = TRUE;" in meta_window_x11_update_sync_request_counter() from window-x11.c)

*** Bug 770367 has been marked as a duplicate of this bug. ***

So on Xorg, forcing sync eliminates the black box flickering but causes it on Xwayland?
Could this be a Xayland bug as suggested in bug 77036?

It's two different things imho. _NET_WM_SYNC is not to avoid black box flickering of the drop shadows on X11 (as these are outside the client toplevel window), it's to synchronize frames redraw and avoid resizing faster than the client can cope.

For reference, see Owen's description of the the extended synchronisation version of _NET_WM_SYNC used by gtk+ and mutter.

It could be a Xwayland bug, possibly, but so far I haven't really found hard evidence of that.

Besides, on X11/Xwayland with server side decorations, mutter draws the drop shadows underneath the windows.

Quick update - I have been looking into this lately to see if we could use Pekka's proposal of adding an _XWAYLAND_ALLOW_COMMITS property to tell Xwayland when to commit the surface:

  https://lists.x.org/archives/xorg-devel/2016-November/051913.html
  https://lists.x.org/archives/xorg-devel/2016-December/051947.html

and

  https://patchwork.freedesktop.org/series/16610/

Before so, I tried to understand what would be causing the back border issue in the first place and traced it down to meta_window_actor_update_opaque_region().

And it's not called from the same code path when using an application that uses _NET_WM_SYNC protocol (e.g. gtk-demo from gtk2) and one who doesn't (e.g. plain old xterm).

With XSync protocol:

  meta_window_x11_update_sync_request_counter()
    meta_compositor_sync_updates_frozen()
      meta_window_actor_sync_updates_frozen()
        meta_window_actor_set_updates_frozen()
          meta_window_actor_thaw()
              meta_window_actor_handle_updates()
              check_needs_reshape()
                meta_window_actor_update_opaque_region();

Without XSync protocol:

  clutter_clock_dispatch()
    master_clock_update_stages()
      _clutter_run_repaint_functions()
        meta_window_actor_pre_paint()
          meta_window_actor_handle_updates()
            check_needs_reshape()
              meta_window_actor_update_opaque_region();

Created attachment 341942 [details] [review]
[PATCH] MetaWindowActor: Update shape in pre-paint

When dealing with clients who support the _NET_WM_SYNC protocol, the
shape update is done from meta_compositor_sync_updates_frozen(), but tha
can leave (on Xwayland primarily) visual traces of the previous hsape as
being cleared by a resize, causing ugly black border flickers.

Avoid the issue by updating the shape in pre_paint() as with other
clients which do not support _NET_WM_SYNC.
--
Note: that (simple) patch seems to avoid the issue, without needing additional synchronization mechanism.

Created attachment 341943 [details] [review]
[PATCH] MetaWindowActor: Remove uneeded test

The is_frozen() function tests for freeze_count >0, so no need to test
it twice.
--
Note: Trivial simplification, while at it...

Review of attachment 341942 [details] [review]:

It breaks synchronization on X11 so it's not suitable in the general case

Also experience this on Fedora 25 (Gnome 3.22 on Wayland).
i5-2410M with Sandybridge integrated graphics.
Is there any way to help?

I'm also experiencing this issue with applications such as Firefox and Qt Creator, but not with i.e Blender and Chromium and most pre-insalled applications. I'm using Fedora 25 with Gnome 3.22.3 on Wayland, running on an Intel i5-2500 with its own integrated graphics.

testing 3.26 beta... this bug is still happening.