Created attachment 128635 [details]
This shows the problem with mutter and gnome-display-properties

wacky. probably somehow related to the override-redirect window that gnome-display-properties creates?

Probably entirely a mutter bug, not gnome-shell, so you may want to file a bug at bugzilla.o-hand.com

Okay, I have created a bug-report at the o-hand Bugzilla.

http://bugzilla.openedhand.com/show_bug.cgi?id=1461

I am not entirely convinced this is a mutter bug; I used to see this all time with regular metacity when I used to run Ubuntu, and it went away when I switched to debian about a year ago. The list looks like it is full of empty items on the top that were not taken into account when the list size was calculated, so it has to show the arrows; as soon as  you scroll the list, the empty items are gone and the arrows disappears. It is possible that metacity is moving the OR window to the wrong location when it maps, but I cannot reproduce this either with metacity or mutter.

Tomas - actually this is something different - it's not the standard (as designed) behavior where GTK+ tries hard to put the current item under the mouse when you pop an option menu up.

This is a much weirder behavior where menu items that are over the parent window aren't selectable. It feels like there is an input only window that is getting misstaked or something. It may be specific to particular versions of gnome-display-properties - maybe omething to do with the little monitor labels it puts up.

Right; Mutter doesn't handle any pointer events, the stage window input shape is set to 0x0, and we do not create any windows ourselves. So the three scenarios I can think of are (a) the Shell is blocking input (changing input shape of stage, for example), or (b) there is an input-only window created by the application itself that is incorrectly sized/placed, or (c) the Metacity stack is messed up, with the menu being stacked below the application. Looking at the video again, (c) seems a very distinct possibility -- should be easy to verify with xwininfo/xprop, but I can't reproduce it.

yes, the immediate problem seems to be the stacking order, but more generally, the problem seems to be that in gnome-shell the main gnome-display-properties window is always stacked above every other gtk window (including the bottom panel and gnome-display-properties's override-redirect monitor label) even though it doesn't have any of the properties that would cause it to be stacked specially.

i'll keep investigating

Is it by any chance stacked above the stage window ?

There's only one window in the "xwininfo -tree -root" output that is full-screen-sized, so I'm assuming that the stage window you're talking about, but it's at the very very bottom of the window stack.

gnome-display-properties behaves normally if either (a) you tweak the glade file so that it has window hint "normal" instead of "dialog", or (b) you hack gnome-desktop so that GnomeRRLabeler creates a GTK_WINDOW_TOPLEVEL instead of a GTK_WINDOW_POPUP. Perhaps mutter is mistakenly treating the main window as being TRANSIENT_FOR the monitor label window?

OK, I think I see what is going on here. The menu that gnome-display-properties shows is a pop-up menu, and looking at the gtk code this is made transient for the application top-level window. So, this gets stacked based on the transiency just above the app window.

I suspect that the override-redirect window that is created by the labeler is not marked as transient (or transient to root) and so ends up in the META_LAYER_OVERRIDE_REDIRECT layer that is above everything else. If I am correct, making this window also transient to the application should fix this problem (the reason this would not happen with regular metacity is because it does not track OR windows, so if the menu is created after the labeler, it will be stacked above).

OK, running with METACITY_VERBOSE:

    STACK: Adding window 0xc00003 (gnome-disp) to the stack

That's the override-redirect labeler window

    STACK: Recomputing layers
    STACK: Window 0xc00003 (gnome-disp) on layer 7 type = 15 has_focus = 0

layer 7 = META_LAYER_OVERRIDE_REDIRECT. Everything is good so far.

    STACK: Adding window 0xc00020 (Monitor Re) to the stack

That's the main window

    STACK: Recomputing layers
    STACK: Promoting window 0xc00020 (Monitor Re) from layer 2 to 7 due to group membership

And here's the bug. The main window gets promoted to META_LAYER_OVERRIDE_REDIRECT, but it shouldn't.

(In reply to comment #10)
> If I am
> correct, making this window also transient to the application should fix this
> problem

That might work around the problem, but it seems like the bug is in mutter, not gnome-display-properties; apps are generally allowed to assume that the wm will ignore override-redirect windows, and will not change the behavior of other windows in the application based on what the app does with the override-redirect ones.

Created attachment 129571 [details] [review]
Ignore override-redirect windows when restacking based on window groups.

This fixes gnome-display-properties. There may be other patches needed
here or elsewhere for similar problems?

Many thanks for tracking this down; the patch looks sound, I put it on my TODO list to test that nothing unexpected breaks.

As a side note, this merely confirms my conviction that override-redirect windows are truly evil (and often abused; lot of stuff that ORs are used for can be accomplished using EWMHs); the assumption that the WM will (can) ignore ORs only really holds when no compositing is involved, once you start compositing the entire concept of an OR window becomes somewhat meaningless. But that's me just ranting ...

Patch applied in commit e0915f4daa7feea10de42b18108139661f346dff.

Merged into gnome-shell branch.