GNOME Bugzilla – Bug 758658
[REGRESSION] Nested mutter Wayland session is broken
Last modified: 2016-02-25 03:52:28 UTC
As of commit commit 8ec0c99ff4f5a47cd6078cec97527d93dd0c47e8 Author: Ray Strode <rstrode@redhat.com> Date: Wed Dec 17 13:29:13 2014 -0500 core: start as wayland display server when XDG_SESSION_TYPE=wayland This commit gets rid of the need for --display-server and --wayland when mutter detects that a wayland session is registered. https://bugzilla.gnome.org/show_bug.cgi?id=741666 it is no longer possible to run a nested mutter Wayland instance on top of another Wayland session. In effect one can no longer run debug non-KMS related gnome-shell/mutter issues without having to run it on top of KMS. Trying to run mutter --wayland from a gnome-terminal running in a Wayland gnome-shell session will result in the following error message: mutter-WARNING **: Can't initialize KMS backend: Could not take control: GDBus.Error:System.Error.EBUSY: Device or resource busy The reason for it to break is that when the XDG_SESSION_TYPE == wayland, which it is in such a case, forces the eglnative clutter backend. So we need another way than just checking the XDG_SESSION_TYPE variable for choosing the clutter backend.
Created attachment 316283 [details] [review] Add --nested command line argument As of "core: start as wayland display server when XDG_SESSION_TYPE=wayland" it is no longer possible to run a nested mutter Wayland session on top of another Wayland session. This patch adds a command line argument to make it possible to force mutter to start as a nested compositor.
I could not come up with a method using the sd_session API for detecting when we are in a wayland logind session, but should not start as a display server. We cannot check for a WAYLAND_DISPLAY variable, as AFAIU a long term wish is to get rid of that variable by default. We could try to initialize clutter backends and fallback to running nested, but not sure that's very desirable. Any opinions or ideas I didn't think of myself?
maybe a variable such as MUTTER_MODE? that can be set to wm (for being run as an x11 window manager), displayserver, or nested? With these recent changes, I'm also not sure how to properly start gnome-shell as a 'baremetal' Wayland session with gnome-session from my custom bash login manager as well (without custom tweaking the .desktop file)... (I could call gnome shell directly, but then I lose the ability for the logoff button to actually work IIRC?) Although, with an env var, you don't really get the same documentation with the --help output, but it would allow control...
Any objection to pushing this one? Cherry-picking this one is still the only way I have of debugging mutter without having to mess with VT switching and KMS.
Review of attachment 316283 [details] [review]: I'm okay with this patch. let's call it option 1. Some other ideas: 2) if the session type is wayland and isatty(stdin) then do nested 3) try to init the native backend and if it fails, fall back instead of exit 4) call the ReleaseControl logind api up front and fall back to nested if it errors with BUS_ERROR_NOT_IN_CONTROL 5) do nested if the controlling tty of the process is a pty 6) add a lock file in XDG_RUNTIME_DIR to see if mutter is already running and fall back to nested if it is I, personally, don't care which of the 6 options you go with, so pick your favorite.
I like the explicitness of --nested. The trial-and-error init seems a lot more complex and error prone, checking stdin is unreliable since I don't want nesting when starting from a VT, the same with checking the tty if I have a screen in VT, and adding a lock file would only work if its nesting from within mutter. 4) seems doable, if it is done before any other backend init, but I won't bother with that now.
Comment on attachment 316283 [details] [review] Add --nested command line argument Pushed the patch as is.