After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 749390 - GDM 3.16 breaks hardware acceleration in VirtualBox
GDM 3.16 breaks hardware acceleration in VirtualBox
Status: RESOLVED NOTGNOME
Product: gdm
Classification: Core
Component: general
3.16.x
Other Windows
: Normal normal
: ---
Assigned To: GDM maintainers
GDM maintainers
: 746850 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2015-05-14 16:57 UTC by araban
Modified: 2018-02-12 15:06 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Server-managed FDs for vboxvideo (6.80 KB, patch)
2015-11-14 09:58 UTC, Michael Thayer
none Details | Review

Description araban 2015-05-14 16:57:07 UTC
With gdm 3.14 hardware acceleration for gnome-session works correctly on linux guests in VirtualBox.  Upgrading to gdm 3.16 breaks hardware acceleration for gnome-session in VBox.  

Launching gnome-session from Lightdm (instead of gdm 3.16) works correctly (hardware acceleration works correctly). Downgrading all gnome components to 3.14 also fixes it.  Disabling wayland does not fix it.

It may be related to Bug 747339 (GDM Runs on TTY1, While GNOME Shell Started in TTY2), as attempting to switch to tty1 in the virtual environment causes a hard lock that requires powering off the virtual machine from outside the virtual environment.
Comment 1 Ray Strode [halfline] 2015-05-14 17:32:14 UTC
does your distribution build Xorg with --enable-suid-wrapper ?

if you add needs_root_rights=yes to /etc/X11/Xwrapper.config does it fix the problem?
Comment 2 John Woo 2015-05-14 17:35:52 UTC
Confirmed. 
VBox 3D accel not provided if gdm 3.16 in startup. Another dm or downgrade solves the issue.
Comment 3 John Woo 2015-05-14 17:44:58 UTC
I neither used --enable-suid-wrapper nor have Xorg in systemd.
Comment 4 araban 2015-05-14 17:47:36 UTC
My guest distro (Arch) does build Xorg with --enable-suid-wrapper

Adding needs_root_rights=yes to my guest's Xwrapper.config and rebooting the VM did not fix the issue.
Comment 5 araban 2015-05-15 19:38:07 UTC
Ray, do you need any additional info from me?  Happy to provide journal excerpts, glxinfo output, etc.
Comment 6 araban 2015-05-28 22:01:09 UTC
Any word on what additional info might be needed at this point?

Other Arch users are running into this issue as well: http://bbs.archlinux.org/viewtopic.php?id=197885
Comment 7 Eric Donkersloot 2015-05-29 11:31:37 UTC
extra/gdm 3.16.1.1-2 (gnome) [installed]

OpenGL renderer string: Gallium 0.4 on llvmpipe (LLVM 3.6, 128 bits)

Linux virtmac 4.0.4-2-ARCH #1 SMP PREEMPT Fri May 22 03:05:23 UTC 2015 x86_64 GNU/Linux

This bug is still present in the latest gym version on Arch Linux.
Please let us know if you require more information.
Comment 8 Carwyn Edwards 2015-10-04 14:22:48 UTC
I can confirm this issue on Fedora 22 and 23 on VirtualBox 5.0.4 and 5.0.6. I have the guest additions from here https://copr.fedoraproject.org/coprs/sergiomb/vboxfor23/ with akmods from rpmfusion. With GDM I end up with llvmpipe as the renderer. With lightdm I get Chromium (Vboxvideo accelerated).
Comment 9 Ray Strode [halfline] 2015-10-04 18:00:29 UTC
so i'll have to try to reproduce to be sure, but my guess is that the vbox driver doesn't support "server managed fds" / platform devices

I know hans did patches for the vmware driver, but maybe not for the vbox driver.

GDM relies on that support the way it starts X in fedora 22 and 23.  I thought forcing needs_root_rights=yes would work around the issue, but it could be it's giving preference to the modesetting driver over the vbox driver anyway.

Xorg log would probably say for sure if it's picking modesetting driver.

You might also be able to force the vbox driver with a snippet in xorg.conf.d (plus the change in comment 10).
Comment 10 Hans de Goede 2015-10-06 13:47:43 UTC
Hi,

(In reply to Ray Strode [halfline] from comment #9)
> so i'll have to try to reproduce to be sure, but my guess is that the vbox
> driver doesn't support "server managed fds" / platform devices

Erm, right, unless the vbox people implemented that themselves. I think I've overlooked the vbox driver when converting all the drivers.

> I know hans did patches for the vmware driver, but maybe not for the vbox
> driver.

Right.

> GDM relies on that support the way it starts X in fedora 22 and 23.  I
> thought forcing needs_root_rights=yes would work around the issue,

Right.

> but it
> could be it's giving preference to the modesetting driver over the vbox
> driver anyway.

I don't think so, but  never say never say never.

Regards,

Hans
Comment 11 Jasper St. Pierre (not reading bugmail) 2015-10-06 15:16:17 UTC
The modesetting driver should work fine, since VirtualBox installs a DRM driver as part of its guest additions as well.

https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Additions/x11/vboxvideo/
https://www.virtualbox.org/svn/vbox/trunk/src/VBox/Additions/linux/drm/
Comment 12 Ray Strode [halfline] 2015-10-06 15:20:15 UTC
well note, no one is saying their systems don't work. just that there systems are unaccelerated
Comment 13 Ray Strode [halfline] 2015-10-06 16:46:02 UTC
also their systems.
Comment 14 Hans de Goede 2015-11-02 07:42:55 UTC
Hi,

So if there is someone who wants to take a shot at this, here is a reply with what needs to be done, which recently send to Larry Finger who is also interested in this:

Hi Larry,

On 21-10-15 18:17, Larry Finger wrote:
> Hans,
>
> I recently took on the job of maintaining the VirtualBox package for openSUSE. I am not familiar with the internals of the emulator.
>
> Recent changes in the openSUSE setup are causing both Tumbleweed and Leap installations to fail to start X as a VirtualBox VM with a GNOME desktop because X no longer runs with root privilege. I have seen your fix for VMWare as found in http://pkgs.fedoraproject.org/cgit/xorg-x11-drv-vmware.git/tree/0002-Add-support-for-server-managed-fds.patch. Before I start to dig through a lot of unfamiliar code, do you have any references describing what is needed?

No references but I can give you a quick run through.

With server-managed-fds, the server / the driver no longer directly opens device files,
for video driver typically we are talking about opening /dev/dri/card? . Instead
the server now talks to systemd-logind and that opens input and video devices for the session,
and in case of dri devices also does the root-only call to become drm master.

This allows the server to run as non root, but this requires the driver to get the fd
from the server rather then trying to open /dev/dri/card? itself.

The patch you linked to is a somewhat complicated example of how to do this, with some
drivers this is simpler.

Being able to get to the fd from the server requires the driver to use the platform-driver
way of binding (rather then the old pci only pci-method).

In one case I had to first teach the driver to support the platform bus:

http://cgit.freedesktop.org/xorg/driver/xf86-video-qxl/commit/?id=463084aa8e73fbafb8dc404e2bd458548d04e9dc

Once there is support for the platform bus (which hopefully is already there, but may not
be there) you need to save the struct xf86_platform_device pointer passed into the platform probe
function, see the qxl driver for an example. Note the vmware driver uses a somewhat convoluted
manner to get to the platform_device pointer, I do not remember why I did things that way.

Then it is a manner of finding where the driver tried to open the drm node (there may be more
then one place) and add something like this:

+#if defined(ODEV_ATTRIB_FD)
+    if (platform_dev) {
+        fd = xf86_get_platform_device_int_attrib(platform_dev, ODEV_ATTRIB_FD, -1);
+        if (fd != -1)
+            return fd;
+    }
+#endif

     <continue with trying a normal open here>

And also find all the sites where the driver tries to do a drmSetMaster() /
drmDropMaster() and make these calls condition under a condition of:

+#ifdef XF86_PDEV_SERVER_FD
+    if (!(platform_dev && (platform_dev->flags & XF86_PDEV_SERVER_FD)))
+#endif
+    {
         ret = drmSetMaster(qxl->drm_fd);
         ...
+    }

Note the entire check is inverted not just the first operand to the &&.

Last you will need to tell the server the driver now supports server-managed-fds, by extending
the "DriverFunc" to support SUPPORTS_SERVER_FDS: as is done in the first 2 hunks of the
vmware driver patch you linked to.

I hope this helps. You're the second person in a relative short time to inform about
server-managed-fds support for the vbox driver (and the first one with the necessary
coding skills).

Feel free to ask more question, or to send concept patches my way, even if they do not
work, I might be able to spot why they do not work a lot quicker then you (it has been
a while since I last worked on this though).

> Of course, finding a direct fix for the VirtualBox video driver would be even better.

I'm sorry but no one has written a server-manager-fds patch for virtualbox yet AFAIK.

Regards,

Hans
Comment 15 Michael Thayer 2015-11-14 09:56:23 UTC
To be honest, I would prefer moving over to the KMS driver, which is working with rough edges and using that with modesetting.  I did actually locally add platform bus and server managed FD support to vboxvideo two days ago, patch below for anyone interested.  I will probably not commit it though: up until now my problem has always been that 3D support did not work with the KMS driver due to the unconventional (to put it kindly) way that we hijacked Mesa to make it work.  I have locally switched to using ld.so.conf.d to get our OpenGL library loaded, so that is no longer a blocker.  (It won't help us with Wayland of course, as Wayland assumes that 3D and 2D go through the same device and that you can created surfaces with 3D APIs and display them through the KMS driver.  But that is the next building site, as the Germans say.)

Hope that update is helpful.
Comment 16 Michael Thayer 2015-11-14 09:58:15 UTC
Created attachment 315448 [details] [review]
Server-managed FDs for vboxvideo

This is purely for the interest of people subscribed to this bug.
Comment 17 Michael Catanzaro 2016-01-11 18:15:36 UTC
*** Bug 746850 has been marked as a duplicate of this bug. ***