GNOME Bugzilla – Bug 753527
Regression since 3.16.x: pen & touch freezes display on Thinkpad Yoga
Last modified: 2017-07-23 12:20:24 UTC
On a Lenovo Thinkpad Yoga S1 with pen and touch, with Fedora 22 and gnome-shell-3.16.3-1.fc22.x86_64, the pen and touchscreen devices being used simultaneously (either writing with the pen + touching the screen with the other hand, or just writing with the pen while the hand is resting on the screen, or even hovering the pen above the screen while touching it with the other hand) causes the display to freeze completely for a second or two -- the cursor disappears, all applications become seemingly unresponsive. (Things come back on their own after a while, and the pen/touch events generated during the freeze do get processed by applications eventually). The user who first reported this among other Fedora 22 issues indicates that no such problem occurs in the KDE desktop environment, so it's presumably a Gnome Shell issue and not an X server / wacom driver issue. http://forums.fedoraforum.org/showthread.php?t=305444 To reproduce: on a Lenovo Thinkpad Yoga S1 (with wacom pen) running Fedora 22, just use the touchscreen and pen at the same time under Gnome. (To make this very visible, use a drawing program such as xournal or gimp, but just clicking on the desktop background will have the same freezing effect). This may affect other systems with similar pen/touch systems. On the Yoga, the pen is Wacom (xinput device: Wacom ISDv4 EC Pen stylus) while the touchscreen is a different beast (xinput device: ELAN Touchscreen). Packages: all the latest Fedora 22 packages, including: xorg-x11-server-Xorg-1.17.2-2.fc22.x86_64 xorg-x11-drv-intel-2.99.917-12.20150615.fc22.x86_64 xorg-x11-drv-wacom-0.29.0-2.fc22.x86_64 gnome-shell-3.16.3-1.fc22.x86_64 There was no such problem on Fedora 20 with latest updates (gnome-shell 3.10.x with some patches); the report on fedoraforum.org mentioned above suggests no issue with Fedora 21 either (gnome-shell 3.14.x).
Upon further testing: 1. The pen has nothing to do with it. Using the touchscreen + an external mouse simultaneously also causes display freezes (shorter than with the pen -- presumably because a wacom pen sends a much larger amount of X events than a mouse). Whereas pen + mouse works fine. This makes me think the issue might be much broader than just Thinkpad Yoga and might affect all touchscreens. (it's just that on a system with a pen you're much more likely to hit the touchscreen while drawing -- not so common with a mouse, or at least you don't really have to do it :-) 2. The display freeze issue occurs both on plain Xorg (regular gnome session) or on Xwayland (gnome-wayland session). (I don't understand enough about wayland to know whether that helps pinpoint the source of the problem). If I had to make a rough guess: I would imagine that gnome-shell gets stuck trying to decide if the touch events are part of a gesture when they come in mixed alongside events from other devices, or something like that. (Not sure how you are handling touch gestures -- I haven't seen it work yet -- but having written similar code I'd like to point out that it's easy to mangle xinput device event processing when several devices are talking at once if you're not careful.) Denis
Okay, if like me you are desperate for a quick workaround so that you can actually use your pen+touch Tablet PC under gnome 3.16, there's another solution besides downgrading gnome. I just wrote a small tool that disables the touchscreen when the wacom pen reports proximity to the screen. It's only had a few hours of testing so far, so use at your own risk. http://math.berkeley.edu/~auroux/software/xnohands.tar.gz (installation instructions for common use scenario are included in README file) This doesn't fix the underlying gnome-shell issue, of course, and you'll still notice gnome-shell freezing the display when using the touchscreen and a touchpad/mouse/... (anything other than a wacom pen, really) at the same time -- but I don't see any compelling reason to want to do so. Denis
I can confirm this, running Manjaro and Gnome 3.16.2 on a Thinkpad Yoga S1. As desribed above, Gnome-shell freezes when the touchscreen is being used and at the same time either the trackpad or the stylus is used. Freezes are quite long (5+ seconds). Christoph
I can confirm this on Arch Linux and Gnome 3.18.3. My hardware is a Cube i7 Stylus.
I can also confirm this problem occurring between the Elantech trackpad and the touchscreen on my Asus T100 convertible laptop, on current Debian Testing/Unstable (GNOME Shell 3.18.3). I need to wait several seconds before the touchscreen responds to taps after using the trackpad, and during this time, the gnome-shell process has a CPU spike.
I can confirm this, running Ubuntu 15.10 under Gnome 3.16.0-1ubuntu2 on a Thinkpad Yoga S1.
I can also confirm. Running Gnome 3.18.2 (gnome-shell 3.18.3) on Fedora 23. My machines is also the Thinkpad Yoga S1. I haven't noticed the same issue in KDE or openbox environments.
Created attachment 322422 [details] [review] backends: rate-filter the emission of MetaBackend::last-device-changed signal It indirectly triggers expensive operations in gnome-shell (js/ui/keyboard.js), which turns out too expensive if we happen to operate the shell simultaneously with 2 devices that will trigger the operations there. So just rate limit the signal emission, defer to an idle and just emit the last device gotten. Worst that will happen is that we may possibly emit the signal on the same device consecutively.
Review of attachment 322422 [details] [review]: Sure, seems fine
Cheers! Pushed. Attachment 322422 [details] pushed as 741dd67 - backends: rate-filter the emission of MetaBackend::last-device-changed signal
Hello again! First of all, thanks Carlos and Rui for your attention. GNOME handles my hardware the best out of all the major DE's and this is the only niggling issue I have with it. So again, thanks for your great work! I'm sorry that I didn't get to review your patch earlier Carlos. I'm a very new contributor and it took me a while to figure out which package I needed to compile. The fix you provided partially resolves the issue. There are no more 5 second hangs but I still experience a lot of lag in application. Here's a video demonstrating the problem in Xournal: https://youtu.be/wFOIYl5Rh_8 Video transcript: 1. The touch screen is set to be used as a "Hand" tool (for panning). Since the patch, "Pen disables Touch" has no effect since only one device event at any time is delivered to xournal. 2. Show that the touch screen has been correctly designated as such. 3. Dragging with touch screen only. Page moves up and down as expected. 4. Writing with digitiser. Response as expected. 5. Writing with digitiser AND dragging left-to-right on touch screen. Lag is observed. It's very difficult to do regular handwriting with this delay. Rather than sending the most recent device event, would it be possible prioritise mouse devices? For instance, digitiser events should always win over touch screen events. Alternatively, could both device events be sent simultaneously in a way that isn't as taxing? Hope this helps, Josh.
It seems to me then that the patch only partially alleviates the problem but does not solve it -- if there is still a noticeable lag then Gnome Shell is doing something wrong. I don't understand anything about the details -- but Gnome Shell should NOT be in the business of filtering XInput device events for applications, unless it knows for sure that it needs to take them, and it should not freeze or even lag when several devices are talking at once. Any filtering / reordering / queueing you want to do for Gnome Shell's own processing should be done in a separate internal queue from the main X event queue that gets presented to applications, and if device switching causes some heavy processing, that processing should be done in a separate thread. Applications should just receive the X device events as they arrive, on any number of devices, in real time, as is needed by drawing software such as Xournal and others. Or if you really need to do this sort of thing for whatever purpose, please make it easy to disable those features in the configuration. Denis
I echo the above comments. This is not fixed for me on my Asus T100 (Debian testing), either. The behavior is pretty much the same as before: when I start using the touchscreen, there is a couple-of-seconds long lag, while journalctl gets /usr/lib/gdm3/gdm-x-session[4176]: Activating service name='org.gnome.Caribou.Daemon' /usr/lib/gdm3/gdm-x-session[4176]: Successfully activated service 'org.gnome.Caribou.Daemon' org.gnome.Shell.desktop[4343]: (gnome-shell:4343): caribou-CRITICAL **: caribou_group_model_create_group_name: assertion 'group != NULL' failed org.gnome.Shell.desktop[4343]: (gnome-shell:4343): caribou-CRITICAL **: caribou_keyboard_model_populate_group: assertion 'group != NULL' failed This is the case whether or not I actually have caribou installed. If I am not careful and go back to using the mouse/touchpad while the touchscreen is lagging behind, sometimes I even get complete confusion in the form of an apparent "stuck left click", with, for instance, Shell dock icons being dragged wherever I take the mouse pointer.
Is there still anyway working on this? This problem is the reason I can't use Gnome as Xournal is not functional. I read somewhere that caribou is supposed to be removed from Gnome in the future.
Not sure if anyone is really working on it. Part of the problem is that it got marked as "resolved fixed" when it wasn't. That's really too bad. I'll try to undo it (apologies to the developers if they feel they fixed it, but really they didn't). As a workaround, I've been using xnohands successfully. See comment #2 above. I do have very occasional gnome-shell crashes and restarts while writing with xournal, but not frequent enough to be really bothersome. Or you can get a newer Yoga :) the latest models have wacom touchscreens that get automatically disabled when the pen is in proximity -- but the pen driver has a bug that causes xournal to have poor writing quality so I wouldn't necessarily recommend it... Denis
Similar to Dennis, I use a script to work around this issue. It's available here: https://github.com/admiralakber/thinkpad-yoga-scripts
*** This bug has been marked as a duplicate of bug 785309 ***