After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 685676 - keyboard: Switch gsd-input-sources-switcher to use XI RawEvents
keyboard: Switch gsd-input-sources-switcher to use XI RawEvents
Status: RESOLVED OBSOLETE
Product: gnome-shell
Classification: Core
Component: keyboard
unspecified
Other All
: Normal normal
: ---
Assigned To: gnome-shell-maint
gnome-shell-maint
Depends on:
Blocks:
 
 
Reported: 2012-10-07 20:24 UTC by Rui Matos
Modified: 2013-04-26 21:04 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
keyboard: Switch gsd-input-sources-switcher to use XI RawEvents (17.09 KB, patch)
2012-10-07 20:24 UTC, Rui Matos
none Details | Review
keyboard: Switch gsd-input-sources-switcher to use XI RawEvents (27.34 KB, patch)
2012-12-03 22:30 UTC, Rui Matos
none Details | Review

Description Rui Matos 2012-10-07 20:24:51 UTC
The fact that we can't use a shortcut to switch input sources in the
overview is particularly problematic, even more so than other
shortcuts IMO since people often need to type text there and thus
potentially change input source.

We can't unfortunately do anything for the regular media-keys plugin
keybindings for 3.6, but, it occured to me, since the input sources
switcher is already using a different code path and is still quite
young we can experiment with it a bit more and try to fix this issue
at least for these shortcuts.

So, I re-implemented the whole thing by using XI RawEvents which
allows us to totally work around X grabs and thus always works (on XI2
supporting X servers at least, but this is 2012, etc.) It has, of
course, at least 2 drawbacks.

One is that this process will wake up for every key and button press
and release events. This doesn't strike me as too problematic though
since key and button press and release events are not generated at an
high enough rate for it to become a problem.

The other is that doing shortcuts this way can have side effects since
other clients are also getting the events at the same time. This is
not too problematic for these particular shortcuts since they consist
of only modifier keys and also because they are triggered only on key
releases and thus we're able to cancel the sequence as soon as we get
a button event or some other key is pressed or released.

I think this would be a nice fix for 3.6.1 .

Comments welcome.
Comment 1 Rui Matos 2012-10-07 20:24:53 UTC
Created attachment 226000 [details] [review]
keyboard: Switch gsd-input-sources-switcher to use XI RawEvents

XI RawEvents are delivered even while there are grabs in place by
other clients. This feature allows us to switch input sources even on
the gnome-shell's overview or the lock screen which brings us back
closer to how layout switching worked in GNOME < 3.6 .
Comment 2 Bastien Nocera 2012-11-05 17:23:16 UTC
Adding Peter in to review this.
Comment 3 Peter Hutterer 2012-11-07 01:52:27 UTC
First, one assumption is wrong - only as of server 1.12 (XI2.1) are raw events delivered at all times. For earlier servers, you won't get raw events while a grab is on (unless the grab is held by the same client).

because of this, you're playing a bit with fire here. raw events do not provide modifier state and you may not know that you've missed a raw event. so you may be tracking the wrong modifier state and switch input sources erroneously.

To avoid that, you should be requiring XI 2.1 or later (i.e. server 1.12) _and_ you have to announce yourself as XI 2.1+ client.

Also note that raw event delivery is different to normal event delivery - you'll be getting the raw event _before_ the key event. This means you must be careful preventing side-effects here, you may be switching at the wrong time, causing the real key event to be processed with the modified state.

And finally, unlike a grab, the raw event handling will not consume the original event, so the events you parse for switching will still be sent to the application. depending on the shortcut, this could be an issue.

all in all, I think it could be made to work, but it may be quite flimsy and subject to odd bugs that require complex event processing and conditional discarding of events. we generally don't recommend people use raw events for use-cases where they also need real key events.
Comment 4 Rui Matos 2012-11-07 14:29:11 UTC
Thanks for the input!

(In reply to comment #3)
> because of this, you're playing a bit with fire here. raw events do not provide
> modifier state and you may not know that you've missed a raw event. so you may
> be tracking the wrong modifier state and switch input sources erroneously.

I don't understand you here nor how modifier state is important. How can I miss an event if I'm getting all the raw key presses and releases?

> To avoid that, you should be requiring XI 2.1 or later (i.e. server 1.12) _and_
> you have to announce yourself as XI 2.1+ client.

g-s-d already requires XI 2.0 for the media-keys plugin so this doesn't sound too much of a problem. Adding the explicit XIQueryVersion is easy enough.

> Also note that raw event delivery is different to normal event delivery -
> you'll be getting the raw event _before_ the key event. This means you must be
> careful preventing side-effects here, you may be switching at the wrong time,
> causing the real key event to be processed with the modified state.
> 
> And finally, unlike a grab, the raw event handling will not consume the
> original event, so the events you parse for switching will still be sent to the
> application. depending on the shortcut, this could be an issue.

Right, the events will always end up in the application too, but I'm making sure that, e.g. if shortcut is Alt+Shift, when both Alt and Shift are down, if another key event or button press/release shows up I cancel the process so when one of Alt or Shift release event comes the shortcut isn't triggered.

> all in all, I think it could be made to work, but it may be quite flimsy and
> subject to odd bugs that require complex event processing and conditional
> discarding of events. we generally don't recommend people use raw events for
> use-cases where they also need real key events.

Ok. I think the specific use case and the set of shortcuts here don't make it as complex as it might seem but I might very well be underestimating something.

The global shortcuts handling will be moving places to the compositor for 3.8 anyway so this would only be a temporary pain-killer for users of 3.6, so not much of problem anyway. Bastien?
Comment 5 Bastien Nocera 2012-11-07 15:04:52 UTC
(In reply to comment #4)
> The global shortcuts handling will be moving places to the compositor for 3.8
> anyway so this would only be a temporary pain-killer for users of 3.6, so not
> much of problem anyway. Bastien?

I doubt that gnome-shell really wants to go through those hoops. Mutter will still only be an X client, and have to go through the same pain.
Comment 6 Peter Hutterer 2012-11-07 21:52:06 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > because of this, you're playing a bit with fire here. raw events do not provide
> > modifier state and you may not know that you've missed a raw event. so you may
> > be tracking the wrong modifier state and switch input sources erroneously.
> 
> I don't understand you here nor how modifier state is important. How can I miss
> an event if I'm getting all the raw key presses and releases?

you missed some context here. You only get events all the time if you are an XI 2.2+ client. So that means server 1.12 or later.

> g-s-d already requires XI 2.0 for the media-keys plugin so this doesn't sound
> too much of a problem. Adding the explicit XIQueryVersion is easy enough.

not quite as simple as you may think, there are pitfalls there too. For example see http://git.gnome.org/browse/gnome-settings-daemon/commit/?id=a249eb98818547fef4865382c6db2298d268b293
 
> Right, the events will always end up in the application too, but I'm making
> sure that, e.g. if shortcut is Alt+Shift, when both Alt and Shift are down, if
> another key event or button press/release shows up I cancel the process so when
> one of Alt or Shift release event comes the shortcut isn't triggered.

cancel what process? note that because you'll be handling the shortcut out-of-band from normal key events, you may interfere with clients that do require this shortcut. IIRC eclipse has a similar shortcut, not using grabs means you cannot easily discard or replay the events (well, technically they're always replayed).
Comment 7 Rui Matos 2012-11-07 22:18:56 UTC
(In reply to comment #6)
> you missed some context here. You only get events all the time if you are an XI
> 2.2+ client. So that means server 1.12 or later.

Ok, that much I understood. I don't think that that server version requirement is problematic for Gnome at this point.

> > g-s-d already requires XI 2.0 for the media-keys plugin so this doesn't sound
> > too much of a problem. Adding the explicit XIQueryVersion is easy enough.
> 
> not quite as simple as you may think, there are pitfalls there too. For example
> see
> http://git.gnome.org/browse/gnome-settings-daemon/commit/?id=a249eb98818547fef4865382c6db2298d268b293

Yes, I've seen that code.

> > Right, the events will always end up in the application too, but I'm making
> > sure that, e.g. if shortcut is Alt+Shift, when both Alt and Shift are down, if
> > another key event or button press/release shows up I cancel the process so when
> > one of Alt or Shift release event comes the shortcut isn't triggered.
> 
> cancel what process?

"cancel the process" → "not trigger the shortcut action"

> note that because you'll be handling the shortcut
> out-of-band from normal key events, you may interfere with clients that do
> require this shortcut.

Sure, I'm aware of that but I wonder how many apps out there have shortcuts like Alt+Shift or Shift+Caps.

The only way to do this sanely is to be inside the display server's event processing code. Since we can't do that until Wayland I'm not seeing a better alternative.

> IIRC eclipse has a similar shortcut, not using grabs
> means you cannot easily discard or replay the events (well, technically they're
> always replayed).

If applications use shortcuts such as Alt+Shift then yes, there's nothing better we can do for now. Doing grabs, as in the current code, means that the application's shortcut won't work and the user will have to change one of them. Doing it this way means that both the app and this shortcut will get triggered and the user will have to change one of them too to get a sane behavior.

Also, just to make it clear, these shortcuts are not set by default and they are not even currently exposed in the UI (ie in g-c-c).
Comment 8 Peter Hutterer 2012-11-07 23:40:22 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > note that because you'll be handling the shortcut
> > out-of-band from normal key events, you may interfere with clients that do
> > require this shortcut.
> 
> Sure, I'm aware of that but I wonder how many apps out there have shortcuts
> like Alt+Shift or Shift+Caps.

apps may not, but the XKB configuration may change groups on alt-shift, so you need to manage that as well.
 
> The only way to do this sanely is to be inside the display server's event
> processing code. Since we can't do that until Wayland I'm not seeing a better
> alternative.

https://fedoraproject.org/wiki/Features/Grab_override would be the appropriate resolution I guess, except that that never got anywhere

> If applications use shortcuts such as Alt+Shift then yes, there's nothing
> better we can do for now. Doing grabs, as in the current code, means that the
> application's shortcut won't work and the user will have to change one of them.

not quite, synchronous grabs do allow replaying of events, so you can have a grab on the root window for a key combination while an application has the same grab on its top-level window. In the client owning the root window grab you can decide whether to discard the event (once the grab activates) or to re-play it on the next window (i.e. the application). The difference to raw event processing is that you have a choice of doing so, raw events would be similar to always replaying the event.
Comment 9 Rui Matos 2012-12-03 22:30:27 UTC
Created attachment 230597 [details] [review]
keyboard: Switch gsd-input-sources-switcher to use XI RawEvents

--

Rebased on master (also applies to gnome-3-6) for those who can't live
without this for 3.6 (and don't mind compiling on their own).
Comment 10 Bastien Nocera 2012-12-18 16:41:40 UTC
Slinging to gnome-shell.
Comment 11 Igor Gnatenko 2013-04-26 13:33:09 UTC
Since 3.8.x patch not work
Comment 12 Jasper St. Pierre (not reading bugmail) 2013-04-26 15:04:15 UTC
Yeah, the key handling moved to mutter, where it should work forever.
Comment 13 Igor Gnatenko 2013-04-26 15:13:44 UTC
(In reply to comment #12)
> Yeah, the key handling moved to mutter, where it should work forever.
If I set parameter "Modifier-only..." in gnome-control-center keyboard in overview kbd not switch.
Comment 14 Florian Müllner 2013-04-26 21:04:38 UTC
Yes, modifier-only input source switching hasn't landed yet, see bug 697008. The approach here is obsolete and won't be pursued any further.