GNOME Bugzilla – Bug 319735
gok works poorly as an input mechanism on tablet PCs
Last modified: 2011-10-14 10:58:15 UTC
A typical tablet PC X configuration has the following: 1) A PS/2 core pointer 2) A synaptics pad with "SendCoreEvents" "true" 3) A cursor device for the tablet, with "SendCoreEvents" 4) A stylus device for the tablet, with "SendCoreEvents" 5) An eraser device for the tablet, with "SendCoreEvents" Ideally, gok would be usable with the cursor device without preventing it from being used as the core pointer (tablets aren't very useful otherwise). At the moment there doesn't seem to be any way of configuring that.
Matthew, thanks for this bug report. It would be good to have a corepointer mode (perhaps a flag) where devices were not automagically unhooked. I'm not 100% sure gok can't operate this way currently... but this area needs to be looked at and clarified a bit. (Note we are also looking into alternatives to using XINPUT extensions for adaptive hardware input) Perhaps we should have a flag: -c, --c Use the core mouse device as is.
Matthew, unfortunately it isn't technically possible to use GOK via the corepointer. Talk the the toolkit folks if you think that's wrong, nothing we can do ATM. David, I don't think 'using core mouse device as is' will help, as it will just cause "hangs" and "lockouts" from the users' POV. Apps are constantly doing pointer grabs, which stops GOK from working. IN any case, GOK currently allows you to "cancel" if you don't want to unhook the core device.
Matthew, is it apparent to you (wearing your gok user hat) that cancel does not unhook the core device?
I would concede the end result of pressing 'Cancel' probably isn't apparent to the user. (Bill)
OK. BTW Bill, I envision the -c option being for users who can use the regular mouse but happen to require an on-screen keyboard (for character input). We would bypass all the hardware detection stuff.
Yes - though even for character input it may act broken, since more and more file choosers and entry widgets and what-not seem to be doing grabs.
On starting gok, regardless of whether I choose to unhook the core pointer or not, I currently get two mouse pointers. One follows the pen around the screen and allows me to interact with other applications - the other remains within gok. This wouldn't be a problem except that the pointer within gok reacts even if the screen-wide pointer is outside gok. So possibly the behaviour I want is simply for gok not to react if the core pointer is outside the gok window - that way, the core pointer could be used to interact with applications and the gok pointer to interact with gok. Would this work? (Sorry, I'm not entirely clear on how the pointers interact with gok, so I may be talking complete nonsense here)
I still feel like this grab behaviour ought to be end-of-life'd somehow.
Matthew, I'm too foggy on gok's input device magic ATM to comment intelligently other than to say I have seen the behaviour you describe. I think you idea for solution has merit as would perhaps not even having a second (gok) pointer for core mode.
Matthew: behavior you describe in #7 sounds like misconfiguration to me. If you're seeing two pointers moving "at once", you have a misconfigured Xserver and/or GOK.
In X, I have: Section "ServerLayout" Identifier "Default Layout" Screen "Default Screen" InputDevice "Generic Keyboard" InputDevice "Configured Mouse" InputDevice "Synaptics Touchpad" InputDevice "cursor" "SendCoreEvents" InputDevice "stylus" "SendCoreEvents" InputDevice "eraser" "SendCoreEvents" EndSection In "actions" in gok, I have event source set to core pointer. Moving the touchpad on the system results in the "gok pointer" (a small cross) moving within the gok window. Moving the pen on the screen results in both the "gok pointer" and the normal system cursor moving. The X setup is the recommended configuration for wacom-based tablet systems, and "cursor", "stylus" and "eraser" are the three different wacom input devices.
The problem is that these 'different input devices' are connected to XInput as extended devices, but are still sending core events. THis is a major problem for GOK, since GOK sees these as non-core input devices (GOK can't tell that SendCoreEvents is set).
Unfortunately, I can't see any workable strategy that doesn't involve them sending core events - it has to be possible to drive the system cursor with the stylus. Is it possible for gok to tell that the system cursor (I've probably got the terminology wrong here - is that in fact the core pointer?) is outside the gok window, and if so not update the gok pointer?
No, the GOK pointer should not be updating at all if the core pointer is being moved. It seems that GOK is getting events from the XInput device channel in this case, and it cannot ignore such events and still work in its recommended configuration (which would be without SendCore on the XInput extended device being used).
Here's the dilemma, I think, that leads to this behavior: 1) USB devices are prone to renumbering across boots; 2) GOK therefore needs to auto-detect unexpected XInput events, since otherwise a renumbering will cause the pre-configured GOK XInput device connection to fail without offering the user a solution. 3) however, SendCore devices are unfortunately not identified by the XInput extension as "core" events (though CorePointer events are). 3 means that all SendCore devices will generate XInput events which GOK cannot distinguish from case #1 above, and the code to auto-detect in #2 gets invoked, causing GOK to receive these events as XInput events. Because they are SendCore, although GOK is seeing them as extended devices, the devices in question still move the core pointer, and are still subject to the pointer grab problems as a result of application and toolkit pointer grabs. I don't see an obvious solution here, other than (configurably) turning off the auto-detection of XInput devices by GOK.
Stepping outside of the current and immediate X constraints, would: (a) libusb allow us to solve this problem? [could Matthew still get his events that way?] or (b) XEvIE, especially if extended to support multiple clients?
verily if we could but cast aside those shackles and GOK freely... There is some serious question as to whether libusb can be used with devices which are serving as core pointer and/or recognized already as HIDs. We suspect not. XEvie, if extended to multiple clients, could do it for us.
Tablet PCs generally (actually, pretty much exclusively) have devices connected via a serial interface, not USB. We can probably make guarantees about the device that they're connected to, but I'm not sure if that's useful.
In response to comment #17, libusb can be used with devices that have been recognized HID by detaching the device from the kernel driver. Libusb provides a function to do this but it seems to be Linux specific right now. Once the device is detached from the kernel driver, it no longer serves as the core pointer because the underlying entry in /dev is no longer present. I'm still not exactly sure if this will help with the problem at hand.
A bit of an aside: WRT the whole issue of having a non-core gok mouse to avoid mouse grabs... I wonder if we could trap mouse grab calls and react at that stage: - not allow the mouse grab? - unhook from core at that point, re-hooking after the grab? I doubt this is possible currently, but it might be a future compromise (with a bit of work).
There's no way to trap mouse grab calls, unless you're the X server. Client applications can't do that. It makes more sense to extend XEvie, since allowing client applications to snoop calls from other clients is very troublesome for lots of reasons (i.e. it breaks the X model, introduces security issues, causes performance roundtrips, etc. etc.)
Yeah I hesitated to move the conversation in this direction because the libusb/xevie/etc work is important. I notice: "Option "AllowDeactivateGrabs" "boolean" This option enables the use of the Ctrl+Alt+Keypad-Divide key sequence to deactivate any active keyboard and mouse grabs. Default: off. " -- http://wiki.x.org/X11R6.8.0/doc/Xorg.1.html I'd like to see this deactivation assigned to gestures. Not now, but something to think about?
Bear in mind that this deactivation is for "active" grabs, i.e. only works when a grab is already in effect. The other grabs that applications do are "passive grabs", which basically means "pending grabs", which take effect (i.e. become active) when a particular key combination is pressed. It's possible that GOK could be made to programmatically make this call if it stops receiving valuator input, I am not sure.
OK, I've created bug 322655 for the mouse grab discussion.
Apologies for spam... ensuring Sun a11y folks are cc'ed on all current accessibility bugs.
I have essentially the same setup as Matthew (comment #11). My problem is that I cannot achieve shift, control or alt key effects using the onscreen Compose keyboard, either with the touchpad mouse pointer or with the tablet stylus. If I touch the Caps Lock button, the little triangle lights up and I can achieve capital letters, but with the other keys the little triangle doesn't light up and the button doesn't function. Bizarrely, if I hold down the Shift or Control button on the conventional laptop keyboard and then touch a key with either mousepointer, the shifted or controlled function of that key now works (and the little triangle in the GOK button lights up). This is not, of course, a useful way to use the onscreen keyboard! The same problem is encountered on a conventional desktop computer using ordinary keyboard and mouse; the Control, Alt and Shift keys don't appear to function when pointed to by th eon-screen device, though their function is activated by touching the corresponding key on the main keyboard. Setup is fairly conventional: Desktop with Mepis 32 v 6.5, GOK v 1.0.10; laptop is Toshiba Satellite R10-101 with same versions of software. Wacom stylus etc devices are found at /dev/ttyS4. Any thoughts?
Chris, do you have Sticky Keys set up and working?
As soon as it launches, I am told in a dialog box: 'GOK has enabled sticky keys, which it needs'. I don't know whether or not this is true, and what is its significence? I also get these messages at the terminal: (gok:7640): Gnome-WARNING **: Accessibility: failed to find module 'libgail-gnome' which is needed to make this application accessible GTK Accessibility Module initialized /dev/js0: No such file or directory /bin/sh: /usr/bin/esd: No such file or directory 134958736 event types available login mode = false Word prediction dictionary contains a total of 3000 words I have installed libgail-gnome from the repository; it doesn't solve the problem. I cannot find esd as such in the repositories, though it appears as a suffix to several packages; it seems to have something to do with sound so is probably irrelevant to my problem. I presume /dev/js0 is a joystick, which is not present in my system.
gok (GNOME on-screen keyboard) development has been stalled and it has been replaced by caribou [1]. Maintainers don't have future development plan so i am closing all the bugs as WONTFIX. [1] https://mail.gnome.org/archives/gnome-bugsquad/2011-October/msg00001.html