GNOME Bugzilla – Bug 647711
Stickykeys does not have a notification system
Last modified: 2021-07-05 14:32:13 UTC
When using the Stickykeys a11y feature, there is currently no way for the user to know what state (latched, locked, etc) the various sticky keys are in. GNOME 2.x provided the AccessX Status Applet, which displayed icons for Alt, Ctrl, etc and their state in a panel applet. There appears to be no equivalent in GNOME Shell, which represents something of an a11y hurdle for those of us who rely on Stickykeys. apologies if this is filed against the wrong package or if I've failed to find a dupe while searching.
In the universal access menu, there should be a toggle that displays the current status.
There's a toggle which shows the current status of stickykeys (ie, stickykeys as a feature, enabled or disabled), but there is no way to find out whether any given key is latched or locked. The gnome 2.x equivalent was a panel applet called AccessX Status Applet. Install that and you'll see what I mean. Icons representing ctrl, alt, shift, super, etc. which change colour depending on that key's state.
As mat brown said in the previous comment, what it is missing right now is a visual feedback of which key is currently enabled. If you take a look to this mail: https://mail.gnome.org/archives/gnome-accessibility-devel/2010-October/msg00009.html It includes a screenshot, and at the top panel, between last.fm icon and user name (infapi00) you can see that applet, with one of them selected.
(In reply to comment #2) > There's a toggle which shows the current status of stickykeys (ie, stickykeys > as a feature, enabled or disabled), but there is no way to find out whether any > given key is latched or locked. ... I don't have any experience of sticky keys. Can you explain why this is required? I would expect someone to know if they have just pressed Alt or Ctrl - am I missing something?
So, there are three states a modifier key can have in sticky keys - normal, latched and locked. Taking a copy operation as an example: Normal is as an untouched key, unpressed, as on a keyboard with no hands on it. Latched is where the key is 'held' for one further keypress, then released. eg: 1. press ctrl, 2. release ctrl, 3. press 'C' - a copy is performed and the keyboard returns to normal. Kind of like how the shift key works on some old typewriters. Locked is where the access key is 'held down' until it's pressed again to unlock it. Less useful (for me) than latching, but has it's place. You cycle through these states with sequential presses of a modifier key - one press of ctrl puts ctrl into latched, two presses into locked, three takes it back to normal. What gets confusing is when you're not sure if you pressed 'c' after pressing ctrl - is ctrl then latched, or not? If you press ctrl again, it's unclear - maybe you've just locked it, or returned it to normal, or latched it again. Assuming you have a load of text selected, what can you do to figure out what state the key is in? In this case, 'c' is OK, but you then have to take your cursor elsewhere and attempt a paste to see if you successfully copied (and bear in mind you might have ctrl locked while you try to alt-tab to another app) It doesn't happen a lot, for an experienced user with fairly limited access problems (like me) but imagine someone who is more disabled, someone perhaps with motor issues who (despite bounce keys) sometimes makes double-presses by accident, possibly even unknowingly. This kind of thing could be a real deal-breaker on being able to use their computer effectively. Hope that's clear. Please let me know if not.
Just for reference, here's how the other guys handle it. Windows does it very badly, with a tiny indicator in the system tray: http://disability.illinois.edu/sites/default/files/win_sticky_keys_icon.png MacOS does a superb job, with nice big icons overlaid onscreen: http://i.imgur.com/YNZPU.png (Apple + Shift in that example) Ubuntu doesn't. https://bugs.launchpad.net/ubuntu/+source/unity/+bug/773078 KDE didn't either, last I looked, but that was a while ago.
Thanks for all the information, Mat - that's incredibly helpful. An OSD seems like a good approach for this, perhaps located in the bottom-right hand corner of the screen.
OSD is definitely the way to go - a panel indicator is good, but OSDs can be larger and easier to see. MacOS's implementation is by far the best I've used. Personally, my preference for location would be top right, but either way. Perhaps it could be configurable - if only through a gsettings option rather than a full-on gui tool. If there's anything else I can do to help, graphical mockups or anything, please do let me know. I have some experience of UI design.
(In reply to comment #8) ... > If there's anything else I can do to help, graphical mockups or anything, > please do let me know. I have some experience of UI design. Sure! Feel free to attach mockups here.
http://imgur.com/a/kMP8y Top to bottom: ctrl key latched, alt latched and ctrl locked, ctrl locked. The way I picture it working is that the OSD position is ranged right, so a single notification just appears, then additional ones push anything onscreen to the right, much like entering text in a word processor in "right align" mode. This operates in a way which is familiar to the right-to-left writers among us, which is (I'm guessing) a majority? Top right is a place where information/notifications already exist in Shell, so eyes are accustomed to things appearing up there. Black text with a heavy white outline to mean locked - colours inverted to mean latched (which is a 'lighter' state for a key to be in) - should be visible on just about any sort of wallpaper or on top of a window. These are set at 50% opacity so they're not too intrusive, but are quite large so as to be easily visible peripherally, rather than requiring concious attention. As they are just plain text, it could be possible to let knowledgeable users adjust font size via gsettings, possibly into a gui at a later date. Although here, to get them to appear the same size and level with each other, the caret for ctrl is considerably more point sizes than the glyph for alt. Just some thoughts, hope they're of use.
(In reply to comment #7) > Thanks for all the information, Mat - that's incredibly helpful. An OSD seems > like a good approach for this, perhaps located in the bottom-right hand corner > of the screen. More info about how it works on OSX, with a little video: http://mactips.info/2010/04/sticky-keys-can-be-very-handy In that case, that OSD can be moved by the user. But I guess that this would mean complicate this feature, and start first with a top-right or bottom-right would be enough.
(In reply to comment #11) > would be enough. Would be enough as a first approach/solution to the problem. That could improve in the future.
(In reply to comment #10) > http://imgur.com/a/kMP8y > > Top to bottom: ctrl key latched, alt latched and ctrl locked, ctrl locked. > > The way I picture it working is that the OSD position is ranged right, so a > single notification just appears, then additional ones push anything onscreen > to the right, much like entering text in a word processor in "right align" > mode. ... Thanks for this Mat. Some comments/suggestions: * I'd use the standard GNOME OSD styling here - a black background with white symbols/characters. (In which case you would need a different visual treatment for the difference between latched and locked.) * You're using the Mac symbols - it'd be nice to see what this looks like with the standard keyboard letters (Alt, Ctrl, etc). * As already stated, my preference would be to have the OSD in the bottom right hand corner, since we tend to have transient notifications at the bottom of the screen. Also, using the top-right could interfere with the system status menus. That said, we do have popups from the message tray in the bottom right - it might be worth thinking about having the sticky keys OSD move out of the way when they are displayed.
All reasonable points, Alan. Just to clarify, the reason I didn't suggest using standard OSD styling was twofold - firstly, the notifications sent by libnotify (or whatever it is these days) are relatively slow to animate in. Sticky keys, for me at least, is a accelerator as much as an access tool, so waiting for that animation - or watching it queue up, could be a problem. I'll have latched and released Ctrl before the standard OSD has fully revealed itself - for this notifier to work well for the most people it needs to be quick and responsive, like the caps-lock led on my keyboard. Whether OSD animation speed is configurable, I don't actually know, so forgive me if my concern is misplaced on this front. The other reason was a bit more subtle - current OSD events, with backgrounds and boxes around them and so on - are designed to get your attention. That is obviously a good thing when you're telling the user something they need to look at. The slightly less-obvious style I used was supposed to be visible enough to give the user the information they need, but without grabbing their concious attention. Using symbols rather than words for the signifiers was also part of this, but it was also how the old Access-X Status applet used to work - however, the symbols used in that aren't in UTF-8 and I didn't want to spend time drawing them out just for mockups! As for positioning - top right, bottom right, same same - in my ideal world that would be a gsettings option anyway, but it's not a big deal, just my personal preference. Anyway, that was my thought process - I wasn't just charging off away from standard Gnomeishness at random :)
Might be worth looking at the proposed wacom osd for inspiration ? I've just seen a screencast of it in action in bug 679062
I think that the video by Willie Walker is very informative to know what the sticky keys are and how they worked in GNOME 2: http://people.gnome.org/~wwalker/demos/keyboard-enhancements.avi
Created attachment 229173 [details] simple mockup Here's a simple mockup of what the sticky keys osd could look like. I'm imagining that we'd just make the text bold when the key is locked.
Works for me.
Allan, so the mockup is shown when the OSD keyboard is present. I assume that that without OSD keyboard, the Stickykeys indications are only shown in the top bar, doesn't it ?
(In reply to comment #19) > Allan, so the mockup is shown when the OSD keyboard is present. OSD keyboard? What's that? Do you mean the OSD stickykeys notification system (that is the only new thing shown at comment 17 mockup)? Or the OSK keyboard? > I assume that > that without OSD keyboard, the Stickykeys indications are only shown in the top > bar, doesn't it ? AFAIU, top bar will not have stickykeys icons. From all the comments, starting from comment 7, the approach for those notifications are the OSD. OSD was not planned as a complementary solution for the top bar icons. OSD is the GNOME3 solution. Top bar (panel) icons was the solution for GNOME2. Anyway, fwiw, this is what I understood from all the comments and last mockup, Allan Day could confirm that.
Sorry, I was biased because I have a strong link betweeen the OSD and keyboard :-) I'm concerned about how the OSD stickykeys notification position can interfere with some applications. If we go this way, I think the user should be able to easily change the position of the notifications in order to avoid the clash. In the pro side of OSD notifications, they are more noticiable than bar top notifications. In the pro side of the top bar notifications approach, they never interfere in the interaction with the applications.
(In reply to comment #21) > Sorry, I was biased because I have a strong link betweeen the OSD and keyboard > :-) > > I'm concerned about how the OSD stickykeys notification position can interfere > with some applications. If we go this way, I think the user should be able to > easily change the position of the notifications in order to avoid the clash. See comment 11. I said exactly that. But in the same way, having a initial implementation with some fixed position can be a good first step. > In the pro side of OSD notifications, they are more noticiable than bar top > notifications. In the pro side of the top bar notifications approach, they > never interfere in the interaction with the applications. See comment 6. A good solution implies having good sized icons (so noticiable). Top bar are somewhat small, more if the icons change depending on which keys you press. And now some of my personal opinion: a) I don't like the idea of implementing the same indicators twice. Making just one implementation would mean a non-trivial amount of work than someone would need to do. Having the same indicator twice would mean the double of future maintenance. b) IMHO (disclaimer, not a designer), adding more icons to the top panel breaks some of the coherence. And would make that a mess if the user want to add extensions that add more icons. Current proposal is cleaner. c) As we found a good solution for the problem, I don't see the need of a second solution as a fallback for hypothetical problems..
To address the concerns that it might clash, there's several things we can do: - make the OSD transparent for input - ie clicks would go to the underlying window - make the osd jump to the other corner if the mouse goes over it In any case, I agree that having a simple implementation of this would be a great starting point.
(In reply to comment #23) > To address the concerns that it might clash, there's several things we can do: > > - make the OSD transparent for input - ie clicks would go to the underlying > window That's what I had in mind (and it is how our other OSDs already work). We could also try fading out the sticky keys OSD if the pointer is over it, but I would prefer to try an implementation that is consistent with the existing OSDs first.
Created attachment 241477 [details] [review] Add an OSD for sticky modifiers This commit adds an OSD that displays which modifiers are currently latched or locked. This is commonly used together with sticky keys.
Created attachment 241478 [details] screenshot
Here is a quick sketch. This just shows the currently latched and locked modifiers. It doesn't do anything fancy like jump or fade on mouseover. It could be extended to show state of mouse keys, bounce keys, etc, like the accessx applet in GNOME2 did. We probably also want to make this opt-in and only display it if sticky keys are enabled.
From a quick look at the patch: - sticky keys are handled by g-s-d's keyboard-a11y plugin, no? Wouldn't it make sense to handle the OSD there (via DBus, like media-keys does) - if we want to handle the OSD completely in the shell, can we at least reuse Main.osdWindow instead of reimplementing it?
Two thoughts here: - The ShowOSD api is not quite up to handling this. We need custom positioning and markup to make this work. - In the Wayland future, input is all handled in the compositor anyway, so might as well make a start here...
Another consideration: if the on-screen keyboard is enabled, we don't need this indicator; we should indicate latched/locked modifiers in the keyboard, in that case.
Created attachment 244692 [details] [review] Add an OSD for sticky modifiers This commit adds an OSD that displays which modifiers are currently latched or locked. This is commonly used together with sticky keys.
> - if we want to handle the OSD completely in the shell, can > we at least reuse Main.osdWindow instead of reimplementing it? I've tried, but the two popups are different in multiple aspects: - center-of-screen vs lower-right alignment - big with square aspect ratio vs small and non-square - require icon vs just text - plain text vs markup - auto-hide vs stays on screen
I mentioned this on IRC, putting it here so it doesn't get lost. It looks like the OSD might cover the input field that's currently in use. Is it better not to know the state of the stuck/latched keys, or to have your input (possibly) hidden when using them? Would this patch need focus tracking to avoid the latter case?
Given the mess I've got into with not knowing which state keys are in (and how hard it is to recover from), I'd rather have my input hidden than not know what StickyKeys was doing. I can always move a window out from under the OSD if needed.
Also see bug 722437
*** Bug 756902 has been marked as a duplicate of this bug. ***
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a new ticket at https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/ Thank you for your understanding and your help.