GNOME Bugzilla – Bug 648259
Document the use of an object attribute to expose additional role/purpose information
Last modified: 2021-06-10 11:27:45 UTC
We currently have ATK_ROLE which corresponds quite closely to widget type; we lack a means by which to express the purpose of a widget.
ATK_ROLE_LABEL might be for:
* Static text ('Fill out this form')
* The main/proper label for a widget ('Account number')
* Descriptive text for a widget ('Example: 123456789')
* Error text for a widget ('Account number could not be found')
* A notification
* A ticker/marquee
* A clock or timer
* Something else
ATK_ROLE_TEXT might be for:
* A document
* A place to type chat messages
* A place to display chat history
* A place to display installation progress
* A place holder on a presentation slide
* A place to specify a search string
* Something else
ATK_ROLE_TREE might be for:
* A list of mail messages
* A buddy list
* A list of search results in a database
* Something else
And so on, and so forth.
When a screen reader like Orca is attempting to decide what to present to the user, it uses a combination of heuristics and scripts for known applications:
* When text gets inserted into a non-editable, multi-line object of ROLE_TEXT in Pidgin, odds are that the chat history got updated and we should speak the incoming message.
* When an item in a tree in Evolution gets added, odds are that a new mail message arrived -- unless focus is in an editable object of ROLE_TEXT which is contained in a toolbar, and then it could be that the user did a search.
As should be obvious from the above, this assumes knowledge of the app, the presence of a custom script for that app, and a darned good chance for "guessing wrong." :-/
A user (and an AT like Orca) is far more interested in why something is, rather than what something is. Therefore, it would be extremely cool if ATK could give us hints about the purpose of a widget. Then a screen reader like Orca could do something like, "if it's a focused chat-history widget, speak inserted text" without ever having to know what the app is or guess what the widget might be for.
Please and thank you. :-)
BTW: While ARIA roles might not cover all that we need or want, they do seem to include quite a bit more purpose-y information. Perhaps ARIA should be looked to for more than just "web apps." (??)
ATK Hackfest (workgroup) conclusions:
* Using ARIA roles as a purpose hint makes sense.
* We can follow the lead of Gecko, which already provides this information to us in the form of object attributes (xml-roles).
* Need to document this as best practice, etc.
> BTW: While ARIA roles might not cover all that we need or want, they do seem to
> include quite a bit more purpose-y information. Perhaps ARIA should be looked
> to for more than just "web apps." (??)
Along these lines, there is a movement that a role based approach is lacking,
and that an affordance approach might be better. An affordance approach
focuses on "-able"s -- a widget is scrollable, editable, scalable, and so on.
This is more along the lines of Microsoft's UI Automation where control
patterns (invoke, toggle, etc.) can be combined to describe behaviour.
But, I'm not sure that even this is high level enough for things like: "When an
item in a tree in Evolution gets added, odds are that a new mail message
arrived -- unless focus is in an editable object of ROLE_TEXT which is
contained in a toolbar, and then it could be that the user did a search."
(In reply to comment #2)
> Along these lines, there is a movement that a role based approach is lacking,
> and that an affordance approach might be better. An affordance approach
> focuses on "-able"s -- a widget is scrollable, editable, scalable, and so on.
We get this sort of information from the ATK_STATE already. Mind you, some states might be missing. In which case, other bugs could/should be opened.
Having said that, whilst a role-based approach may be lacking in some ways, it can still be quite useful. Consider, for instance, object:selection-changed events in a non-focused widget. Under most circumstances, we have no idea why the signal is being emitted and we don't care: It is (likely) irrelevant to the user -- UNLESS the user is performing a search. In that case, if the user is blind, those selection-changed events are EXTREMELY relevant because they are an indication of the real-time search results and something their screen reader should provide easy access to. But how is the screen reader to identify if the user is performing a search or not?
Thus being told that focused entry's purpose or role is 'search' is super useful. In contrast, knowing that this entry happens to be editable simply tells us that text *can* be inserted or removed; not why. Likewise, that the object in which the selection is changing happens to be scrollable (or not) doesn't communicate what the thing is; Just that it has the potential to contain more stuff than can fit within its visible area.
Maybe one day there shall be a better way. Today we need the role/purpose.
[Mass-reassigning open atk bug reports for better trackability as requested in https://bugzilla.gnome.org/show_bug.cgi?id=653179 .
If you have watched the previous assignee of this bug report as a workaround for actually getting notified of changes in atk bugs, you yourself will now have to add email@example.com to your watchlist at the bottom of https://bugzilla.gnome.org/userprefs.cgi?tab=email to keep watching atk bug reports in GNOME Bugzilla.
Sorry for the noise: Feel free to filter for this comment in order to mass-delete the triggered bugmail.]
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org.
As part of that, we are mass-closing older open tickets in bugzilla.gnome.org
which have not seen updates for a longer time (resources are unfortunately
quite limited so not every ticket can get handled).
If you can still reproduce the situation described in this ticket in a recent
and supported software version of atk, then please follow
and create a ticket at
Thank you for your understanding and your help.