GNOME Bugzilla – Bug 462156
No announcements for ARIA trees
Last modified: 2008-07-22 19:32:19 UTC
This ARIA tree http://archive.dojotoolkit.org/nightly/dojotoolkit/dijit/tests/test_Tree.html has good ARIA markup and at-spi events. As such, it can be considered a good test case for implementing ARIA tree support in Orca. Currently, Orca is not outputting any information for this widget. The widget consists of a tree object (which never receives focus) with list items for children. Dynamic 'panel's appear during expansion events containing a series of list items for the new branch of the tree. We receive object:children-changed:add events during expansion and object:children-changed:remove events during collapse. Currently, Orca does not implement speech/Braille output for list items. I added _getSpeechForListItem() in default.py and was able to get output for the list items of the tree. Event handlers will have to be added to announce collapse/expansion events. A couple questions: 1) Is there a good GTK+ tree example that someone could point out? 2) Should we leverage the 'table' code as has been done for other trees?
Created attachment 92810 [details] [review] first version of No announcements for ARIA trees Patch includes work done from bug #459584 , ARIA widget labeling. Work done on this problem includes: * addition and integration of _getSpeechForListItem into speechgenerator * addition of default:onStateChanged() code to handle tree expanding/collapsing Test cases: * tree examples at http://developer.mozilla.org/en/docs/Accessible_DHTML#Supported_roles * http://archive.dojotoolkit.org/nightly/dojotoolkit/dijit/tests/test_Tree.html
Created attachment 92813 [details] [review] second version of No announcements for ARIA trees Up to date version created after ARIA labeling fix was committed.
> Created an attachment (id=92813) [edit] > second version of No announcements for ARIA trees I'm confused about the speechgenerator mods -- they merely add a method that defers to the default speech generator. I believe the speechgenerator.py:getSpeech() method will do this automatically if a generator for a specific role is not defined. If there are plans to provide custom behavior for list_item, however (read below), then it makes sense to keep this. For the custom behavior for list_item, I think the default.py mods might be better off in speechgenerator.py:_getSpeechForListItem. :-) Take a look at how the expanded/collapsed stuff is handled in speechgenerator.py:_getSpeechForTableCell for an idea of how it is done there. Make sure you also consider similar behavior for braille. In addition, if we find that there is just too much duplicated code to handle this, it might make sense to try to provide a utility to obtain a string (or something) regarding the expanded/collapsed state. Hope this makes sense.
(In reply to comment #3) > > Created an attachment (id=92813) [edit] > > second version of No announcements for ARIA trees > > I'm confused about the speechgenerator mods -- they merely add a method that > defers to the default speech generator. I believe the > speechgenerator.py:getSpeech() method will do this automatically if a generator > for a specific role is not defined. A speechgenerator is found in speechgenerator.py:getSpeech(). The problem is the generator that is found is gecko:_getSpeechForListItem(). This method in turn calls speechgenerator:_getSpeechForListItem() for ARIA widgets which would not be there without the patch. We need getSpeech() to find the speechgenerator:_getSpeechForListItem() in the first place. > For the custom behavior for list_item, I think the default.py mods might be > better off in speechgenerator.py:_getSpeechForListItem. :-) Take a look at > how the expanded/collapsed stuff is handled in > speechgenerator.py:_getSpeechForTableCell for an idea of how it is done there. I don't think we will be able to do this because the required states are not present. Mozilla bug??? Also, getSpeechForListItem() is not called when an expansion/collapse event occurs because there is not change in focus or caret location. > Make sure you also consider similar behavior for braille. > Yea, I'll look into this. Any pointers here would be great.
> the generator that is found is gecko:_getSpeechForListItem(). > This method in turn calls speechgenerator:_getSpeechForListItem() > for ARIA widgets Does it? Looking at it, I would think that it would instead call _getDefaultSpeech() if the list item is not focusable and otherwise call _getSpeechForObjectName(). If I'm wrong, my apologies! And ignore the next bit. :-) Otherwise.... Don't the ARIA widgets have STATE_FOCUSABLE? If so, couldn't you add some handling within gecko._getSpeechForListItem()?
(In reply to comment #5) > > the generator that is found is gecko:_getSpeechForListItem(). > > This method in turn calls speechgenerator:_getSpeechForListItem() > > for ARIA widgets > > Does it? Looking at it, I would think that it would instead call > _getDefaultSpeech() if the list item is not focusable and otherwise call > _getSpeechForObjectName(). If I'm wrong, my apologies! And ignore the next > bit. :-) Otherwise.... > Are you up to date? You may be missing the following in gecko._getSpeechForListItem() if self._script.isAriaWidget(obj): sg = speechgenerator.SpeechGenerator return sg._getSpeechForListItem(self, obj, already_focused)
I guess I'm not up to date. :-) Thanks! Why are we doing that?
(In reply to comment #7) > I guess I'm not up to date. :-) Thanks! > > Why are we doing that? We want to treat ARIA widgets like GTK+ widgets, so we defer to the default speech generator. Stupid me didn't check to see if the default speech generator had a method for list items. D'Oh! Nice catch, Scott. > > For the custom behavior for list_item, I think the default.py mods might be > > better off in speechgenerator.py:_getSpeechForListItem. :-) Take a look at > > how the expanded/collapsed stuff is handled in > > speechgenerator.py:_getSpeechForTableCell for an idea of how it is done there. > > I don't think we will be able to do this because the required states are not > present. Mozilla bug??? Also, getSpeechForListItem() is not called when an > expansion/collapse event occurs because there is not change in focus or caret > location. If something thinks it's expandable and wants to throw object:state-changed:expanded events, it sure would make sense that it has the EXPANDABLE state. Not having this sounds like a bug to me. The state change handling code is managed in default.py:onStateChanged. That code is a little old and krufty. There's a table at the bottom of default.py that says which state changed events we care about for which roles. It might make sense to add the following line to the end of the file (remembering to make sure the file ends in a new line so it doesn't wreak havoc on other things ;-) ;-)): state_change_notifiers[rolenames.ROLE_LIST_ITEM] = ("expanded", None) With this, when a object:state-changed:expanded event is received for a list item, onVisualAppearanceChanged will be called, which will then ultimately call getSpeechForListItem. This will then get the appropriate amount of information to speak (e.g., just "expanded" or "collapsed" if the list item already had focus).
Created attachment 92886 [details] [review] third version of No announcements for ARIA trees This patch follows the guidance Will laid out in the previous comment. Also included was code to short circuit gecko.sayCharacter() when isAriaWidget() == True. Your thoughts on this?
(In reply to comment #9) > Created an attachment (id=92886) [edit] > third version of No announcements for ARIA trees > > This patch follows the guidance Will laid out in the previous comment. Also > included was code to short circuit gecko.sayCharacter() when isAriaWidget() == > True. Your thoughts on this? > I think the better thing to do here might be as follows. This adds the expanded/collapsed state when someone first navigates to the item, and it also restricts announcement of expanded/collapsed to items that are EXPANDABLE. This, of course, means that the ARIA support needs to include the EXPANDABLE state: + if not already_focused: + utterances = self._getDefaultSpeech(obj, already_focused) + else: + utterances = [] + + if obj.state.count(atspi.Accessibility.STATE_EXPANDABLE): + if obj.state.count(atspi.Accessibility.STATE_EXPANDED): + # Translators: this represents the state of a node in a tree. + # 'expanded' means the children are showing. + # 'collapsed' means the children are not showing. + # + utterances.append(_("expanded")) + else: + # Translators: this represents the state of a node in a tree. + # 'expanded' means the children are showing. + # 'collapsed' means the children are not showing. + # + utterances.append(_("collapsed")) The ARIA widget's choice to grab left/right arrow is very unfortunate since it conflicts with caret navigation. I'm not quite sure what to do there, but this seems like a design choice gone bad. :-( Is it possible to request that they use something else, such as + and -?
The expandable state bug has been filed here https://bugzilla.mozilla.org/show_bug.cgi?id=390629 > Is it possible to request that they use something else, such as + and -? I doubt it. These key bindings are defined by the web developer in Javascript and are thus out of our control.
As development is still in progress here I'm unclear what input is needed from me at this point.
> > Is it possible to request that they use something else, such as + and -? > > I doubt it. These key bindings are defined by the web developer in Javascript > and are thus out of our control. In yesterday's distributed Orca team meeting, we agreed that Scott would ask the Dojo developers to consider something such as +/- because the left/right arrow keys are used for caret navigation and text selection. Thanks Scott!
The Dojo developers have agreed to *ADD* '+' and '-' for tree expansion/collapsing. The Dojo ticket can be seen here http://trac.dojotoolkit.org/ticket/4065
A demonstration of the requested key binding change can be seen here http://david.atrc.utoronto.ca/dijit/tests/test_Tree.html . The change has not yet made it into the dojo code base.
(In reply to comment #15) > A demonstration of the requested key binding change can be seen here > http://david.atrc.utoronto.ca/dijit/tests/test_Tree.html . The change has not > yet made it into the dojo code base. Woo woo! I notice left/right arrows are still in there. Is that the intent?
> I notice left/right arrows are still in there. Is that the intent? Yes, is that satisfactory?
(In reply to comment #17) > > I notice left/right arrows are still in there. Is that the intent? > Yes, is that satisfactory? > I'm not sure. What is the behavior/interaction with caret navigation? Does a single arrow press result in multiple operations happening at the same time (e.g., open a list item and move the caret)? If so, that doesn't seem like desired behavior. :-(
Dojo trees are not the only trees that use the arrows for expansion/collapsing. All tree linked from http://developer.mozilla.org/en/docs/Accessible_DHTML#Supported_roles use the arrows in this manner. The Mozilla guys fixed the EXPANDABLE bug https://bugzilla.mozilla.org/show_bug.cgi?id=390629 . However, the Dojo guys still have a problem here, see http://trac.dojotoolkit.org/ticket/3971 . Use the Mozilla tree examples for testing. I will now get a final patch together for review.
> Dojo trees are not the only trees that use the arrows for expansion/collapsing. > All tree linked from > http://developer.mozilla.org/en/docs/Accessible_DHTML#Supported_roles use the > arrows in this manner. I think we should extend the request to the Dojo team to use +/- for anything that does expanding/collapsing. The team seems accommodating, which is great. :-) IMO, the keyboard gestures should be consistent across the various widgets and they should not conflict with gestures used for other common operations (e.g., caret navigation and text selection). In addition, Eitan also noted that the use of arrow keys can be confusing for people from right-to-left locales. Back to my question, though -- What is the behavior/interaction with caret navigation when both +/- and the arrow keys are used for expanding/collapsing? Does a single arrow press result in multiple operations happening at the same time (e.g., open a list item and move the caret)? If so, that doesn't seem like desirable behavior. :-(
Created attachment 93367 [details] [review] fourth version of no announcements for ARIA trees Patch includes check for state EXPANDABLE as outlined in Will's earlier comment. Testing should not include the Dojo tree for now. They have a known issue as outlined in the dojo ticket linked above. Also of note is that the Mozilla example trees do not have the 'sayCharacter' issue that the Dojo tree has during expansion/collapsing. This is due to the Dojo tree triggering a caret-moved event while the Mozilla trees do not. Suggestions?
(In reply to comment #20) > > Dojo trees are not the only trees that use the arrows for expansion/collapsing. > > All tree linked from > > http://developer.mozilla.org/en/docs/Accessible_DHTML#Supported_roles use the > > arrows in this manner. > > I think we should extend the request to the Dojo team to use +/- for anything > that does expanding/collapsing. I will certainly ask. > Back to my question, though -- What is the behavior/interaction with caret > navigation when both +/- and the arrow keys are used for expanding/collapsing? > Does a single arrow press result in multiple operations happening at the same > time (e.g., open a list item and move the caret)? If so, that doesn't seem > like desirable behavior. :-( > Yea, both operations happen, only for the Dojo widgets as mentioned in my previous comment.
(In reply to comment #21) > Created an attachment (id=93367) [edit] > fourth version of no announcements for ARIA trees > > Patch includes check for state EXPANDABLE as outlined in Will's earlier > comment. > > Testing should not include the Dojo tree for now. They have a known issue as > outlined in the dojo ticket linked above. This looks much better! Thanks! The patch now needs something similar for braille. > Also of note is that the Mozilla example trees do not have the 'sayCharacter' > issue that the Dojo tree has during expansion/collapsing. This is due to the > Dojo tree triggering a caret-moved event while the Mozilla trees do not. > Suggestions? We've been having some mid-air collisions on this, so I just want to be sure we're in sync. The main problem is the result of the UI designer's choice to use arrow keys for expanding and collapsing the nodes, right?
(In reply to comment #23) > The patch now needs something similar for braille. OK, I'll look into this. > > > Also of note is that the Mozilla example trees do not have the 'sayCharacter' > > issue that the Dojo tree has during expansion/collapsing. This is due to the > > Dojo tree triggering a caret-moved event while the Mozilla trees do not. > > Suggestions? > > We've been having some mid-air collisions on this, so I just want to be sure > we're in sync. The main problem is the result of the UI designer's choice to > use arrow keys for expanding and collapsing the nodes, right? > The dojo widgets are triggering different events from the Mozilla examples, in particular a caret-moved event. My question to you was why? What have the web developers done differently to cause or prevent this? The caret-moved event translates into a call to default._presentTextAtNewCaretPosition() which in turn calls gecko:sayCharacter(). Is there a way we can get the best of both worlds by identifying that it is an ARIA widget and not announcing the character or are your goals more lofty here, namely to set good markup standards?
> The dojo widgets are triggering different events from the Mozilla examples, in > particular a caret-moved event. My question to you was why? What have the web > developers done differently to cause or prevent this? /me shrugs shoulders -- all toolkit developers can create completely different user experiences if they want to. We can work to accommodate these differences, but we also need to make some simplifying assumptions that the toolkit developers can be influenced into doing the right thing. > The caret-moved event > translates into a call to default._presentTextAtNewCaretPosition() which in > turn calls gecko:sayCharacter(). It looks at the last keyboard event to infer why the caret moved and then presents the character, word, line, selection, etc. > Is there a way we can get the best of both worlds by identifying that it is an > ARIA widget and not announcing the character or are your goals more lofty here, > namely to set good markup standards? I'd really like to set good examples for good accessible design, especially in toolkits like Dojo.
Created attachment 93399 [details] [review] fifth version fo no announcements for ARIA trees Patch includes Braille output for trees (implemented as list items). The best working examples are the two UIUC tree examples. They do not seem to have any issues. The dojo examples suffers from their EXPANDABLE bug as well as the caret-moved event as commented earlier. The Mozilla examples do not always update the collapsed/expanded status and often trigger caret-moved events. > I'd really like to set good examples for good accessible design. Agreed. I'll contact Aaron about their tree design. However, we still may need to protect ourselves from 'rouge' caret-moved events generated by ARIA widgets in general.
(In reply to comment #26) > Created an attachment (id=93399) [edit] > fifth version fo no announcements for ARIA trees > > Patch includes Braille output for trees (implemented as list items). Looks close, but I'm not sure the "collapsed"/"expanded" strings end up in the right spot for braille. Mike, can you describe where they should be on the braille display with relation to label/text/role/etc? > > I'd really like to set good examples for good accessible design. > > Agreed. I'll contact Aaron about their tree design. However, we still may > need to protect ourselves from 'rouge' caret-moved events generated by ARIA > widgets in general. I agree. Poor design choices by toolkit and application developers are the bane of our existence. I think a11y has enough of a bite these days, however, that we can actually make demands and be heard.
What you want to do here is basically follow the example for naudilus. Set your view to the list view and notice what you get when you arrow on to a folder. You want to present the item with focus followed by the state followed by the number of items followed by the role. Hope this helps.
That should be an easy fix. I will add the level information to speech output as well, however, level information for Mozilla is currently broken. It always reports level information as 1. See https://bugzilla.mozilla.org/show_bug.cgi?id=391671 .
Created attachment 93444 [details] [review] sixth version of no announcements for ARIA trees This patch includes the proper ordering of roles, states, etc. for both Braille and speech. I am concerned that the role and level information should not be placed in _getSpeechForListItem() because this information is not in _getSpeechForTableCell(). Instead, the role and level information comes from a locusOfFocusChanged() call for table cells. This will not work for ARIA trees because a locusOfFocusChanged() is not triggered when the user navigates these trees.
(In reply to comment #30) > Created an attachment (id=93444) [edit] > sixth version of no announcements for ARIA trees Getting better! More comments: + # Translators: this is the role name for this object + # + utterances.append(_('list item')) You should use rolenames.getSpeechForRoleName when possible. In addition, speechgenerator.py:_getDefaultSpeech (called earlier in the new _getSpeechForListItem method you're adding with this patch) already includes the role, so I don't think the chunk of code above is actually necessary. Do you hear the rolename being spoken twice when you test this? > I am concerned that the role and level information should not be placed in > _getSpeechForListItem() because this information is not in > _getSpeechForTableCell(). Instead, the role and level information comes from a > locusOfFocusChanged() call for table cells. The node level/depth information is viewed as a contextual thing -- we only announce it when it changes. Thus, if you navigate from one node to another, but the level/depth has not changed, the depth/level will not be presented. > This will not work for ARIA trees > because a locusOfFocusChanged() is not triggered when the user navigates these > trees. What AT-SPI events are emitted when the user arrows up/down these trees? Is it possible to queue off them to call orca.setLocusOfFocus?
(In reply to comment #31) > > + # Translators: this is the role name for this object > + # > + utterances.append(_('list item')) > > You should use rolenames.getSpeechForRoleName when possible. Yes, I will change. > In addition, > speechgenerator.py:_getDefaultSpeech (called earlier in the new > _getSpeechForListItem method you're adding with this patch) already includes > the role, so I don't think the chunk of code above is actually necessary. Do > you hear the rolename being spoken twice when you test this? No, the call to gecko._getSpeechForObjectRole() filters out list items. Do we even want to hear 'list item' at all then? > > > This will not work for ARIA trees > > because a locusOfFocusChanged() is not triggered when the user navigates these > > trees. > > What AT-SPI events are emitted when the user arrows up/down these trees? Is it > possible to queue off them to call orca.setLocusOfFocus? > All trees that I have inspected trigger caret-moved and state-change:focus events. Some give true focus events. Should I try the same strategy that I used on an earlier bug, namely using state_changed_notifier[rolenames.ROLE_LIST_ITEM] = ("focused", None) or something similar?
> > You should use rolenames.getSpeechForRoleName when possible. > > Yes, I will change. Thanks! > > speechgenerator.py:_getDefaultSpeech (called earlier in the new > > _getSpeechForListItem method you're adding with this patch) already includes > > the role, so I don't think the chunk of code above is actually necessary. Do > > you hear the rolename being spoken twice when you test this? > > No, the call to gecko._getSpeechForObjectRole() filters out list items. Do we > even want to hear 'list item' at all then? Aha. Sneaky little method. I think we made a decision at one point to eliminate 'list item' when we were looking at HTML content. ARIA objects are different beast, though, and we probably should treat them like GTK+ widgets. Mike, what is your take on whether 'list item' should be spoken or not? Can you try the proposed patch against the test URL and see what you think? Patch: http://bugzilla.gnome.org/attachment.cgi?id=93444 Test: http://archive.dojotoolkit.org/nightly/dojotoolkit/dijit/tests/test_Tree.html > > What AT-SPI events are emitted when the user arrows up/down these trees? Is it > > possible to queue off them to call orca.setLocusOfFocus? > > > > All trees that I have inspected trigger caret-moved and state-change:focus > events. Some give true focus events. Yech - so they always issue state-changed:focused events but not focus: events? If so, that's a bug. Both should be emitted whenever focus changes. > Should I try the same strategy that I > used on an earlier bug, namely using > state_changed_notifier[rolenames.ROLE_LIST_ITEM] = ("focused", None) or > something similar? There's some code in default.py:onStateChanged that handles the object:state-changed:focused events. It should hopefully be getting called, but maybe the decision criteria for calling onFocus (i.e., event.detail1 != None) is not being met.
This does seem to improve things. How about committing the patch but lets leave the bug pending until we can confirm the fix to the existing Dojo bug.
Committed to repository. Bug left open pending Dojo widget fix.
The Dojo guys fixed the tree item EXPANDABLE issue and Orca is now performing well of their tree as well.