GNOME Bugzilla – Bug 486972
Expanding/collapsing tree nodes in a table should not speak nodename again
Last modified: 2008-07-22 19:32:43 UTC
Steps to reproduce: 1) Run the Tree View -> Tree Store demo 2) Select the "January" cell 3) Press Shift+Left and Shift+Right to collapse/expand the node. 4) Because "January" already has focus, Orca should only present "collapsed" or "expanded" and not "January collapsed" or "January expanded".
See also test/keystrokes/gtk-demo/role_tree_table.py
Will, just started looking at this one. In speechgenerator.py, in the __init__ method, if the role is ROLE_TREE_TABLE, it's calling self._getSpeechForTable() not self._getSpeechForTreeTable(). Is that correct? (It's the same for Orca in the gnome-2-20 branch). In fact, it looks like _getSpeechForTreeTable() currently isn't being used at all. In fact, I'm not hearing "expanded..." or "collapsed ..." at all at the moment. Looking at the source code, "expanded/collapse" state is only uttered in _getSpeechForListItem() and _getSpeechForTableCell(), so I'm a little confused.
> In speechgenerator.py, in the __init__ method, if the role is > ROLE_TREE_TABLE, it's calling self._getSpeechForTable() not > self._getSpeechForTreeTable(). > > Is that correct? (It's the same for Orca in the gnome-2-20 branch). It's probably OK. I have a feeling neither method gets called in this case since we're most likely dealing with a cell and not the {tree}table. There are probably some dead code paths that can be trimmed from the generators at some point. > In fact, I'm not hearing "expanded..." or "collapsed ..." at all > at the moment. Hmm...there might be another bug lurking. Try arrowing down to February and back up to January. Then try expanding/collapsing January. You should hear the expanded/collapsed presentation. I just tried it and that's what happened. > Looking at the source code, "expanded/collapse" > state is only uttered in _getSpeechForListItem() and > _getSpeechForTableCell(), so I'm a little confused. I think we're dealing with a cell here, I'd suspect the method getting called is _getSpeechForTableCell. Hope this helps.
Created attachment 98143 [details] [review] Patch to hopefully fix the problem. Before I adjust the associated test, is this what we want here? Note that it still says "n items" after the "expanded/collapsed". Should that too be suppressed if we are already focused on this item?
Note also that if we are initially coming from the column headers down to the "January" list item, we do not get a "object:active-descendant-changed" event, so can't speak the collapsed/expanded state. Is there a separate bug that we should be filling because of this?
> Before I adjust the associated test, is this what we want here? > Note that it still says "n items" after the "expanded/collapsed". > Should that too be suppressed if we are already focused on this > item? I'm not quite sure what to do here. I'd guess that on expanded, it would be nice to present "expanded" and the number of items, but on collapsed we should just present "collapsed". Mike would be the best person to ask about this. > Note also that if we are initially coming from the column > headers down to the "January" list item, we do not get a > "object:active-descendant-changed" event, so can't speak > the collapsed/expanded state. > > Is there a separate bug that we should be filling because > of this? Yeah - the UI seems very strange. When I first down arrow the first time the dialog appears, January gets a dotted box around it, but it is not highlighted. I can expand/collapse it just fine, but it is not selected. When I down arrow a second time, focus stays on January, but January is now selected. I've had to recently reinstall my machine and have not yet set up accessibility for my login shell yet. So, I'm not able to look at what events we're getting in the various steps above. Kind of relevant events are you seeing? See also bug 486895. I wonder if it might be related to the lack of events we're getting?
The new item count should be spoken if a node is expanded but not if it is closed.
Created attachment 98590 [details] Orca debug output generated whilst testing this problem. Mike wrote: > The new item count should be spoken if a node is expanded but not if it is > closed. Okay. That's what the current patch is doing. Is it ready for being committed? Will wrote: > I've had to recently reinstall my machine and have not yet set up > accessibility for my login shell yet. So, I'm not able to look at > what events we're getting in the various steps above. Kind of > relevant events are you seeing? See attached Orca debug for the full output. But here is the difference in a nutshell. From column headers Down to January, we get: ---------> QUEUEING EVENT object:state-changed:focused ---------> QUEUEING EVENT object:state-changed:focused ---------> QUEUEING EVENT focus: From the last January sub-entry Down to February, we get: ---------> QUEUEING EVENT object:state-changed:selected ---------> QUEUEING EVENT object:selection-changed ---------> QUEUEING EVENT object:active-descendant-changed It's the "object:active-descendant-changed" event that's triggering the speaking of "expanded/collapsed etc...", and we aren't seeing this initially. Should I be filing a new bug on this? Does it seem like a Gtk+ bug to you? And because of this, it's not possible to "fixup" the role_tree_table.py script at the moment.
Created attachment 98591 [details] Orca debug output generated whilst testing this problem. Mike wrote: > The new item count should be spoken if a node is expanded but not if it is > closed. Okay. That's what the current patch is doing. Is it ready for being committed? Will wrote: > I've had to recently reinstall my machine and have not yet set up > accessibility for my login shell yet. So, I'm not able to look at > what events we're getting in the various steps above. Kind of > relevant events are you seeing? See attached Orca debug for the full output. But here is the difference in a nutshell. From column headers Down to January, we get: ---------> QUEUEING EVENT object:state-changed:focused ---------> QUEUEING EVENT object:state-changed:focused ---------> QUEUEING EVENT focus: From the last January sub-entry Down to February, we get: ---------> QUEUEING EVENT object:state-changed:selected ---------> QUEUEING EVENT object:selection-changed ---------> QUEUEING EVENT object:active-descendant-changed It's the "object:active-descendant-changed" event that's triggering the speaking of "expanded/collapsed etc...", and we aren't seeing this initially. Should I be filing a new bug on this? Does it seem like a Gtk+ bug to you? And because of this, it's not possible to "fixup" the role_tree_table.py script at the moment.
Sorry about attaching the last one twice. You can blame in on Bugzilla being truly unresponsive.
I think this one is good to commit.
This one now seems to work fine.
Thanks Mike. Can't close it yet. Needs a decision on whether we need to open a new bug (see comment #9).
(In reply to comment #8) > Created an attachment (id=98590) [edit] > Orca debug output generated whilst testing this problem. Thanks! To add more detail to what is going on: 1) The first Down arrow from the header draws a dotted line around an unhighlighted 'January' row. Here's more detail on the events seen in comment #8: A) First a loss of focus on the column header: OBJECT EVENT: object:state-changed:focused detail=(0,0) app.name='gtk-demo' name='Holiday' role='table column header' state='enabled focusable selectable sensitive showing visible' relations='' B) Then focus going to the table (object:state-changed:focused and focus: events are redundant and expected): OBJECT EVENT: object:state-changed:focused detail=(1,0) app.name='gtk-demo' name='None' role='tree table' state='enabled focusable focused sensitive showing visible manages descendants' relations='' OBJECT EVENT: focus: detail=(0,0) app.name='gtk-demo' name='None' role='tree table' state='enabled focusable focused sensitive showing visible manages descendants' relations='' At this point, orca speaks and brailles 'tree table' with no information about being on the 'January' cell. Without any other information being given to us (e.g., focus on a table cell that we might need -- see #2 below), this is probably accurate. This seems, however, like it might be a gail or gtk+ bug -- somewhere between here and #2, the table cell becomes focused instead of the table and we never get an event for it. Can you please file a bug on that one? I'd say start with gail and let Li help us with the triage. 2) Pressing Shift+Left to collapse the "January" node when it is unhighlighted, but with the dotted rectangle around it yields the following event: OBJECT EVENT: object:state-changed:expanded detail=(0,0) app.name='gtk-demo' name='January' role='table cell' state='enabled expandable focusable focused selectable sensitive showing single line transient visible' relations='node child of' Orca doesn't present anything here. I wonder if some Orca magic should be done to check to see if the FOCUSED state is set and force the locus of focus state to this object? 3) Pressing Shift+Right to re-expand the "January" node gives a similar event as above, but with detail1 == 1 and some additional state info: OBJECT EVENT: object:state-changed:expanded detail=(1,0) app.name='gtk-demo' name='January' role='table cell' state='enabled expandable expanded focusable focused selectable sensitive showing single line transient visible' relations='node child of' 4) Pressing Down again highlights the "January" cell (its background goes from white to dark blue on my machine). The following events are delivered, but Orca doesn't present anything: A) The January cell is selected: OBJECT EVENT: object:state-changed:selected detail=(1,0) app.name='gtk-demo' name='January' role='table cell' state='enabled expandable expanded focusable focused selectable selected sensitive showing single line transient visible' relations='node child of' B) Same as A, but from the point of view of the table: OBJECT EVENT: object:selection-changed detail=(0,0) app.name='gtk-demo' name='None' role='tree table' state='enabled focusable focused sensitive showing visible manages descendants' relations='' The fact that Orca doesn't present anything here is probably a separate Orca bug. Can you look into this more and file one if needed? 5) Pressing Down again highlights the "New Years Day" line and draws a dotted rectangle around it. The following events are generated: A) Selection is lost on the January cell: OBJECT EVENT: object:state-changed:selected detail=(0,0) app.name='gtk-demo' name='January' role='table cell' state='enabled expandable expanded focusable focused selectable sensitive showing single line transient visible' relations='node child of' B) Same as A, but from the point of view of the table: OBJECT EVENT: object:selection-changed detail=(0,0) app.name='gtk-demo' name='None' role='tree table' state='enabled focusable focused sensitive showing visible manages descendants' relations='' C) The table's active descendant changes to presumably the New Years Day cell. Orca speaks/brailles as a result of this event: OBJECT EVENT: object:active-descendant-changed detail=(78,0) app.name='gtk-demo' name='None' role='tree table' state='enabled focusable focused sensitive showing visible manages descendants' relations='' Anyhow, this all seems to be a hodgepodge of several bugs. I'm not sure the tree table is handling focus and manages descendants correctly. In this case, I believe the tree table probably should only be the thing with focus and the selected cell should be handled via active descendant logic and not focus logic. We probably need help from Li here. After all this, phew, I think maybe this specific bug is fixed, but one or more need to be opened to reflect all the strange behavior we are seeing.
We discussed this bug in the Orca team meeting today. For the problem of not reporting anything when we initially Down arrow into the table, we are going to try out the following approach to see if it will fix the problem. (Note that if you Down arrow a second time, it will select that first entry.) If we are now selected but weren't previously, then we should speak/braille the expanded/collapsed state. Returning bug state to normal while this is investigated.
Mike - I think we need some design input from you. This may turn into a separate bug, but the new situation is that the table in question has the following multiselect behavior as brought to our attention by Joanie: * A dotted rectangle around the row lets you know about the current cell of interest. There's only one dotted rectangle. Moving the dotted rectangle up and down without changing selection is managed via Ctrl+{Up,Down}. * The highlighting of a cell indicates if it is selected or not. There can be multiple discontiguous sets of cells selected. The toggling of selection for the current cell of interest is managed via Ctrl+Space. Selection is also implicitly done when just Up or Down are pressed. When just Up or Down are pressed, all selection is cleared and then the newly landed on thing (i.e., the thing that gets the dotted rectangle) becomes the sole thing selected. We need your user experience design on this. In particular, I think a general model what should be presented when: * Navigating the dotted rectangle to a cell -- I'm assuming the contents should be spoken following the "read table cell or entire row" setting. But, I'm assuming the indication of whether the thing is selected or not is also important and should be presented. What should the presentation be in speech and braille for this and under what conditions should the presentation be done (e.g., only when unselected?). * Expanding/collapsing a cell that's not selected/highlighted. I'm assuming something should be spoken here, and it should match the output we get when we expand/collapse the node when it is selected? * Changing the selection state of the cell. I'm assuming we should also present something here. What should the presentation be in speech and braille for this? For technical details... When the dotted rectangle moves, we seem to get object:active-descendant-changed events: OBJECT EVENT: object:active-descendant-changed detail=(126,0) app.name='gtk-demo' name=None role='tree table' state='ENABLED FOC USABLE FOCUSED SENSITIVE SHOWING VISIBLE MANAGES_DESCENDANTS' relations='' When the selection changes, we seem to get object:state-changed:selected events: OBJECT EVENT: object:state-changed:selected detail=(1,0) app.name='gtk-demo' name='Army Day' role='table cell' state='ENABL ED FOCUSABLE SELECTABLE SELECTED SENSITIVE SHOWING SINGLE_LINE TRANSIENT VISIBLE ' relations='NODE_CHILD_OF' NOTE: handling this might require some rethinking of logic in default.py. For example, in default.py:onSelectionChanged, there's this comment: # Avoid doing this with objects that manage their descendants # because they'll issue a descendant changed event. # if event.source.getState().contains(pyatspi.STATE_MANAGES_DESCENDANTS): return In the light of this multiselect behavior, the assumption made in the comment is false: selection can change for the current active descendant.
> * Navigating the dotted rectangle to a cell -- I'm assuming the contents should > be spoken following the "read table cell or entire row" setting. But, I'm > assuming the indication of whether the thing is selected or not is also > important and should be presented. What should the presentation be in speech > and braille for this and under what conditions should the presentation be done > (e.g., only when unselected?). If the item is selected it should be underlined in braille honoring orca's existing braile marking setting. In speech "selected" should be the last thing spoken after the information which is currently presented about the cell or row. If an item is not selected nothing different should be spoken or brailed. > > * Expanding/collapsing a cell that's not selected/highlighted. I'm assuming > something should be spoken here, and it should match the output we get when we > expand/collapse the node when it is selected? How would one perform an action on something that wasn't selected? > > * Changing the selection state of the cell. I'm assuming we should also > present something here. What should the presentation be in speech and braille > for this? In braille the marking should be shown or taken away based on the new selection state. In speech "selected" or "unselected" should be spoken. With speech the cell/row should not be repeated when the selection changes.
> > * Expanding/collapsing a cell that's not selected/highlighted. I'm assuming > > something should be spoken here, and it should match the output we get when we > > expand/collapse the node when it is selected? > How would one perform an action on something that wasn't selected? There are two things happening here: 1) There is a dotted rectangle around the row indicating it is the thing of interest. Any keyboard action typically operates on this object. 2) There is a separate notion of selection. When something is selected, it is highlighted. Yes, it is confusing. I believe the reason things are done this way is to allow keyboard users to create multiple discontiguous selections. For example, if I want to select rows 4 and 6 of a table, I would: 1) Arrow to row 4. This would select 4 as a side effect of an unmodified arrow operation (i.e., I didn't hold the control key down) doing both a select and a move. 2) Ctrl+arrow to row 6. This would leave row 4 highlighted and move the dotted rectangle to row 6. 3) Press Ctrl+Space. This would select row 6, also leaving row 4 selected. The dotted rectangle remains on row 6.
> There are two things happening here: > > 1) There is a dotted rectangle around the row indicating it is the thing of > interest. Any keyboard action typically operates on this object. > > 2) There is a separate notion of selection. When something is selected, it is > highlighted. > > Yes, it is confusing. I believe the reason things are done this way is to > allow keyboard users to create multiple discontiguous selections. For example, > if I want to select rows 4 and 6 of a table, I would: > > 1) Arrow to row 4. This would select 4 as a side effect of an unmodified arrow > operation (i.e., I didn't hold the control key down) doing both a select and a > move. > > 2) Ctrl+arrow to row 6. This would leave row 4 highlighted and move the dotted > rectangle to row 6. > > 3) Press Ctrl+Space. This would select row 6, also leaving row 4 selected. > The dotted rectangle remains on row 6. > So if I am understanding this case correctly the only thing we should speak is the state change because we have already spoken the item with focus. Am I correct or did I miss something?
> So if I am understanding this case correctly the only thing we should speak is > the state change because we have already spoken the item with focus. > Am I correct or did I miss something? I think this sounds about right. In the team meeting, I think we also agreed to leave this bug open as a means for tracking the rework of dealing with active-descendant and selection change events when working with multiselect objects like this.
Mike believes the fix for this bug will fix a lot of odd interactions he's seeing with tables all over the place, so I'm marking this for 2.21.3 (02-Dec). I think test/keystrokes/gtk-demo/role_tree_table.py will probably also need some new tests added to it: 1) Navigation via Ctrl+{Up,Down} to move the dotted rectangle 2) Shift+{Left,Right} to expand/collapse both unselected and selected nodes 3) Ctrl+Space to select/unselect a node 4) {Up,Down} from an unselected node
Bug #486908 has been retitled "Selection and navigation in multiselectable items are not properly handled", and the comments related to it have been copied to that bug. We do not need to keep two bugs open for the same problem and as a fix for the original reported problem in this bug has been committed, I'm closing this bug as FIXED.