GNOME Bugzilla – Bug 483023
Orca should give the user the ability to move between objects in order
Last modified: 2008-07-22 19:33:28 UTC
Orca should allow the user to move between the next and previous object in order. This is especially useful for moving out of form controls such as lists. The keys assigned to this feature should be: orca+left/right arrow. An example of this functionality: If the user is navigating a list and decides that he/she wishes to exit the list and continue arrowing through the web page he/she will press orca+right and immediately be placed on whatever happens to follow the list. .
Mike, what are the implications of this RFE w.r.t. caret navigation? I ask for a couple of reasons: 1. Bug #473009 (Cannot arrow to the end of an HTML entry if Orca is controlling the caret). 2. The broader performance refactor. To recap for the sake of documentation here: We recently made changes to how we handle navigation w.r.t. lists in forms. Namely, you can essentially walk right up to one, but not into one. Having walked up to one, you can Tab or Orca+Tab should you want move into it. Once this RFE has been implemented, you will also be able to use Orca+Left/Right should you want to move out of it. I'm wondering if we want to apply that "walk right up to, but not into" behavior to form fields in general. If so, I would assume bug #473009 would become a non-issue because Orca would no longer have a need to control the caret if we were in an HTML entry as our "escape" would be Orca+Left/Right. Thoughts? Thanks!
> I'm wondering if we want to apply that "walk right up to, but not into" > behavior to form fields in general. If so, I would assume bug #473009 would > become a non-issue because Orca would no longer have a need to control the > caret if we were in an HTML entry as our "escape" would be Orca+Left/Right. I would not be in favor of this change. I personally like being able to arrow on to buttons, checkboxes and edits.
Sounds good. But one more question I'm afraid. <smile> Do you like to be able to arrow into an entry and out of an entry using the plain arrow keys?
(In reply to comment #3) > Sounds good. But one more question I'm afraid. <smile> Do you like to be able > to arrow into an entry and out of an entry using the plain arrow keys? yes
Cool. Now that I'm thinking about it some more.... <smile> We already have find{Next, Previous}Object() methods that should apply quite nicely. I'm thinking we can rely upon those for the most part and special case stuff. Based on the original RFE, I am assuming that if we're in a list in a form, the next/previous object in order is not the next/previous list item, but rather whatever object comes after/before the entire list. Likewise, if we're on a menu item in a menu in a combobox, we're interested in whatever object comes after/before the combobox. So far so good. But what about other objects? A few that spring to mind: 1. If we're on a list item in a bulleted/numbered list, is the next/previous object whatever comes after/before that list item or after/before the entire list? 2. If we're at the beginning of a paragraph that has a link in the middle, would next object take us from our location at the beginning of the paragraph to the link, and from there to the text that follows the link, or something else? 3. If we have form fields with labels, should this functionality move us to the label and then the thing being labeled, or should it treat those two things as a single item? (and if the latter, should label guess also do its best to work this out?) Also: If we're at the very top/bottom of the page, should this functionality wrap? Sorry to keep asking you questions like this, but they keep popping up in my mind. This might could be something that would benefit from a spec.... Thanks!!
> 1. If we're on a list item in a bulleted/numbered list, is the next/previous > object whatever comes after/before that list item or after/before the entire > list? > I would prefer the entire list > 2. If we're at the beginning of a paragraph that has a link in the middle, > would next object take us from our location at the beginning of the paragraph > to the link, and from there to the text that follows the link, or something > else? > We want to move from the text at the begining of the paragraph to the link and then to the text following the link. > 3. If we have form fields with labels, should this functionality move us to the > label and then the thing being labeled, or should it treat those two things as > a single item? (and if the latter, should label guess also do its best to work > this out?) I think they should be treated as a single item using label guess if possible. In my mind this will give the user the most contextual information when moving around in this way. > > Also: If we're at the very top/bottom of the page, should this functionality > wrap? yes
New commands/keybindings mean new strings. Retargeting in the hopes that I can get this in before string announcement. :-)
Joanie - is this something that is achievable by the 2.21.5 deadline (Jan 14)?
Dunno. Hopefully. As I just commented on another bug, priorities changes a bit over the winter break. :-) I'll look at this soon.
Created attachment 102738 [details] [review] revision 1 Okay, here's the first go to see what we like and don't like. It should: 1. Wrap as Mike specified. 2. Position the caret before/after links within paragraphs, sections, etc. as specified. 3. Allow you to quickly and easily bail out of focusable lists and combo boxes, which I believe was the primary motivator for this RFE. 4. Skip over anything that is a proper label (as opposed to one we had to guess) that is labelling something on the same line, no matter what that something is. 5. Skip over anything that we *guess* is a label *for an entry*. I decided to just do guessing with entries for now for several reasons: a. Label guess is an expensive proposition performance-wize. Entry "label" text usually precedes the entry on the same line, so it's easy and performant to check for that. b. Issues like, should we examine the line that follows the line with the object to see if it contains form controls, and if it does, use extents to try to determine if anything on the current line might be a label for it, and if it does, eliminate that text -- or would we leave it in? c. Label guess is named appropriately. We don't do badly in our guesses, but I certainly wouldn't bet the farm on our accuracy. By presenting the objects in the order they appear on the screen, we're not tainting/misrepresenting the content. And besides: d. My understanding is that this functionality is not primarily intended as a way to navigate *to* form fields. We've got both Tab and Orca+Tab for that. And when you use either of them to give the form field focus, label guess kicks in in all its glory. 6. Read the remainder of the object contents up to the next object boundary. Thus if the next object is a link, Orca will speak the full link text. If the next object is paragraph text, Orca will read that text from the current caret location to either the end of the paragraph or the start of the next object (e.g. the next link that follows inside that paragraph). Mike, I assume that is what you wanted. Please let me know. It does *not* go from full (un)ordered list to list as Mike suggested. I looked at doing this, but there were more issues like, what if the list is just a way to layout a single line of links? What if the list is made up of links, but it's not for layout purposes only? What if it contains paragraphs? What if the list items are a mixture of objects? When do we skip to the next list and when do we decide that an object in the list is not something to be skipped over and how do we provide a user experience that is sufficiently consistent as to be predictable and understandable? I finally gave up and asked Mike how badly he wanted this. <smile> He seemed to be far more concerned about not going from list item to list item in *form field lists*. I believe I've got you covered there, Mike. It *does* pylint to 10.0, but holy cow it takes for-bleeping-ever to complete. I know pylint is not the most speedy checker, but I pylint my Gecko.py work constantly and it NEVER takes this long. Will, any ideas what horrible pythonic crimes I might have committed to cause this? So, guys, let me know what you want done differently. Thanks! By the way, this patch will likely need a slight change if we commit the combo box patch I posted earlier. Therefore, please test this *without* the combo box patch.
As an aside, periodically users coming to us from the Windows world ask for the ability to navigate document content as though each object were on a line by itself. Most users (and our UI guy) protest vehemently at this point. <grin> But the requests persist. The functionality implemented in the current patch largely mirrors the functionality being requested, so we might be killing two birds with one stone: Users could either use Mike's binding if they only need it on occasion, or if they really hate the default behavior, they can delete the keybindings associated with Up and Down Arrow and rebind Up and Down to the new object navigation. The one difference that will remain -- assuming we don't make drastic "user experience" changes to what I've implemented here -- is that a large chunk of text gets treated as one "object" and is all spoken when the user moves there. If we can assume that the users in the OOPL (one object per line) camp will be happy using Orca+Left/Right all of the time, then this is not an issue: They can switch to Up/Down when large chunks of text are encountered. On the other hand, if these folks are likely to rebind Up/Down Arrow to the new object navigation functionality, we'll need another solution. My wrist is tired from testing Orca + Left and Right, so I'm leaning towards another solution. <smile> Presumably what we could do is add a setting which, if enabled, would stop speaking the object as soon as a new object is encountered (what it does already) *or* a line break is encountered. While I haven't tried it yet, it think it will be easy to do. In fact, one of the challenges in implementing this functionality is working out how to find the "beginning" of the object when the object doesn't "begin" at offset 0 and you're moving backwards. (See the "mayHaveGoneTooFar" variable and subsequent comments in goPreviousObjectInOrder() :-) ) So.... Thoughts?
(In reply to comment #10) > 5. Skip over anything that we *guess* is a label *for an entry*. > I decided to just do guessing with entries for now for several > reasons: > > a. Label guess is an expensive proposition performance-wize. > Entry "label" text usually precedes the entry on the same > line, so it's easy and performant to check for that. How frequently will the in order movement feature be used, and how bad is the performance impact? > b. Issues like, should we examine the line that follows the > line with the object to see if it contains form controls, > and if it does, use extents to try to determine if anything > on the current line might be a label for it, and if it does, > eliminate that text -- or would we leave it in? I'm not sure -- I think we need Mike's input on the user experience on that one. > c. Label guess is named appropriately. We don't do badly in > our guesses, but I certainly wouldn't bet the farm on our > accuracy. By presenting the objects in the order they appear > on the screen, we're not tainting/misrepresenting the content. I this is probably the best argument for not doing the label guess. :-) > It *does* pylint to 10.0, but holy cow it takes for-bleeping-ever > I know pylint is not the most speedy checker, but I pylint my Gecko.py work > constantly and it NEVER takes this long. Will, any ideas what horrible > pythonic crimes I might have committed to cause this? No clue. I saw an increase where the total time went from 1 minute to 7 minutes! I wonder if it has something to do with pylint's feature of checking for similarities in code? I mean holy freaking file size, bat man, that file is like over 9700 lines long. Gag me with a spoon. Naw...gag me with a humongous pile of code -- that Gecko certainly would leave a big lump in the belly of even the largest Python. At some point it might be good to try to figure out a way to break that file apart into smaller modules. In any case, We supposedly disable this in pylintrc, but maybe it just disables the output of similarity info and not the computation? > So, guys, let me know what you want done differently. Thanks! I'm just curious about the following diff. We might need some documentation in the code telling us what's going on -- I see it elsewhere in the code for handling lists, so I assume the same logic applies here: - if not obj.getState().contains(pyatspi.STATE_FOCUSABLE): + if not obj.getState().contains(pyatspi.STATE_SELECTABLE): In any case, if this tests well and it does what Mr. Mike is wishing for, then I say commit it.
Note to self: Need to make a revised patch to take the new combo box change (bug 509068) into consideration. I had to do something special for lists (because we walk up to them but not into them) and will now need to do that for combo boxes. Other than combo boxes, this patch should still work as expected. <smile> Putting my name on it as a prompt. (ADHD is fun, kids!) :-)
When the fix for comboboxes gets made here I'm happy with this patch.
Actually, I just tried the existing patch with combo boxes. I think it works as is. Mike please test some more and let me know. Thanks!
To me this seems like what we want.
The patch has been committed. Moving to pending. I'll make the string change announcements.
This one is really useful and is already getting possitive user feedback. thanks
And it seems like it in its current form it may be addressing the needs of the folks who have been asking for the "treat each object as if it were on its own line" line navigation. (i.e. Perhaps there is no need for an additional setting as I proposed earlier in this bug). Yea! Closing as FIXED. Thanks Mike.
(In reply to comment #19) > And it seems like it in its current form it may be addressing the needs of the > folks who have been asking for the "treat each object as if it were on its own > line" line navigation. (i.e. Perhaps there is no need for an additional setting > as I proposed earlier in this bug). Yea! Woo hoo! This is good to hear. Although we had our opinions about the requests coming in, we were still going to get these requests. So, it's good that we've addressed the desires of the community on this one. Thanks! Joanie - if you get a chance, can you please write something up regarding this on http://live.gnome.org/Orca/Firefox so users coming from Windows can easily find it?