GNOME Bugzilla – Bug 586399
Orca should provide support/access to "Mouse Overs" in web content
Last modified: 2009-11-09 21:35:10 UTC
Content providers implement features/content which only appears when the user hovers the mouse over an object. These items are not keyboard accessible. Orca should alert the user to the presence of these objects and make it possible for users to interact with them. Example: Last.fm, hovering over an artist causes an 'x' to appear in the album cover allowing you delete the artist.
Mike: We'll of course need a spec from you on this. As you write it, you may find the following of use: 1. Not all items with associated mouse overs are necessarily focusable (e.g. it might be an image which is not inside a link). 2. At least in the case of last.fm (though my guess is that this may prove true for all content), there does not seem to be any action, attribute, or other magical foo that tells us that this item happens to have an associated mouse over action. In other words, it seems that we find out when the user finds out after he/she hovers the mouse over the thing. 3. An item might have both an associated action and a mouse over. For instance, in the case of last.fm, clicking on the majority of the image link will bring you to the artist page, but clicking on the tiny 'x' which appears in the upper left corner of the image link will delete the artist from your library.
Here are some thoughts based on these comments. All deas welcome of course. I wonder if we might want a user-toglable mode in where the mouse is moved along with the orca FF caret navigation. > > 1. Not all items with associated mouse overs are necessarily focusable (e.g. it > might be an image which is not inside a link). Hopefully we can still click these with flat review or a braille touch cursor. > 2. At least in the case of last.fm (though my guess is that this may prove true > for all content), there does not seem to be any action, attribute, or other > magical foo that tells us that this item happens to have an associated mouse > over action. In other words, it seems that we find out when the user finds out > after he/she hovers the mouse over the thing. > That would be where my above mouse movement feature would come into play. > 3. An item might have both an associated action and a mouse over. For instance, > in the case of last.fm, clicking on the majority of the image link will bring > you to the artist page, but clicking on the tiny 'x' which appears in the upper > left corner of the image link will delete the artist from your library. > I this case orca shouldpresent two separate clickable objects on one line. As I've never really been able to work with mouseovers this is going to need to be an iterative process.
Created attachment 138303 [details] [review] proof of concept/starting off point Step 1: Apply the pointer routing patch (bug 588403) Step 2: Apply this patch Step 3: Route the mouse pointer to an object you know has an associated onMouseOver which causes a new child to be added (e.g. last.fm, facebook friends) Orca should announce the new thing that has appeared. In the case of last.fm's delete link, the link is in the tab order and you can just Tab to get to it. In the case of the facebook friends menu thingy, its of role section and does not seem to be keyboard focusable. Presumably I can grab focus on it or one of its children; I haven't tried yet. Question: What should we be doing? In other words, what don't you like about what I've done so far *and* what should the next step be for the user to get to the thing that appeared when it's not in the tab order? (Note: I'm not saying this is how we should do things. I need a spec/guidance though on the desired user experience. Thanks.)
Created attachment 138647 [details] [review] Pretty functional proof of concept This adds a new command which will move focus to the object which appeared as a result of hovering the mouse over an item in web content. This command is bound to Orca+KP_Multiply for the Desktop layout and Orca+0 in the laptop layout. The way you would take advantage of this command is to do the following: 1. Move focus to an object which you know/suspect has an associated mouseover. 2. Route the pointer to that object using Orca+KP_Divide/Orca+9. If a new mouseover object appears as a result, Orca should announce what that new object is. 3. Move focus to the new object with Orca+KP_Multiply/Orca+0. This works with both the last.fm images and the facebook friends menu. Mike and Will: Thoughts?
Having worked on this a bit, I'm starting to notice mouseovers more. :-) In your Hulu queue, there is a mouseover associated with each title in the table. The resulting object contains the details of the given episode (original air date, rating, close captioned, description, etc.) Good news: My most recent patch enables Orca users to access that content where as they otherwise could not. Question: After I've read the episode details, I want to dismiss this informational object. The way sighted users dismiss such an object is by moving the mouse away from the object they had been hovering over. Do we want to do anything about/around this? Or do we expect the user to conclude that they're in a mouseover object and need to stop hovering over the current object by moving the mouse elsewhere? Note to self: I should probably store the just-before-we-moved-to-the-mouseover context so that I can place the user back there after an informational mouseover object is dismissed. (I'll do this tomorrow as I'd like to watch TV now. :-) )
I'm really liking the way this feature is coming together. I'll do more testing tomorrow as I'm headed out sailing shortly.
Created attachment 138716 [details] [review] Add clean-up functionality This is my answer to the question and note to self I posed last night. It seems to clean things up nicely in cases where you no longer want to be in the mouse over -- or the mouse over disappears. Usage notes/explanation: ~~~ There is a new command which will move focus to and away from the object which appeared as a result of hovering the mouse over an item in web content. This command is bound to Orca+KP_Multiply for the Desktop layout and Orca+0 in the laptop layout. The way you would take advantage of this command is to do the following: 1. Move focus to an object which you know/suspect has an associated mouseover. 2. Route the pointer to that object using Orca+KP_Divide/Orca+9. If a new mouseover object appears as a result, Orca should announce what it is. 3. Move focus to the new object with Orca+KP_Multiply/Orca+0. 4. If the object is something like a menu or a link, interact with it as you normally would. If you decide the mouse-over object is not of interest after all, or if the mouse-over object merely presented information, you can press Orca+KP_Divide/Orca+9 a second time. This will move the mouse pointer back to where it was before you routed it. The act of doing so should cause the mouse over object to disappear. The locusOfFocus will then be restored to the object you were on prior to accessing the mouse over. ~~~ Please let me know what you think. Thanks!
Oops. Typo in the usage notes. In step 4, if you decide the mouse over object is no longer of interest, you want to use Orca+KP_Multiply/Orca+0. In other words, there is just one command for moving into -- and later out of -- a mouse-over. Sorry 'bout that.
Mike have you had a change to look at this one? There are string additions and as a general rule I believe we try to aim for getting those in before the *.5 release which is this Monday.
I'd like to do a bit more testing but I like what you have come up with. I'll finish my testing either tonight or tomorrow morning.
Created attachment 139214 [details] [review] corrected usage notes and two new regression tests All regression tests passed. I also created two new ones -- one simple one with a javascript alert (annoying, but it's a test) and one based on accessing Yahoo's YUI menus -- at which point I promptly discovered that our new feature works great as long as you don't attempt to enter a nested mouseover object (menu within a menu). At that point nothing bad happens; you simply cannot work your way down the nested items because the command to get into a mouseover is the same as the command to work your way out of a mouseover. I still think it's important to minimize the complexity as far as the end user interaction is concerned. Therefore, in order to deal with this new issue, I plan to add smarts to our mouseover code to handle nested objects. But I'm going to call that another bug and worry about it a bit later. :-) This patch has been committed.
I just opened up bug 589726 as an uber bug for users to report the issues they find. Closing this one as FIXED.