GNOME Bugzilla – Bug 792820
adding ScrollTo to at-spi-core?
Last modified: 2018-05-17 01:52:27 UTC
Hello, While working on accessibility of firefox, we have found that when continuously reading a web page, the page gets scrolled automatically only line by line. On windows, NVDA gets firefox to scroll paragraph by paragraph, which makes reading nicer. More generally, AT technologies such as magnifiers etc. could have a use for a primitive to tell application how scrolling should behave. On windows, IA2 provides ScrollTo RPCs http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/interface_i_accessible2.html#afd79074abd176643d560444058da55fc which allows the screen reader to act on application scrolling: bring some object to the top or bottom of the screen, at some specific location, etc. This is what allows NVDA to get firefox to scroll paragraph by paragraph. I'm wondering whether adding this to AT-SPI could be considered? Samuel
Thanks for filing this. Note that it will need to be added to atk first.
? I don't understand why. AIUI, at-spi2-core defines the at-spi protocol (in xml/), and then *atk implement the application part of it. Anyway, my question is rather: would such addition be considered? If so, I can work on defining it, provides patches, etc.
That's true--it is really just at-spi2-atk that needs the atk implementation first--but, yes, I think that that change makes sense, and we want to keep ATK/AT-SPI and IAccessible2 in synch where it makes sense.
Created attachment 367655 [details] [review] interface draft Here is a draft of what it would like, really mapped in IAccessible2, that should be alright I guess? (beware, I didn't even try to compile it).
Ah, in https://bugzilla.gnome.org/show_bug.cgi?id=789340 it is mentioned that it'd be useful to be able to tell to scroll to a particular offset of the object, notably for components and text interfaces. But in the IAccessible2 interface, there is no such thing...
1. Just because it's not in IAccessible2 doesn't mean we cannot have it. 2. IAccessible2 is not a frozen API. I agree it would be nice to keep things in sync. And Windows users might also benefit from this API. So why not add the offset functionality to IA2 as well?
Mike, what do you think? We can add ScrollTo to atspi2's Accessible interface like on IA2, as well to atspi2's Component and Text interfaces, and those could be added to IA2? I don't know how to push such additions to IA2 though.
(In reply to Samuel Thibault from comment #7) > Mike, what do you think? > > We can add ScrollTo to atspi2's Accessible interface like on IA2, as well to > atspi2's Component and Text interfaces, and those could be added to IA2? I > don't know how to push such additions to IA2 though. https://github.com/LinuxA11y/IAccessible2
Ah, I didn't know IAccessible2 was such an open thing, I thought it was controlled by Microsoft. About the addition to the Accessible interface like on IA2, one issue is that ATK's _AtkObjectClass has only one pad entry left, which is not enough for both ScrollTo and ScrollToPoint. Is it perhaps the opportunity to bump the ATK ABI interface to reserve more padding space? Or we can introduce just ScrollTo for now and leave ScrollToPoint for later (for instance NVDA doesn't use it, and I don't think we have plans to use it in Orca).
(In reply to Samuel Thibault from comment #9) > About the addition to the Accessible interface like on IA2, one issue is > that ATK's _AtkObjectClass has only one pad entry left, which is not enough > for both ScrollTo and ScrollToPoint. Is it perhaps the opportunity to bump > the ATK ABI interface to reserve more padding space? Or we can introduce > just ScrollTo for now and leave ScrollToPoint for later (for instance NVDA > doesn't use it, and I don't think we have plans to use it in Orca). As I understand, If we expect synchronization betwween screen reader and screen magnifier Orca would implement ScrollToPoint to make it possible to do. Best regards.
Well, depends how synchronization is done. AIUI from discussion in https://bugzilla.gnome.org/show_bug.cgi?id=789340 the screen reader would just call scrollto, and that would trigger a notification that the screen magnifier would catch, which would e.g. contain the coordinates, so that the screen magnifier knows where to move magnification to. An actual use case for ScrollToPoint would be that for some reason we really want something to get scrolled to a specific location of the screen. I don't see current situations where we would have the need for this, actually.
Regarding pads, that's a known issue. But why not put this in the component interface? That's the interface used for things like bounding boxes and points and the like.
Ah, Component is actually provided by all objects? I thought it was a grouping object.
In my experience things in the accessibility tree implement this interface. So, yeah, provided by all objects. Moreover, according to https://developer.gnome.org/atk/stable/AtkComponent.html ---- "AtkComponent should be implemented by most if not all UI elements with an actual on-screen presence, i.e. components which can be said to have a screen-coordinate bounding box. Virtually all widgets will need to have AtkComponent implementations provided for their corresponding AtkObject class. In short, only UI elements which are *not* GUI elements will omit this ATK interface. A possible exception might be textual information with a transparent background, in which case text glyph bounding box information is provided by AtkText." ---- So given a paragraph of text (e.g. the contents of the 'p' element in html), we'd have an accessible object with the role of ATK_ROLE_PARAGRAPH, it would implement (amongst other things) AtkComponent and AtkText. The former lets us know (amongst other things) the bounding box of the screen. The latter lets us know (amongst many other things) the bounding box of one or more characters. Related note: I just saw atk_component_set_position(). Maybe that could be used for the component (non-offset-specific) need. Dunno. Worth a try anyway....
Ugh. Component let's us know the bounding box of the <strike>screen</strike> paragraph.
Mmm, the current implementation of component_set_position in gtk is only to ask the window manager to move the toplevel-window to some place. I'm a bit afraid of merging the notion of set_position and scroll_to: moving a widget within a window makes sense independently from the notion of scrolling.
Fair enough. Looking at set_size() and set_extents(), those methods might be for purposes I cannot currently fathom. ;) Regardless, do you agree that scrolling the object into view is appropriate for AtkComponent?
Yes, I fixed the code I'm currently working on accordingly. That actually also fixes some #include issues :)
Created attachment 368544 [details] [review] proposed implementation Here is a tested proposed implementation. Will post the corresponding patches to atk and at-spi2-atk.
Here they are: https://bugzilla.gnome.org/show_bug.cgi?id=793587 and https://bugzilla.gnome.org/show_bug.cgi?id=793588
(and https://bugzilla.gnome.org/show_bug.cgi?id=794778 for pyatspi)
Comment on attachment 368544 [details] [review] proposed implementation Thanks for the patch. Looks good to me in general. A couple of very minor comments: + * @type: a #long indicating where the object should be placed on the screen. Should be documented as a #AtspiScrollType. + * @type: a #long indicating whether the coordinates are relative to the screen, + * to the window, or to the parent object. Similar comment to above; it's an AtspiCoordType. I think that documenting it as such will create the correct cross-reference in the documentation. Also, the parameter is named "coords"; the doc should match.
Created attachment 370351 [details] [review] proposed implementation Thanks for the review, here is a fixed version.
Comment on attachment 370351 [details] [review] proposed implementation Looks good to me. Thanks.
Created attachment 371604 [details] [review] proposed implementation Here is a patch generated with format-patch
Pushed to master. Thanks for the patch. Closing.