After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 652548 - AtkDocument interface required ...
AtkDocument interface required ...
Status: RESOLVED OBSOLETE
Product: atk
Classification: Platform
Component: atk
git master
Other Linux
: Normal normal
: ---
Assigned To: ATK maintainer(s)
ATK maintainer(s)
Depends on:
Blocks: 638537
 
 
Reported: 2011-06-14 11:41 UTC by Michael Meeks
Modified: 2021-06-10 11:27 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Michael Meeks 2011-06-14 11:41:57 UTC
To make document read-throughs, and serious document navigation work well - without resorting to impossible attempts to export peers for all document contents, and then try to lifecycle manage them; there needs to be a way to scan a 'viewport' across a document; and provide auxiliary helpers for managing the movement & context of that eg. 'goto next header'

Currently this is done (eg. in OO.o) with horrible hacks. If you want to "read through" a document - you get to move the cursor / caret to the last apparent accessible, and then -bing- by magic, the document is scrolled down - hopefully so that the next word is visible :-)

This is an inelegant hack IMHO, that makes writing a simple read-through-a-document app un-necessarily unpleasant.

I paste some mail fragments from old GNOME a11y discussions:


> >     Are you bearing in mind that all the reference counting pain is to
> > satisfy a situation that doesn't need to exist & should never have been
> > postulated ? :-) [ exposing an entire DOM ].
> 
> Indeed I am, but I haven't had a brainwave on a better solution. Got any
> ideas?

        Sure: it's mostly a matter of socialisation:

        * first we ask the ATs what they actually use "the whole DOM"
          for:
                + eg. "count number of headings in a document"
                + "skip to next heading quickly"

        * Then we create a new 'Browse' interface that can be used for
          these things:
                + this interface would broadly behave like a set of 
                  keybindings: skip to next heading eg. which would
                  substantially alter the view / invalidate a load of 
                  previously 'live' peers - but get what you want.
                + Other ideas might be a separate 'View' concept that
                  would be an 'off-screen' view of headings eg.

        This is IMHO the only sane way to do this anyway; eg. exposing infinite
spaces: eg. "all time", or "1 million x 16k spreadsheet cells" via the
current interface is somewhat tortured anyway.

...

To: 	Michael Curran <mick@kulgan.net>
Cc: 	accessibility@lists.freestandards.org
Subject: 	Re: [Accessibility] Objects contained in documents and tables
Date: 	Wed, 23 Jan 2008 09:24:15 +0000


Hi Mick,

On Tue, 2008-01-22 at 23:19 +1100, Michael Curran wrote:
> However, accessibility APIs don't really have a particular protocol for 
> communicating with documents.

        I'm gagging for a Document API such as you outline, in order to avoid
(once and for all) having to expose the entire (potentially infinite)
DOM to the AT, and allow far more effective & accurate client-side
caching of the entire accessible tree :-)

> As I am only representing myself here, I am not sure about what other 
> ATs' needs really are, or whether these issues even effect them. 
> However, I at least think it would be healthy to review all this stuff 
> together, and share tips and ideas, and see if we can make things easier 
> for ATs and app developers, on more than just one specific platform.

        Yep; particularly infrastructurally this sounds extremely interesting
to me.
Comment 1 Michael Meeks 2011-06-14 11:43:44 UTC
With more blurb: I'm sure back in some distant past I did a better sketch of this API.


From: 	Michael Meeks <michael.meeks@novell.com>
Reply-to: 	michael.meeks@novell.com
To: 	Willie Walker <William.Walker@Sun.COM>
Cc: 	Mark Doffman <mark.doffman@codethink.co.uk>, gnome-accessibility-devel@gnome.org
Subject: 	Re: [g-a-devel] ATK - Signal indicating new AtkObject creation.
Date: 	Tue, 13 Jan 2009 15:30:20 +0000


Hi Willie,

On Tue, 2009-01-13 at 09:40 -0500, Willie Walker wrote:
> The following is going to sound wishy-washy.  The main thing is that  
> I'm on the fence about this and think we really need a good performance  
> analysis to determine which way to go.  I'm CC'ing Michael Meeks for a  
> number of reasons and hope he can also provide some insight.

        :-) I think Mark is advocating a position close to what I tend to think
is optimal. AFAICS there are not a vast number of accessibles on (or
near) the screen at any one time, and exposing them all allows for
extremely fast iteration, and querying of a consistent object hierarchy
by the AT.

> Keep in mind that not every AT needs to know the complete hierarchy.   
> Orca right now, for example, tends only to look at localized areas  
> (e.g., the object with focus).

        Sure - OTOH, with the remote application working away and the AT doing
things too, there is huge scope for getting out of sync. One advantage
of caching really a lot of data in the client, is that while the AT runs
some code: eg. fetching all relations, checking their targets, types
etc. - the data is 100% consistent, until the AT yields to the mainloop.
Once we have to go across the process boundary of course that model
breaks, but we can avoid doing that for a lot of the common cases.

        An extra advantage is of course that the application itself can push
great wodges of data asynchronously to the ATs without blocking or
needing to respond to so many (any?) requests. That should reduce IPC
pressure and improve performance in many cases [ I suspect ].

> I think one of the motivations for lazy creation is dealing with very  
> large hierarchies: opening a large OOo document, going to a URL with  
> tons of content, opening a spreadsheet in OOo, etc.  The thought is  
> that it is more efficient to only create/use what you need.

        Hokay - so these are indeed difficult cases. But again, I'm fairly
certain that if we limit the accessible data to what is visible, (or
near visible, eg. menu items) - ie. it is madness to export the whole
spreadsheet, then we can still do a good job here.

>   In addition, I recall some things (OOo?) may not create accessible
> peers for objects until they are rendered on the screen. 

        Sure; this is the only way to do it AFAICS, consider worse cases like
"all of time" in a calendar / month-view widget or somesuch ;-) For
these cases I think we need a new 'paging' API, that allows people to
move down a page, or scroll-to-keep-cursor-centred or whatever - last we
talked this looked very useful to the impaired, for screenreaders & so
on.

> Note also that in order for Orca to be able to do things such as more
> sophisticated navigation (e.g., "go to the next table in this document"),
> it needs access to the whole document.

        Or better we need to expose only what is on-screen, and expose a nice
powerful table navigation API: "skip to next <Foo>" or whatever (also
"get count of headings", "skip to heading <N>"). OTOH - those guys are
going to be really efficient, and of course most applications are built
around optimising screen rendering & thus surely shouldn't struggle to
create a set of a11y peers for what is on-screen (?).

> As you mention, this would impact more than just GTK+.  It's been a  
> long time since I've worked on the internals of Java/Swing, but I  
> believe I recall I made the implementation there be lazy as well.  That  
> is, I don't believe the Accessible peer for it is created until one  
> asks for it:  

        There were at least some problems with this lazy creation AFAIR - I
forget the precise details, but IIRC if eg. you get a focus signal, and
then truly unref the currently focused object - it should go away, which
then leaves you with a new and hence in-comparable object reference
(that looks different), and of course if you truly un-ref it you don't
get any signal notifications on the object: which is surprising.

        So - AFAICS from a consistency POV having notifications from everything
that becomes visible is -very- much nicer, and of course - it should be
-really- fast to prune bits of the hierarchy we don't care about inside
the AT and ignore irrelevant events: because we can traverse the
hierarchy ~instantly.

> Another thing to consider are objects that have the "manages  
> descendants" property.  They can end up having a HUGE number of  
> children, which are perhaps not even known until one scrolls to them.

        Sure - and AFAICS these are broken. HUGE is limited by 2^32 in the
child indexes, and writing code that correctly handles these guys is
highly unpleasant :-) I'd prefer not to see any manages-descendants
objects - instead having the visible sub-set, and some nice traversal
APIs.

> In any case, I think the best thing to do here is experiment and get  
> performance numbers, especially for very large hierarchies.

        But of course, this is a great plan. Ultimately, the whole nightmare I
want to get rid of is the ref-counting life-cycle thing which is just
terrible: and if cocked up can easily leave ever-growing application
side leaks caused by badly behaved ATs (at least it could last I
looked). If we get rid of that, we can be confident that there is a well
defined, bounded memory usage, easily calculable from what is on-screen,
and which is likely to be far smaller than the X server memory use to
render the output on modern screens anyway :-)

> > In a more abstract way I think the 'push' model is conceptually
>> better. Say we were to use the GtkWidget 'realize' signal to declare
>> that a new AtkObject has been created. In this case the GtkWidget is
>> 'published' to the AT at the same time that it starts using screen
>> resources.

        Fair enough for most gtk+ widgets; there are a number of things that
need a11y peers that are not widgets though; so I'd be concerned to
ensure that these get created under the control of the 'gail' layer. And
of course, things like not-yet-revealed menu items (a nicely bounded
set), or toolbar items that don't quite fit on the screen ;-) are well
worth presenting in a nice flat hierarchy even though they may not be
realised yet.

> > I'm a fan of having accessibility turned on-by-default in Gnome, so
> > I don't have the excuse that this will only use extra resources when
> > accessibility is switched on. This change will use more memory for
> > Gtk applications. I'd need to do some investigation to find out how
> > much.

        Well; however little it is, some people will want to save those cycles,
on subnotebooks or whatever :-) Having said that, we could save a lot of
time and effort if we move many of the gail implementations into gtk+
itself; and - ultimately, all the real 'state' that has to be stored
should be in the widgets themselves so ... the memory cost can't be
-that- high, surely.

> > I'm not sure what the correct implementation in Gtk is, but its  
> > probably as simple as adding gtk_widget_get_accessible somewhere in
>> the GtkWidget construction.

        I guess people construct those beasts, and then set lots of properties
on them, and then whack them in the hierarchy. I'd be inclined to defer
the work until they become a child of a toplevel.

        Otherwise, this sounds like a positive step to me at least :-)
Comment 2 Joanmarie Diggs (IRC: joanie) 2011-06-14 18:43:52 UTC
See also: https://bugzilla.gnome.org/show_bug.cgi?id=345750

Which is not the same, of course, but strikes me as relevant -- and perhaps worth considering as part of the bigger "Improve ATK Document support" picture.(?)
Comment 3 André Klapper 2011-06-23 22:05:51 UTC
[Mass-reassigning open atk bug reports for better trackability as requested in https://bugzilla.gnome.org/show_bug.cgi?id=653179 .
PLEASE NOTE:
If you have watched the previous assignee of this bug report as a workaround for actually getting notified of changes in atk bugs, you yourself will now have to add atk-maint@gnome.bugs to your watchlist at the bottom of https://bugzilla.gnome.org/userprefs.cgi?tab=email to keep watching atk bug reports in GNOME Bugzilla.
Sorry for the noise: Feel free to filter for this comment in order to mass-delete the triggered bugmail.]
Comment 4 André Klapper 2021-06-10 11:27:00 UTC
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org.
As part of that, we are mass-closing older open tickets in bugzilla.gnome.org
which have not seen updates for a longer time (resources are unfortunately
quite limited so not every ticket can get handled).

If you can still reproduce the situation described in this ticket in a recent
and supported software version of atk, then please follow
  https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines
and create a ticket at
  https://gitlab.gnome.org/GNOME/atk/-/issues/

Thank you for your understanding and your help.