With more blurb: I'm sure back in some distant past I did a better sketch of this API.


From: 	Michael Meeks <michael.meeks@novell.com>
Reply-to: 	michael.meeks@novell.com
To: 	Willie Walker <William.Walker@Sun.COM>
Cc: 	Mark Doffman <mark.doffman@codethink.co.uk>, gnome-accessibility-devel@gnome.org
Subject: 	Re: [g-a-devel] ATK - Signal indicating new AtkObject creation.
Date: 	Tue, 13 Jan 2009 15:30:20 +0000


Hi Willie,

On Tue, 2009-01-13 at 09:40 -0500, Willie Walker wrote:
> The following is going to sound wishy-washy.  The main thing is that  
> I'm on the fence about this and think we really need a good performance  
> analysis to determine which way to go.  I'm CC'ing Michael Meeks for a  
> number of reasons and hope he can also provide some insight.

        :-) I think Mark is advocating a position close to what I tend to think
is optimal. AFAICS there are not a vast number of accessibles on (or
near) the screen at any one time, and exposing them all allows for
extremely fast iteration, and querying of a consistent object hierarchy
by the AT.

> Keep in mind that not every AT needs to know the complete hierarchy.   
> Orca right now, for example, tends only to look at localized areas  
> (e.g., the object with focus).

        Sure - OTOH, with the remote application working away and the AT doing
things too, there is huge scope for getting out of sync. One advantage
of caching really a lot of data in the client, is that while the AT runs
some code: eg. fetching all relations, checking their targets, types
etc. - the data is 100% consistent, until the AT yields to the mainloop.
Once we have to go across the process boundary of course that model
breaks, but we can avoid doing that for a lot of the common cases.

        An extra advantage is of course that the application itself can push
great wodges of data asynchronously to the ATs without blocking or
needing to respond to so many (any?) requests. That should reduce IPC
pressure and improve performance in many cases [ I suspect ].

> I think one of the motivations for lazy creation is dealing with very  
> large hierarchies: opening a large OOo document, going to a URL with  
> tons of content, opening a spreadsheet in OOo, etc.  The thought is  
> that it is more efficient to only create/use what you need.

        Hokay - so these are indeed difficult cases. But again, I'm fairly
certain that if we limit the accessible data to what is visible, (or
near visible, eg. menu items) - ie. it is madness to export the whole
spreadsheet, then we can still do a good job here.

>   In addition, I recall some things (OOo?) may not create accessible
> peers for objects until they are rendered on the screen. 

        Sure; this is the only way to do it AFAICS, consider worse cases like
"all of time" in a calendar / month-view widget or somesuch ;-) For
these cases I think we need a new 'paging' API, that allows people to
move down a page, or scroll-to-keep-cursor-centred or whatever - last we
talked this looked very useful to the impaired, for screenreaders & so
on.

> Note also that in order for Orca to be able to do things such as more
> sophisticated navigation (e.g., "go to the next table in this document"),
> it needs access to the whole document.

        Or better we need to expose only what is on-screen, and expose a nice
powerful table navigation API: "skip to next <Foo>" or whatever (also
"get count of headings", "skip to heading <N>"). OTOH - those guys are
going to be really efficient, and of course most applications are built
around optimising screen rendering & thus surely shouldn't struggle to
create a set of a11y peers for what is on-screen (?).

> As you mention, this would impact more than just GTK+.  It's been a  
> long time since I've worked on the internals of Java/Swing, but I  
> believe I recall I made the implementation there be lazy as well.  That  
> is, I don't believe the Accessible peer for it is created until one  
> asks for it:  

        There were at least some problems with this lazy creation AFAIR - I
forget the precise details, but IIRC if eg. you get a focus signal, and
then truly unref the currently focused object - it should go away, which
then leaves you with a new and hence in-comparable object reference
(that looks different), and of course if you truly un-ref it you don't
get any signal notifications on the object: which is surprising.

        So - AFAICS from a consistency POV having notifications from everything
that becomes visible is -very- much nicer, and of course - it should be
-really- fast to prune bits of the hierarchy we don't care about inside
the AT and ignore irrelevant events: because we can traverse the
hierarchy ~instantly.

> Another thing to consider are objects that have the "manages  
> descendants" property.  They can end up having a HUGE number of  
> children, which are perhaps not even known until one scrolls to them.

        Sure - and AFAICS these are broken. HUGE is limited by 2^32 in the
child indexes, and writing code that correctly handles these guys is
highly unpleasant :-) I'd prefer not to see any manages-descendants
objects - instead having the visible sub-set, and some nice traversal
APIs.

> In any case, I think the best thing to do here is experiment and get  
> performance numbers, especially for very large hierarchies.

        But of course, this is a great plan. Ultimately, the whole nightmare I
want to get rid of is the ref-counting life-cycle thing which is just
terrible: and if cocked up can easily leave ever-growing application
side leaks caused by badly behaved ATs (at least it could last I
looked). If we get rid of that, we can be confident that there is a well
defined, bounded memory usage, easily calculable from what is on-screen,
and which is likely to be far smaller than the X server memory use to
render the output on modern screens anyway :-)

> > In a more abstract way I think the 'push' model is conceptually
>> better. Say we were to use the GtkWidget 'realize' signal to declare
>> that a new AtkObject has been created. In this case the GtkWidget is
>> 'published' to the AT at the same time that it starts using screen
>> resources.

        Fair enough for most gtk+ widgets; there are a number of things that
need a11y peers that are not widgets though; so I'd be concerned to
ensure that these get created under the control of the 'gail' layer. And
of course, things like not-yet-revealed menu items (a nicely bounded
set), or toolbar items that don't quite fit on the screen ;-) are well
worth presenting in a nice flat hierarchy even though they may not be
realised yet.

> > I'm a fan of having accessibility turned on-by-default in Gnome, so
> > I don't have the excuse that this will only use extra resources when
> > accessibility is switched on. This change will use more memory for
> > Gtk applications. I'd need to do some investigation to find out how
> > much.

        Well; however little it is, some people will want to save those cycles,
on subnotebooks or whatever :-) Having said that, we could save a lot of
time and effort if we move many of the gail implementations into gtk+
itself; and - ultimately, all the real 'state' that has to be stored
should be in the widgets themselves so ... the memory cost can't be
-that- high, surely.

> > I'm not sure what the correct implementation in Gtk is, but its  
> > probably as simple as adding gtk_widget_get_accessible somewhere in
>> the GtkWidget construction.

        I guess people construct those beasts, and then set lots of properties
on them, and then whack them in the hierarchy. I'd be inclined to defer
the work until they become a child of a toplevel.

        Otherwise, this sounds like a positive step to me at least :-)

See also: https://bugzilla.gnome.org/show_bug.cgi?id=345750

Which is not the same, of course, but strikes me as relevant -- and perhaps worth considering as part of the bigger "Improve ATK Document support" picture.(?)

[Mass-reassigning open atk bug reports for better trackability as requested in https://bugzilla.gnome.org/show_bug.cgi?id=653179 .
PLEASE NOTE:
If you have watched the previous assignee of this bug report as a workaround for actually getting notified of changes in atk bugs, you yourself will now have to add atk-maint@gnome.bugs to your watchlist at the bottom of https://bugzilla.gnome.org/userprefs.cgi?tab=email to keep watching atk bug reports in GNOME Bugzilla.
Sorry for the noise: Feel free to filter for this comment in order to mass-delete the triggered bugmail.]

GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org.
As part of that, we are mass-closing older open tickets in bugzilla.gnome.org
which have not seen updates for a longer time (resources are unfortunately
quite limited so not every ticket can get handled).

If you can still reproduce the situation described in this ticket in a recent
and supported software version of atk, then please follow
  https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines
and create a ticket at
  https://gitlab.gnome.org/GNOME/atk/-/issues/

Thank you for your understanding and your help.