After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 433802 - Orca prints warnings when Firefox starts
Orca prints warnings when Firefox starts
Status: RESOLVED FIXED
Product: at-spi
Classification: Platform
Component: registry
0.0.1
Other All
: Normal normal
: ---
Assigned To: Li Yuan
Li Yuan
Depends on:
Blocks:
 
 
Reported: 2007-04-27 06:26 UTC by Ginn Chen
Modified: 2007-06-26 09:31 UTC
See Also:
GNOME target: ---
GNOME version: 2.17/2.18


Attachments
at-spi part of patch (860 bytes, patch)
2007-05-09 09:18 UTC, Li Yuan
none Details | Review
new patch to handle document: (479 bytes, patch)
2007-05-10 10:13 UTC, Li Yuan
committed Details | Review

Description Ginn Chen 2007-04-27 06:26:10 UTC
Please describe the problem:
I got 3 warnings in both Firefox and Orca terminals.

Steps to reproduce:
1. Start Orca
2. Start Firefox trunk nightly
3. 


Actual results:
Got 3 warnings,
** (orca:30701): WARNING **: Invalid object type load-complete


** (orca:30701): WARNING **: Invalid object type load-stopped


** (orca:30701): WARNING **: Invalid object type reload


Expected results:
No warnings

Does this happen every time?
Yes

Other information:
Comment out listeners["document:..."] = self.OnDoucment... in Gecko.py or
change it to listeners["Gtk:AtkDocument:..."] = 
can solve this issue.

In at-spi/atkbridge/bridge.c it's using "Gtk:AtkDocument:...", it works.
I don't know what's correct way in python.
Comment 1 Willie Walker 2007-05-01 16:17:06 UTC
Interesting.  It seems as though the "document:" forms of the events are issued by Firefox, but something in the infrastructure is unhappy about someone listening for them.  I'm not convinced this is an Orca bug.  :-(  Maybe Li can help?

BTW, here's an example of the debug output:

vvvvv PROCESS OBJECT EVENT document:load-complete vvvvv
OBJECT EVENT: document:load-complete                   detail=(0,0)
    app.name='Minefield'          name='Google' role='document frame' state='FOCUSABLE SHOWING VISIBLE' relations=''
^^^^^ PROCESS OBJECT EVENT document:load-complete ^^^^^

In addition, we seem to get expected results when listening for the "document:" forms, but will receive what appears to multiple duplicate events when listening for the "Gtk:AtkDocument:" forms.  For example, here's some debug output of events received when loading www.ebay.com:

---------> QUEUEING EVENT document:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete
---------> QUEUEING EVENT Gtk:AtkDocument:load-complete

Yikes.  That's a lot of events to contend with.

Finally, to bring this into light, what we're trying to accomplish is determine when a web page has completed loading.  The "document:" forms of the events seemed to accomplish this for us rather nicely with little extra work.

Li, do you know why listening for the various "document:" events would cause invalid object type warnings to be emitted?  Is it possible that Firefox has to do some sort of registration or something?


Comment 2 Ginn Chen 2007-05-03 08:49:46 UTC
I wish somebody can point out how
listeners["document:..."] = self.OnDoucment...
finally works.
I mean how the events get registered.

I think at-spi/atkbridge/bridge.c already registered Gtk:AtkDocument:... events.
Why Orca needs to register it again from Firefox via CORBA?
Shouldn't it be done inside at-spi?

I'm not sure this is a bug of Orca, or the infrastructure.

For Will's question, why it causes invalid object type.
Please refer to http://lxr.mozilla.org/seamonkey/source/accessible/src/atk/nsAppRootAccessible.cpp#241
Any event not starting with "window", the second word will be considered as object name.
i.e. for "document:reload", "reload" is considered as object name.
You can find similar code in gailutil.
It's consistent with spi_atk_register_event_listeners in bridge.c
(http://svn.gnome.org/viewcvs/at-spi/trunk/atk-bridge/bridge.c?view=markup)
 
Comment 3 Willie Walker 2007-05-03 16:56:09 UTC
> I wish somebody can point out how
> listeners["document:..."] = self.OnDoucment...
> finally works.
> I mean how the events get registered.

Wow...it took some time, but I think I figured it out.

First some background.  My understanding (which might be very wrong -- there are so many levels of redirection going on that it is hard to get your mind wrapped around it all) is this:

0) The bridge always registers a set of listeners for event types in bridge.c:spi_atk_register_event_listeners.  This set of listeners is registered with the ATK support in the application.  This set is not the complete set of all possible event types, but includes things like window:* and Gtk:AtkDocument:* listeners.  I'm guessing it does so because the AT-SPI design forced the bridge to do so, and/or the bridge wants to do some bookkeeping of its own.

1) AT's register event listeners with the bridge.  As needed, the bridge then registers its own event listeners with the ATK support in all apps.  The ATK support in the app then registers event listeners with the toolkit it is working with.  

2) When the toolkit in the application issues an event that is being listened for, they are issued to the ATK listener inside the app.  The ATK support then issues the event to the bridge. The bridge then issues the event to the ATs that are listening for them..

As such, Orca doesn't register an event listener with the app directly.  It does so indirectly by telling the bridge that it is interested in a particular event type from any application.  Orca also doesn't automatically get events -- it has to explicitly register for them.

Now...the ultimate culprit of our problem might be the atk-bridge (which can be found in the at-spi module).  I see it doing several things like this in bridge.c:spi_atk_register_event_listeners:
 
 id = atk_add_global_event_listener (spi_atk_bridge_document_event_listener,
                                      "Gtk:AtkDocument:load-complete");
 
This registration of Gtk:AtkDocument:* events thus tells every application to send the event on to the bridge, no matter if any AT requested it or not.  When the bridge gets such an event from the app, the bridge.c:spi_atk_bridge_document_event_listener method then transforms the event as so, replacing "Gtk:AtkDocument" with "document":

  spi_atk_emit_eventv (gobject, 0, 0, &any,
                       "document:%s", name);
 
The document:* events are then issued to any AT's that have registered for them.  Now, the only thing that really knows about the document:* events is the bridge because it's just plain making them up.  So, the AT gets lucky - the ATK support always issues Gtk:AtkDocument:* events to the bridge since it is always listening for them, the bridge converts them to document:* events, and the AT gets these because it happens to have listeners for document:* events.

So, now we're in a Catch-22 situation: the ATs need to register for document:* events to get them, but if they do so, the ATK support in the toolkit squawks.

To fix this, I think two things might be needed:

1) Get rid of the warning.  The complaint seems to happen in gailutil.c:add_listener and there's also similar code in Gecko, as you pointed out.  It seems to be saying "the toolkit that I work with doesn't know of any such event type, so I'm going to let you know."  If my understanding is correct, this warning probably should be eliminated -- I don't think AT-SPI enabled toolkits should assume they know all the possible events types in the world (e.g., imagine a stripped down toolkit).

2) Fix the real problem.  The bridge is transforming the Gtk:AtkDocument:* events to document:* when sending them to ATs, but it is not transforming document:* event type strings to Gtk:AtkDocument:* type strings when the AT registers for document:* event types.  I'd guess the bridge's event listener registration code should detect registration for document:* event types and do the appropriate thing (e.g., just store the fact that the AT cares about document:* events).

By the way, note that http://developer.mozilla.org/en/docs/Accessibility/ATSPI_Support lists document:* events.  If the problem is fixed, I suppose this is still kind of accurate, though the pedantic might describe the Gtk:AtkDocument:* munging being done by the bridge.
Comment 4 Willie Walker 2007-05-07 16:36:31 UTC
Li, any thoughts on this?
Comment 5 Li Yuan 2007-05-09 09:14:20 UTC
I just checked the code. Maybe it is the problem of http://bugzilla.gnome.org/attachment.cgi?id=73090

We should emit object:document:reload here. Not document:reload. Thus in gailutil.c:add_listener, object_type should be "document", signal should be "reload". This will pass gail's check.

So I will make patch for bridge, and you can change your code in Gecko.py. Just change document:reload to object:document:reload should be OK.
Comment 6 Li Yuan 2007-05-09 09:18:16 UTC
Created attachment 87861 [details] [review]
at-spi part of patch
Comment 7 Li Yuan 2007-05-09 09:34:03 UTC
BTW, I was on vacation during May 1 to May 8. Please forgive me if I reply late.
Comment 8 Ginn Chen 2007-05-09 10:08:49 UTC
I think the point here is not passing check of gailutil or mozilla maiutil.
mai or gail's add_global_event_listener should not be called directly from Orca.

I found if it changed to "object:document:*", it would not happen.
Maybe somewhere in the infrastructure, add listeners for Orca for its known events.
"document:*" is not a known event, so it will call add_global_event_listener directly.
And add_global_event_listener doesn't work with this format either,  and prints warning.
Comment 9 Li Yuan 2007-05-09 11:35:35 UTC
Hi Will,

Which function does Orca uses to register listener? Is it Accessibility_Registry_registerGlobalEventListener?
Comment 10 Willie Walker 2007-05-09 13:33:01 UTC
(In reply to comment #9)
> Hi Will,
> 
> Which function does Orca uses to register listener? Is it
> Accessibility_Registry_registerGlobalEventListener?
> 

Hi Li: 

That's what it does.  In general, Orca only communicates with applications via structures defined in the AT-SPI IDL files.  As such, there should be no concern about Orca calling other things (e.g., gail, etc.) -- Orca's solely a consumer of the AT-SPI.  Ginn, you raised a question about this, though.  Is there something that led you to believe Orca might be doing otherwise?

Regarding changing document:* to object:document:*.  This definitely seems like an easy fix, but I don't think it is the complete solution.  That is, it may have been the right thing to do from the beginning, but I think we're kind of stuck with document:* right now.  All the documentation I run into (e.g., http://www.gnome.org/~billh/at-spi-new-idl/html/html/classAccessibility_1_1Document.html and http://developer.mozilla.org/en/docs/Accessibility/ATSPI_Support) specify document:* events and not object:document:* events.  

As such, dropping document:* events and moving to object:document:* events represents what I would consider an incompatible change.  To move forward, I think compatibility needs to be ensured.  

What about something like the following:

1) Modify the bridge's event listener registration code to detect the registration of document:* event types and do the appropriate thing.  For example, maybe the registration function could include a special case check for "document:" and convert it into "object:document:" before doing anything else.  It may also need to somehow store the notion of the AT listening for "document:*" events somewhere.  I'm not sure where that is done.

2) On the event emission, emit both 'document:%s' and 'object:document:%s' events.  Thus, ATs can listen for  one or the other and things will still work and the AT will only get the events it registered for.

While this may be a little more code, it helps us in a couple ways:

1) ATs continue to work with the existing AT-SPI infrastructure., although the warnings will still be issued.  Keep in mind these are just warnings, and as described in comment #3, things will work as expected from the AT side.  We can live with that.

2) When the AT-SPI infrastructure is updated, the ATs will not need to be modified and the warnings will go away.

To contrast this with migrating to object:document:* and dropping document:*:

1) We cannot put accessibility on hold until GNOME 2.20 is prevalent throughout all distributions.  As such, ATs will need  to continue to listen for document:* events in order to work with the existing infrastructure. As a result, warnings will still be issued from the current and future versions of the AT-SPI, and we really did nothing to accomplish the goal of eliminating the warnings.

2) ATs will also need to listen object:document:* events in order to work with the new infrastructure.  This makes the ATs jobs a bit more complex..

So, a little more work on the AT-SPI infrastructure side can help ensure compatibility and can also help to eliminate the warnings in a future-proof way.

Will
Comment 11 Li Yuan 2007-05-10 10:13:29 UTC
Created attachment 87944 [details] [review]
new patch to handle document:
Comment 12 Li Yuan 2007-05-10 10:48:00 UTC
Will and Ginn, please try my new patch (without the first one).
Because registryd doesn't recognize "document:", it put the listener to the wrong list (mouse and the toolkit list). The patch put the "document:" listener to the right list. 

If the event type is toolkit, registryd need to register it to application, this is why firefox output the warnings.
Comment 13 Li Yuan 2007-05-10 10:52:47 UTC
Will,

If I emit the "object:document:*" in gnome 2.18 (still emit "document:*"), does it break any gnome policies?
Comment 14 Willie Walker 2007-05-10 13:41:11 UTC
> If I emit the "object:document:*" in gnome 2.18 (still emit "document:*"), does
> it break any gnome policies?

I'm guessing not, mostly because the AT-SPI event types seem largely unspecified.  It is a little odd to get 2 different events for the same thing, though.  If anything, however, keeping things as just "document:*" and not exposing "object:document:*" seems like the more compatible thing to do.
Comment 15 Willie Walker 2007-05-10 15:15:19 UTC
> Will and Ginn, please try my new patch (without the first one).
> Because registryd doesn't recognize "document:", it put the listener to the
> wrong list (mouse and the toolkit list). The patch put the "document:" listener
> to the right list. 

Li: this still issues the warnings for me.  I wonder if more work needs to be done here -- is it possible that ETYPE_OBJECT is assuming that a string that is always prefixed with "object:"?

In any case, are you able to reproduce this yourself?
Comment 16 Willie Walker 2007-05-10 15:19:57 UTC
I've reassigned this to at-spi since it really appears to be a problem with
at-spi and not Orca.
Comment 17 Li Yuan 2007-05-11 02:10:48 UTC
Will, did you logout after you build at-spi? Because the modification is in registryd, so you may need to logout to restart registryd. The warning does disappear on my machine.
Comment 18 Willie Walker 2007-05-11 13:40:47 UTC
(In reply to comment #17)
> Will, did you logout after you build at-spi? Because the modification is in
> registryd, so you may need to logout to restart registryd. The warning does
> disappear on my machine.
> 

I did a power cycle.  But, it looks like I just assumed things were installed where Ubuntu would expect them to be if I used "./autogen.sh --prefix=/usr".  When I search for at-spi-registryd on my machine, it looks like I made a bad assumption:

wwalker@wwalker-laptop:~/at-spi/trunk$ ls -l /usr/lib/at-spi/at-spi-registryd /usr/libexec/at-spi-registryd
-rwxr-xr-x 1 root root  48740 2007-04-10 08:59 /usr/lib/at-spi/at-spi-registryd
-rwxr-xr-x 1 root root 175630 2007-05-11 09:13 /usr/libexec/at-spi-registryd

wwalker@wwalker-laptop:~/at-spi/trunk/registryd$ ps -elf | grep at-spi
0 S wwalker   5307  5261  0  75   0 -  3436 367255 May10 ?        00:01:21 /usr/lib/at-spi/at-spi-registryd

I don't recall running into this problem in the past.  Do you have suggestions for how I should build/install at-spi?  Should I use "--prefix=/usr --libexec=/usr/lib/at-spi"?  (I'll try that now).
Comment 19 Willie Walker 2007-05-11 22:10:40 UTC
> I don't recall running into this problem in the past.  Do you have suggestions
> for how I should build/install at-spi?  Should I use "--prefix=/usr
> --libexec=/usr/lib/at-spi"?  (I'll try that now).

Seemed to do the trick and the warnings are now gone for me, too.  Thanks!
Comment 20 Li Yuan 2007-05-14 05:55:17 UTC
Yes, the registryd is installed in /usr/libexec by default. But Ubuntu guys install it in /usr/lib/at-spi. On Solaris it is installed in /usr/lib. So if you want to build at-spi on these platforms, setting --libexec option is necessary.

Does the "document:" signals work now?
Comment 21 Willie Walker 2007-05-14 13:36:23 UTC
> Does the "document:" signals work now?

Yes - this works as it used to and the warnings are gone.  Thanks!

Comment 22 Willie Walker 2007-06-19 16:16:38 UTC
Was this committed and released in a recent version for GNOME 2.19?
Comment 23 Li Yuan 2007-06-26 09:30:48 UTC
I'm sorry, I forgot to commit the patch. The patch will get into at-spi 1.19.5.