After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 617674 - Orca fails to present changes to labels in SWT
Orca fails to present changes to labels in SWT
Status: RESOLVED DUPLICATE of bug 568611
Product: orca
Classification: Applications
Component: general
2.30.x
Other All
: Normal enhancement
: 2.32.0
Assigned To: Joanmarie Diggs (IRC: joanie)
Orca Maintainers
Depends on:
Blocks:
 
 
Reported: 2010-05-04 17:52 UTC by Carolyn_MacLeod
Modified: 2010-09-20 10:52 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Snippet340.java with live region code (2.81 KB, text/plain)
2010-05-04 21:56 UTC, Carolyn_MacLeod
Details

Description Carolyn_MacLeod 2010-05-04 17:52:00 UTC
Using Orca 2.30.0 on Ubuntu 10.04 with Eclipse build I20100504-0800.

I have a little test with a gtk_label that is updated based on what a user types in a gtk_text_view. The text_view is "describedBy" the label (using a relation). And the object properties of the label include aria-live:assertive and aria-atomic:true. After calling gtk_label_set_text with a new string, I call g_object_notify with accessible_description to notify that the description has changed. I am expecting to hear the new description spoken when the user types various values.

This little test works with JAWS on Windows. But Orca does not announce changes in the label. I gather that Orca only looks for live regions in web applications, but this is just a plain window with a label and a text view.

It would be great if we could take the live region support that currently exists for web applications and generalize it to include any application.
Comment 1 Carolyn_MacLeod 2010-05-04 19:12:52 UTC
Here's a link to get Eclipse build I20100504-0800:
http://download.eclipse.org/eclipse/downloads/drops/I20100504-0800/index.php
I assume you would probably want the "Linux (x86/GTK 2)" build.

Future bleeding edge builds of Eclipse can be found here:
http://download.eclipse.org/eclipse/downloads/

Unfortunately, the little test I wrote isn't actually in the Eclipse UI - it is actually what we call an "SWT Snippet", and it can be found here:
http://dev.eclipse.org/viewcvs/index.cgi/%7Echeckout%7E/org.eclipse.swt.snippets/src/org/eclipse/swt/snippets/Snippet340.java

Here's a page listing several hundred SWT Snippets, and briefly describing what an SWT Snippet is: http://www.eclipse.org/swt/snippets/
(this page also describes how to set up an Eclipse workspace to run snippets using the SWT standalone download, however please ignore the "import SWT into your Eclipse workspace" link on this page because that is for users of a stable SWT, and what we want is the bleeding edge... <g>)

For debugging this problem, I think it would be best to go straight to the current source code, which can be found in the CVS repository at dev.eclipse.org, in HEAD (which I believe is equivalent to "trunk" in Git).
Here are instructions for setting up your Eclipse workspace to run SWT from CVS:
http://www.eclipse.org/swt/cvs.php
In addition to checking out:
  org.eclipse.swt
  org.eclipse.swt.examples
  org.eclipse.swt.gtk.linux.x86 (I am assuming that this is your platform here)
please also check out:
  org.eclipse.swt.snippets

Once you have ControlExample (steps 6 & 7) up and running from HEAD, then you can run any of the snippets in org.eclipse.swt.snippets/src/
org.eclipse.swt.snippets. The live region one is Snippet340.java.
Comment 2 Carolyn_MacLeod 2010-05-04 21:56:38 UTC
Created attachment 160299 [details]
Snippet340.java with live region code

I am attaching a slightly different version of Snippet340 which has the extra "live region" object attributes added for the label. The reason that the released snippet had this code removed is that I discovered that JAWS was actually speaking the description changes in the "describedBy" relation based on receiving "description changed" events. In other words, it doesn't even need the live region attributes. This is interesting, and apparently it is by design.

Would you consider making Orca do this too? (I guess this would be a separate task from generalizing live region support. Please let me know if i should file a new feature request for this).
Comment 3 Carolyn_MacLeod 2010-05-04 22:03:12 UTC
If you open type (ctrl + shift + T) AccessibleObject and look for a line of code near the top that says:
  static final boolean DEBUG = Device.DEBUG;
then if you change that to:
  static final boolean DEBUG = true;
Then the next time you run the snippet, you will see debug statements in the Eclipse console. This may be useful.
Comment 4 Joanmarie Diggs (IRC: joanie) 2010-05-04 22:13:51 UTC
> Would you consider making Orca do this too?

This is where Orca's custom scripts come in. Would I consider making it the default behavior across the board? Probably not. Would I consider making it the default behavior just for Eclipse? Heck yeah. :-)

Since I'm still DayJobbing (admittedly with a bit of multitasking), I've not had time to read through -- let alone try -- what you've attached here. Sorry! However:

> (I guess this would be a separate
> task from generalizing live region support. Please let me know if i should file
> a new feature request for this).

I agree that this is a separate task. Thus a new RFE would be awesome. Thanks!!

In addition, if you'll attach to the new RFE a full "debug.out"* captured at the time your Snippet340 did it's thing (and Orca ignored it), I might be able to generate a patch to cause Orca to present the description changes.

* http://live.gnome.org/Orca/Debugging. Also, please attach the full file -- as opposed to just the events leading up to the description change -- because I want to see which script we're falling back on.

Thanks again VERY much!
Comment 5 Joanmarie Diggs (IRC: joanie) 2010-05-04 23:45:05 UTC
New summary. :-) I have questions and I might as well ask them here. Thus scratch the new RFE request.
Comment 6 Joanmarie Diggs (IRC: joanie) 2010-05-05 00:36:41 UTC
Looking at this with Accerciser (http://live.gnome.org/Accerciser) in Ubuntu Lucid, here's what I'm seeing:

1. For this particular test case, there are all sorts of (appropriate) accessible events to choose from. Including the "description changed" event mentioned by Carolyn.

2. The name of the accessible application is 'SWT'.

3. The toolkit being reported is GAIL.

Thus, to handle this exact snippet exactly as it exists, we could create a new script called SWT.py make it a subclass of the default script, and then decide on which of the aforementioned events were the most reliable.

But this is merely a test case. Therefore:

<big-picture discussion>

1. Where are we going to see this (live regiony, description-changy) behavior in the wild? Will it always be in a stand-alone application called SWT? Will it instead be part of Eclipse? Will it be in all sorts of applications with all sorts of different names which were projects developed in Eclipse? In other words, if I wanted to isolate presenting these updates to just the stuff Carolyn cares about, how could I reliably do so?

2. Should this even be a live region? Maybe it should, I'm merely tossing out the question, because it's how all of this got started.

My understanding of the purpose behind live regions is that they are a means for users to access dynamically-updating content being displayed in areas which are independent of -- perhaps completely unrelated to -- the focused widget/task. Sports scores, stock prices, news tickers, etc.

Here, on the other hand, the purpose seems to be to provide direct feedback to the user in response to something the user just did, namely input a string in the focused text widget.

</big-picture discussion>

Regardless of the above.... Given the fact that we do have the appropriate describedBy relationship and the necessary events to tell us when this change has occurred, I'm quite tempted to withdraw my statement from comment 4:

> Would I consider making it the default behavior across the board?
> Probably not.

This might indeed be something we would want to do across the board when:

* there is an object:property-change:accessible-description event
* the event is for an editable field
* that field has focus
* we've got a describedBy relationship we can use to get the label
* we've verified that the label contents have actually changed

Yeah, that does sound a tad neurotic. :-) But I prefer to be conservative when making across-the-board changes that might lead to unexpected chattiness where we didn't count on it popping up.

So I'm going to play with making the across-the-board change. And also thoroughly regression-test any changes I think might work. Stay tuned.
Comment 7 Carolyn_MacLeod 2010-05-05 02:57:44 UTC
Thanks, Joanie. Here are 2 examples of where this description-changey behavior occurs in the wild. (i.e. in Eclipse). These places have not yet been made into describedBy relations with nice description changed events, because I needed to be sure that my little test snippet worked ok on multiple platforms before I recommended this approach to the Eclipse UI developers.

Example 1) File->New->Java Project

a) Notice the "Enter a project name" text near the top of the dialog. The current value ("Enter a project name") is not very useful, but watch what happens if you type a bogus project name. Type a dot ('.'). The text changes to a message "'.' is an invalid name on this platform". And there is a red x.  (One could type shift+tab have this spoken, but that's not exactly intuitive).

Example 2) Here's another, similar example. Type ESC to close the previous dialog. Now go to File -> Import...

a) Notice a similar text area at the top of the Import dialog. ("Choose an import source")

b) Type TAB to get to the tree, then type right arrow to expand the General category.

c) Type down arrow a few times, noticing that the text at the top of the dialog changes to give some useful information.

I am happy to just go with the "conditional across-the-board description change" fix. Your 5 * preconditions make sense to me.
Comment 8 Joanmarie Diggs (IRC: joanie) 2010-05-05 03:27:01 UTC
(In reply to comment #7)
> I am happy to just go with the "conditional across-the-board description
> change" fix. Your 5 * preconditions make sense to me.

With one potential snag I'm afraid: In your second example, the label which is changing is not giving direct feedback in response to input into an editable field (i.e. an object with ATK_STATE_EDITABLE and, likely, of ATK_ROLE_{TEXT,PASSWORD,ENTRY}).

If you think that the Example 2 label changes should also be spoken (something which might be worth obtaining user feedback on), my temptation would be to do the following:

1. Implement the across-the-board solution you and I seem to agree upon.

2. Create the new Orca script for Eclipse I've been alluding to remove the "editable field" requirement.

At the moment I'm wiped (I'm on the U.S. East Coast), but I'll have something for you to try by this weekend -- and very likely sooner.
Comment 9 Carolyn_MacLeod 2010-05-05 03:55:55 UTC
Midair collision - it is difficult to be faster than you are. <grin>

I was about to say very nearly the same thing as you (I noticed the snag after commiting comment 7)...
---

Oops - I said "Your 5 * preconditions make sense to me."
Yes, they do, except for this one:
> * the event is for an editable field

Example 2 in comment 7 shows the selection changing in a tree causing the
description property changed events. I guess any type of object that has the
focus could have pretty much any type of user-generated change that the
application might want to comment on using this technique. Do you still think
this is something you would support across the board?

> Should this even be a live region?

If screen readers are ok with supporting "describedBy description changed",
then I prefer not to resort to using live regions for this - you are correct
that this is simpler than full-blown live regions.

> Will it be in all sorts of applications with all sorts of different names
> which were projects developed in Eclipse?

Yep.  :)

---
(In reply to comment #8)

I am not sure that I understand your second point:
> 2. Create the new Orca script for Eclipse I've been alluding to remove the
> "editable field" requirement.

However, I am east coast, too (a tad more north, in Canada), and I am more than willing to leave this until tomorrow. Thanks so much for your incredible customer service!  :)
Comment 10 Joanmarie Diggs (IRC: joanie) 2010-05-05 17:14:56 UTC
(In reply to comment #9)
> Midair collision - it is difficult to be faster than you are. <grin>

Perhaps on commenting on bugs. I think you've got me beat on the turn-around time for fixes. :-)

> Example 2 in comment 7 shows the selection changing in a tree causing the
> description property changed events. I guess any type of object that has the
> focus could have pretty much any type of user-generated change that the
> application might want to comment on using this technique. Do you still think
> this is something you would support across the board?

For the neurotic preconditions I laid out (i.e. not selection changes in trees), at the moment that is indeed still my leaning: Other software packages (and for that matter web sites with forms) seem to be more and more in the habit of using labels to present input errors to the user immediately. Before, the custom seemed to be validate at the end and present a dialog. I have to wonder just how many of those input errors are going unnoticed (at least initially) by Orca users because we're not presenting those changes to the label.

Before I absolutely commit to such a change, however, I need to have time to look at a bunch of different examples, regression test what I'm thinking, consider settings so that users could optionally ignore such announcements, etc. Which is why I suggested I'd aim for doing it this weekend rather than, say, today. :-)

But regardless of whether or not we do it across the board, we can definitely do it in a custom script for Eclipse. Which brings me to:

> > Will it be in all sorts of applications with all sorts of different names
> > which were projects developed in Eclipse?
> 
> Yep.  :)

Okay, so Eclipse and all things which have been developed in Eclipse. Now things get a bit trickier.... :-)

From what you've said and some (very quick, cursory) googling last night, here's my understanding (please tell me if and where I'm wrong):

* Eclipse, and apps created in Eclipse, use the Standard Widget Toolkit

* One of the purposes/features of the Standard Widget Toolkit is that it enables developers to create apps which not only are "write once, run anywhere" but which also behave just like the native toolkit of the user's environment. In the GNOME desktop, that would be Gtk+ (from here on out, I'm ignoring other environments).

* Because SWT apps are supposed to look and act just like Gtk+, they (intentionally or by happenstance) claim that the accessibility toolkit being used is GAIL.

* Because SWT is not actually Gtk+, cool accessibility enhancements made by you in SWT won't necessarily be present in the real, official Gtk+ (a no-brainer, but still an important detail).

Worst case scenario: If for some reason I am not able to implement an across-the-board solution to support what you're proposing, we still want and will support it. However, in order to do so, we would need to be able to identify if we've got an SWT app or not.

If the toolkit being reported is GAIL and if the app name can be anything, we have no way of identifying if we've got an SWT app or not. As a result, we would need to create a custom script for Eclipse, AND a custom script for MyCoolSWTApp, AND a custom script for YourCoolSWTApp, etc., etc., etc. in order to handle your new describedBy feature (again, *if* it turns out that I cannot implement across-the-board support). Under this scenario, any application that we didn't have a script for would fail to respond as expected to your describedBy feature. Even if I can handle describedBy across the board, what nifty new accessibility enhancements might you have up your sleeve? :-) What if I cannot handle *them* across the board?

Therefore, independent of this particular bug/RFE, I'm wondering if perhaps what we need is for SWT apps to claim that their toolkit is SWT rather than GAIL. Then in Orca we'd have a tiny, tiny custom SWT toolkit script which said "hey, pretend this is GAIL." :-) This would allow us to customize how we handle the very, very few differences (accessibility enhancements missing in Gtk+) that we might run across in SWT. It would also mean that all standard SWT apps would behave the same from the perspective of the Orca user -- even in the absense of the YourCoolSWTApp Orca script.

Does this make sense? If so, is it my turn to register as a user of your issue tracker and file an RFE? <grin>

> (In reply to comment #8)
> 
> I am not sure that I understand your second point:
> > 2. Create the new Orca script for Eclipse I've been alluding to remove the
> > "editable field" requirement.

It may be that I've already answered this question through the commentary I just wrote. But just in case I've not.... In a nutshell, Orca works by responding to at-spi events. There is a default script that dictates what happens. For instance, say we get a text-inserted event, the default script does stuff like the following:

A. Is it from the focused widget? (if not, it's irrelevant, so stop processing this event)

B. Did the user just press a mark of punctuation or a space? If so, does the user have a preference to echo words when typing? If so, speak the word to the left of the caret and stop processing the event.

C. Etc., Etc., Etc.

Now consider a chat application. In comes a new message. We get a text-inserted event. That event is for the chat history; not the focused widget. Ergo it's irrelevant. Ergo it's ignored. Oh crap. :-)

So we have custom application (and toolkit) scripts which override the behavior of a default script. A script for a chat application would have its own onTextInserted() which would go something like:

A. Is this a new IM? If so, present it and store it for Orca's chat history feature, then stop processing this event.

B. Is this autocompleted text (i.e. user types 'Jo' and then presses Tab in an IRC chat room and 'Joanie' appears in the editable text area as a result)? If so, present it then stop processing this event.

C. Must be somethin' else. Pass it along to the default script's onTextInserted() for processing.

So getting back to your original question: I do not think that when the selection changes in a tree we should present a change in the label describing that tree. In other words, handling your Example 2 across the board would likely result in unexpected, unwanted "chattiness." But... If you (and Orca+Eclipse users) believe that Orca should indeed present this label change, that's fine by me -- but then we should limit it to just Eclipse or just SWT applications. In order to do so, we might have an onDescriptionChanged() event handler in the default script which presented the label change if and only if my five neurotic requirements were present. In the Eclipse/SWT script, we'd override onDescriptionChanged() to eliminate the 'editable field' requirement.

Make sense?
Comment 11 Joanmarie Diggs (IRC: joanie) 2010-05-22 20:12:06 UTC
Going through all of the open Orca bugs and RFEs, it turns out we already have an open RFE for describedBy. As a general rule, we tend to mark newer bugs as dups of the older ones. So I'm going to do that here. When we fix bug 568611, any describedBy changes in SWT widgets should JustWork(tm).

*** This bug has been marked as a duplicate of bug 568611 ***