First coarse pass at GNOME 2.24 planning.

I don't see an easy way to do this. 

There is a method in default.py called findDisplayedLabel 
(sic) that returns a list of the objects that are 
labelling the given object.

That in turn is used by getDisplayedLabel in default.py
to append all the strings together, returning a single 
string.

getDisplayedLabel is used in multiple (20) places in     
braillegenerator.py, once in default.py to determine
if an object is for layout purposes only and once in 
liveregions.py. It is used in three places in 
speechgenerator.py and once in where_am_I.py.

In speechgenerator.py, the three places are:

1/ The _getSpeechForObjectLabel() method.
2/ The _getSpeechForTerminal() method.
3/ In a while loop in the getSpeechContext() method.

The _getSpeechForObjectLabel() method is called in
14 places in speechgenerator.py, for various types
of object.

I cannot think of a straight forward way to adjust 
this code, so that if there are two or more labels
labelling the given object, speaking/brailling
them will take into account the labels extents and
therefore be before or after the speaking/brailling 
of the given object.

I'd gladly be proven wrong.

Anybody have any ideas?

(In reply to comment #2)
> I don't see an easy way to do this. 

I'm not sure it will be easy.  :-(

> There is a method in default.py called findDisplayedLabel 
> (sic) that returns a list of the objects that are 
> labelling the given object.

As a thought, perhaps this could be modified to return two lists of objects: one for the labels to the left of the object and one for the labels to the right of the object?  While doing this, you could change the name to findDisplayedLabels so it won't sic'en you anymore.  :-)

> That in turn is used by getDisplayedLabel in default.py
> to append all the strings together, returning a single 
> string.

With the above change, this could be modified to return two strings - one for the left and one for the right.

I realize this would touch a lot of code, but I think this is a very useful feature to have.

Thoughts?  Should we also try to consider labels above/below as well?

Oh, I realize it could be done. It just won't be 
easy and straight forward. The above was just an
attempt to evaluate it to see if it was going to
be a quick fix.

Your approach above is certainly feasible.

My thoughts are that this is a potentially risky 
change at this stage of the development cycle, for 
a small benefit, and as this is currently marked 
as Normal/Enhancement, perhaps we should concentrate
on more important bugs.

Thanks for looking at it.

> As a thought, perhaps this could be modified to return two lists of objects:
> one for the labels to the left of the object and one for the labels to the
> right of the object? 

I like this idea.  And it might buy us something else -- or it might introduce something else we need to work out -- though I'm not sure, so....

Consider the following example:

   Margin: <spin button> pixels

Where "Margin:" and "pixels" are each labels for the text object <spin button>.

Right now for English, we say something like "Margin: pixels 10 spin button."  But what happens in rtl languages like Arabic and Hebrew?  Does some miracle of localization cause the dialog for those users to display (essentially)

    pixels <spin button> Margin:

? And, if so, are we then saying (essentially) "pixels <spin button> Margin" to users of those languages?

Knowing the answer to the above would help me sort out my next question, but I'll ask it now anyway:  If we build up speech context to present:

1. Label on the left
2. Object being labeled
3. Label on the right

What happens to the aforementioned users?

If right now some localization miracle is solving the direction problem, I would want to be sure we're not screwing it up by this change.  On the other hand, if right now the way in which we're speaking labels and the objects being labeled is backwards for those users, maybe there's something clever we can do to get things in the right order as we look at the labels' locations spatially??

(Am I making any sense?)

> Thoughts?  Should we also try to consider labels above/below as well?

If we're currently grabbing labels which are above/below (which I assume we are), then I would think we'd also want to consider their locations.

Created attachment 113055 [details]
Orca Preference dialog in Arabic.

The Speech pane of the Orca Preferences dialog, showing how the
spin button with two labels (near the bottom of the pane) is
rendered.

In discussion with Mike, we decided that this problem is not encountered enough to justify the significant change for 2.24.  So, marking this for FUTURE and we probably should get to it early in the 2.26 cycle.

For future reference:

> If right now some localization miracle is solving the direction problem, I
> would want to be sure we're not screwing it up by this change.  

Just a reminder that what we're doing now isn't the greatest -- we're just concatenating things in the order we find them in list of relations.  It's kind of a crap shoot if they come out in the right order or not.  The probability, however, is that the ordering will be correct.

> hand, if right now the way in which we're speaking labels and the objects being
> labeled is backwards for those users, maybe there's something clever we can do
> to get things in the right order as we look at the labels' locations
> spatially??

LTR and RTL concerns are definitely valid (thanks for thinking of this!).  So, instead of left/right lists, maybe we think of the values as before/after lists, where the before/after represent the logical reading order.  This could still be done in findDisplayedLabels, thought it would need some magic in it to determine what before and after meant.  That magic might be spacial layout coupled with knowledge about the layout direction for the window (if we can get it).

There's some algorithm here that could be used to use the text of the label to predict LTR vs. RTL: http://osdir.com/ml/gnome.gtk+.internationalization/2005-08/msg00033.html.  Though, it might be better to try to use something else such as app.getLocale(pyatspi.LOCALE_TYPE_MESSAGES) to get the LANG of the app.  From here, we might be able to infer the layout direction for the window.

For more future reference:

The decision about defaulting to RTL instead of LTR seems to be a human one, and it is done by the translators.  In do_post_parse_initialization of gtkmain.c, it looks for the translated value of "default:LTR":

  /* Translate to default:RTL if you want your widgets
   * to be RTL, otherwise translate to default:LTR.
   * Do *not* translate it to "predefinito:LTR", if it
   * it isn't default:LTR or default:RTL it will not work 
   */
    char *e = _("default:LTR");
    if (strcmp (e, "default:RTL")==0) 
      gtk_widget_set_default_direction (GTK_TEXT_DIR_RTL);
    else if (strcmp (e, "default:LTR"))
      g_warning ("Whoever translated default:LTR did so wrongly.\n");
  }

I'm not sure of the best way for us to find this.  It might be that we should request that 'direction' be exposed via an attribute to obtain via acc.getAttributes() and acc.queryText().getAttributes() (the first for how the widget wants to layout its children, the second for what the actual text is doing at a particular offset).

See also bug #568611, which discusses the DESCRIBED_BY relation we have yet to include.

As an alternative to two lists, we could still return a single list, but have a "None" item that represents a placeholder for the object being labeled.

With work on 570658 and 584529  being done, it might be better to have before, after, above and below, as separate entities, enabling us to put them as elements of the formatting strings.

(3.0 Planning Spam-o-rama. Sorry!)

Note that I am planning (hoping) to have bug 622515 fixed for 3.0.