After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 535221 - Automatic shortcut information spoken, for example with Jaws screen reader
Automatic shortcut information spoken, for example with Jaws screen reader
Status: RESOLVED FIXED
Product: orca
Classification: Applications
Component: speech
2.23.x
Other All
: Normal enhancement
: 2.26.0
Assigned To: Willie Walker
Orca Maintainers
Depends on:
Blocks:
 
 
Reported: 2008-05-28 09:15 UTC by Hammer Attila
Modified: 2009-03-10 14:25 UTC
See Also:
GNOME target: ---
GNOME version: Unversioned Enhancement


Attachments
This is a modifyed speechgenerator.py file. (66.01 KB, text/plain)
2008-12-09 13:19 UTC, Hammer Attila
  Details
svn diff form of Attila's file (1.17 KB, patch)
2008-12-09 21:45 UTC, Willie Walker
none Details | Review
This is the second release this function. Works with all important control element, tested with Orca application preferences GUI window. (4.15 KB, patch)
2008-12-10 12:09 UTC, Hammer Attila
none Details | Review
This is the third release the automatic shortcut spoken feature. The differences see the comment. Working fine. (9.20 KB, patch)
2008-12-10 15:41 UTC, Hammer Attila
none Details | Review
This is a 4. revision of this feature. Doed fixes see the comment. (9.04 KB, patch)
2008-12-12 08:59 UTC, Hammer Attila
none Details | Review
Slightly revised patch that pylints (but doesn't regression test well) (9.04 KB, patch)
2009-01-13 20:18 UTC, Willie Walker
needs-work Details | Review
This is the next release of this patch. Bug fixes see the comment. (11.42 KB, patch)
2009-01-14 13:14 UTC, Hammer Attila
none Details | Review
A patch that seems to regression test well and provide close to what Attila wants (17.38 KB, patch)
2009-01-14 19:08 UTC, Willie Walker
none Details | Review
Patch to use the full 'Alt+x' form for non-menu objects (17.35 KB, patch)
2009-01-15 16:39 UTC, Willie Walker
reviewed Details | Review
Final patch that was committed. (16.21 KB, patch)
2009-01-17 02:23 UTC, Willie Walker
committed Details | Review

Description Hammer Attila 2008-05-28 09:15:44 UTC
Dear Orca Developer!

All dialog controls, menu items have a shortcut keys, but Orca not spoken this information automaticaly. I know, the where am I command spoken this information, but automatic spoken is better and easyest the navigation with dialogs, menus etc.
For example in Jaws For Windows screen reader this is a default service.
Possible doing in Orca preferences/speech page with automatic say shortcut information check box?
If this check box is checked, orca spoken this shortcut information after finished spoken the dialog control type and menu item text. Some example:
If the user using Orca preferences gui window, all controls have a shortcut information.
Enable speech check box checked. alt e
For example with menus in gedit application:
The user pressing f10 key. In first menu item spoken (file menu), but the user not known what the file menu shortcut key. If this check box is checked, this spoken text modifyed:
File menu. alt f
The user pressing right arrow:
Edit menu. alt e
User pressing enter key or down arrow.
Undo u
Cut c
copy p etc.

This automatic shortcut key setting more extend Orca accessibility. Orca is a very good, flexible screen reader.

Possible doing this feature request with final gnome2-24 Orca version?

Thanks your ansvers, and you doing this feature request, all blind people are very happy my openion.

Attila
Comment 1 Willie Walker 2008-06-03 20:01:00 UTC
We'll look at this when we start investigating the various verbosity levels.
Comment 2 Hammer Attila 2008-07-12 07:06:55 UTC
I tryed analyze the speechgenerator.py file with following function:
    def _getSpeechForCheckBox(self, obj, already_focused):
        """Get the speech for a check box.  If the check box already had
        focus, then only the state is spoken.

        Arguments:
        - obj: the check box
        - already_focused: False if object just received focus

        Returns a list of utterances to be spoken for the object.
        """

        utterances = []
        state = obj.getState()
        if state.contains(pyatspi.STATE_INDETERMINATE):
            # Translators: this represents the state of a checkbox.
            #
            checkedState = _("partially checked")
        elif state.contains(pyatspi.STATE_CHECKED):
            # Translators: this represents the state of a checkbox.
            #
            checkedState = _("checked")
        else:
            # Translators: this represents the state of a checkbox.
            #
            checkedState = _("not checked")

        # If it's not already focused, say it's name
        #
        if not already_focused:
            label = self._getSpeechForObjectLabel(obj)
            utterances.extend(label)
            name = self._getSpeechForObjectName(obj)
            if name != label:
                utterances.extend(name)
            if obj.getRole() == pyatspi.ROLE_TABLE_CELL:
                utterances.extend(
                  self._getSpeechForObjectRole(
                    obj, pyatspi.ROLE_CHECK_BOX))
            else:
                utterances.extend(self._getSpeechForObjectRole(obj))
            utterances.append(checkedState)
            utterances.extend(self._getSpeechForObjectAvailability(obj))
        else:
            utterances.append(checkedState)

        self._debugGenerator("_getSpeechForCheckBox",
                             obj,
                             already_focused,
                             utterances)

        return utterances

I understand this code, but how can I doing Orca spoken automaticaly the underlined letter for check boxes, menus, etc?
In where_am_I.py script this function is modifyed:
        utterances = []
        text = self.getObjLabelAndName(obj) + " " + \
               rolenames.getSpeechForRoleName(obj)
        text = text + " " + self._getCheckBoxState(obj)
        utterances.append(text)

        accelerator = self._getObjAccelerator(obj)
        utterances.append(accelerator)

        debug.println(self._debugLevel, "check box utterances=%s" \
                      % utterances)
        speech.speakUtterances(utterances)

I tryed insert this two lines with speechgenerator.py, the outed function:
        accelerator = self._getObjAccelerator(obj)
        utterances.append(accelerator)

But not happen nothing, the underlined letter not spoken.
Can you help me? I would like doing this function. :-):-)

Attila
Comment 3 Willie Walker 2008-07-14 20:35:08 UTC
Hi Attila:

> I understand this code, but how can I doing Orca spoken automaticaly the
> underlined letter for check boxes, menus, etc?
> In where_am_I.py script this function is modifyed:
>         utterances = []
>         text = self.getObjLabelAndName(obj) + " " + \
>                rolenames.getSpeechForRoleName(obj)
>         text = text + " " + self._getCheckBoxState(obj)
>         utterances.append(text)
> 
>         accelerator = self._getObjAccelerator(obj)
>         utterances.append(accelerator)
> 
>         debug.println(self._debugLevel, "check box utterances=%s" \
>                       % utterances)
>         speech.speakUtterances(utterances)
> 
> I tryed insert this two lines with speechgenerator.py, the outed function:
>         accelerator = self._getObjAccelerator(obj)
>         utterances.append(accelerator)
> 
> But not happen nothing, the underlined letter not spoken.
> Can you help me? I would like doing this function. :-):-)

First of all -- yeah!  We'd love to keep growing our group of competent Orca developers.

The problem you're running into is that the _getObjAccelerator method is in the where_am_I.py module and not the speechgenerator.py module.

So...if one were to look into working in this space, they might consider moving where_am_I.py:_getObjAccelerator to default.py:getObjAccelerator and then calling self._script.getObjAccelerator from both where_am_I.py and default.py.

Also, the choice to output the information should be based upon a settings.py boolean so we can allow users to decide whether or not they want to hear this information.
Comment 4 Hammer Attila 2008-07-15 06:57:56 UTC
Dear Will!

You wroted:
"First of all -- yeah!  We'd love to keep growing our group of competent Orca
developers.
 
The problem you're running into is that the _getObjAccelerator method is in the
where_am_I.py module and not the speechgenerator.py module."
Ok, understand. :-):-)
 
"So...if one were to look into working in this space, they might consider moving
where_am_I.py:_getObjAccelerator to default.py:getObjAccelerator and then
calling from both where_am_I.py and default.py."
Copy _getObjAccelerator function from where_am_I.py to default.py, and paste self._script.getObjAccelerator line with where_am_I.py and default.py scripts?
Sorry my beginner question, but I not see whole the script codes. 

How can I doing first with orca-customizations.py for example?

Attila
Comment 5 Willie Walker 2008-07-15 12:46:55 UTC
Hi Attila:

(In reply to comment #4)
> How can I doing first with orca-customizations.py for example?

I think the best thing is to jump right into the code and skip trying to do this with orca-customizations.py.  The user-settings.py and orca-customizations.py files really are intended mostly for modifying settings.py values.

Give it a shot, Attila!  You can do it!  :-)
Comment 6 Hammer Attila 2008-08-05 16:56:28 UTC
I trying analyze John tutorial message patch and settings.py, but I not promise this feature in august 18 (UI freeze).

Attila
Comment 7 Hammer Attila 2008-12-09 12:48:10 UTC
I played the getObjAccelerator function with speechgenerator.py.
I seeing query this function the keybinding with getkeybinding function with a result variable. 
When the result variable length >0, the accelerator variable get result[2] value (for example with Ctrl+x etc).
I inserting an else case, and writing utterances.append(result[0])
And final, I insert the self._script.getobjaccelerator(Obj) with getDefaultSpeech() function.
The result:
Orca spokens menus acceleration key information (for example: accessoryes a
Not work full perfect, spokened acceleration shortcut two times.
I sending an attachment with modifyed speechgenerator.py file, because i would like known: I following right way this function?

Problems my method: If an object not have ctrl+accelerator combination, the normal mnemonic information spokened two time (for example ctrl+x combination).
I not doed another object acceleration spokening (check boxes, radio buttons), only working now menues and buttons shortcut spokening.

The final purpose:
- The shortcut information spokened after the tutorial message 
if enabled the tutor message spokening, not before.

- If a menu have accelerator key and mnemonic key (ctrl+x for example the accelerator and c the mnemonic for example the cut menu item), the accelerator spokening after the menuitem text, and the mnemonic spokening after the tutor message, similar with where_am_I function.

Attila
Comment 8 Hammer Attila 2008-12-09 13:19:41 UTC
Created attachment 124266 [details]
This is a modifyed speechgenerator.py file.

I modify two functions: _getObjAccelerator and getDefaultSpeech.
I don't want now writing a patch, because I don't know I doing right this function good.
Will, if you have a little time, possible look this way?

Attila
Comment 9 Willie Walker 2008-12-09 21:45:27 UTC
Created attachment 124309 [details] [review]
svn diff form of Attila's file

Creating a patch is really easy.  Here's what I did to create the attached patch after modifying speechgenerator.py:

   svn diff > patch.535221-1

The "patch.535221-1" is just a filename convention we use.  It says this is revision 1 of a patch for bug #535221.  I'll take a look at this soon.  The accelerator and mnemonic space tends to be a bit fragile, though, so it'll take some thorough testing with OOo, GTK+, Firefox, Java, etc.
Comment 10 Hammer Attila 2008-12-10 12:09:07 UTC
Created attachment 124340 [details] [review]
This is the second release this function. Works with all important control element, tested with Orca application preferences GUI window.

I don't known what  elegant method choosing: if the verbosity level setted up verbose only spoken the shortcut information this function and if setted up brief not spokening, or make a checkbox this function in settings page, independent the verbosity setting similar with tutor message check box?
Can anybody would like test this patch with different applications, different situations?
Comment 11 Hammer Attila 2008-12-10 15:41:56 UTC
Created attachment 124354 [details] [review]
This is the third release the automatic shortcut spoken feature. The differences see the comment. Working fine.

This is the third release this automatic shortcut spoken feature. The differences with second release:
- I doed a check box with Orca preferences GUI/speech page with speak shortcuts automaticaly label, this check box toggles this function with user needs. If the check box is checked, the shortcut information spokened all important controls, if not, Orca works oldest method.
Will, I would like thank you your instructions.
If need any fix this function, please write with comment.

Attila
Comment 12 Hammer Attila 2008-12-12 08:59:15 UTC
Created attachment 124498 [details] [review]
This is a 4. revision of this feature. Doed fixes see the comment.

I doed following fixes:
1. If the result[1] is empty, getobjaccelerator function returns result[0] value. This fix enables spokening application menu shortcuts (alt+F1 with GNOME, for example accessories a etc).
I found one mistake, but is not my patch problem: When only letter shortcut have for example with hungarian applications menu, Orca spoken this string:
Egyéb menü e
The patch is work right, but betveen the ü and e letter have heared üe string, not have little delaied silence betveen the end of the menu item and shortcut.
I tryed added the . and space character after the menu item text, but not solved this problem.

Will, have any ydea this problem?

Possible put this function the 2.26 version of Orca if we have'nt found another problems with my patch?
I would like doing this function final before the UI freeze.
You look the last patch if have a little time?

Attila
Comment 13 Hammer Attila 2009-01-01 12:23:14 UTC
Urgent if possible: Possible anybody look the last attached patch? 

If possible, please write anybody what important fixes needed this function. I would like doing this function with 2.26 release of Orca because this feature is niceful if works great.

I hope little fixes needed.

Attila
Comment 14 Hammer Attila 2009-01-13 09:15:50 UTC
Will, please look my last patch and tell me possible put Orca 2.26 release of this function, or future version because need lot of work this function. Because UI freeze is very near.

Attila
Comment 15 Willie Walker 2009-01-13 15:05:21 UTC
This is in my queue, so no need to keep pinging me on it.  :-)  Thanks!
Comment 16 Hammer Attila 2009-01-13 15:31:35 UTC
Sorry, please don't angry me. 

What your openion? similar this last patch I do automatic item index spokening for list views, radio buttons etc?
The function automatic spokening for example in a list view how many element in the list, and how many element selected. For radiobuttons, how many buttons are the radio group and what the actual position index.
This is useful function?

Attila
Comment 17 Willie Walker 2009-01-13 15:45:19 UTC
(In reply to comment #16)
> Sorry, please don't angry me. 

No problem - I appreciate your work!  I'm just letting you know that I will look at it very soon, but not right this moment.  Thanks!
Comment 18 Willie Walker 2009-01-13 20:18:35 UTC
Created attachment 126372 [details] [review]
Slightly revised patch that pylints (but doesn't regression test well)

Here's a slightly modified version of Attila's patch.  It seems to work OK, but I have three main questions/comments:

1) We always used to speak the accelerators of menu items.  With this patch, we now only speak them if the new checkbox is checked.  Is this an issue?

2) I'm puzzling over what this piece of code does in speechgenerator.py:

+        else:
+            if result[1] == "":
+                utterances.extend(". " + result[0])
+            else:
+                utterances.append(result[1])

Attila, can you provide some insight?

3) Some of the other subclassed speech generators (e.g., the ones for soffice and Gecko) might need similar treatment to make sure they honor the setting as well.

Thanks!
Comment 19 Willie Walker 2009-01-13 21:41:38 UTC
Adding Mike for his opinion on the loss of the default speaking of accelerators for menu items.  The automatic speaking of accelerators for menu items has been in Orca for about 3 1/2 years, so clumping it with the brand new on/off switch for automatic accelerator/shortcut speaking (which is now off by default) is a concern.  Mike, what are your thoughts?
Comment 20 Mike Pedersen 2009-01-13 21:52:52 UTC
Personally I believe that these are two separate pieces of verbosity and should not be combined.  I'm OK with a new setting to be added with a default to off. 
Comment 21 Willie Walker 2009-01-13 21:57:24 UTC
(In reply to comment #20)
> Personally I believe that these are two separate pieces of verbosity and should
> not be combined.  I'm OK with a new setting to be added with a default to off. 

Are you OK with losing the spoken accelerator information for menu items if the new setting value is off?  Or, are you saying you want to keep the spoken accelerator information for menu items as it has always been?
Comment 22 Mike Pedersen 2009-01-13 22:21:41 UTC
I think we should keep the spoken accelerator information spoken as it has always been. 
Comment 23 Hammer Attila 2009-01-14 07:30:49 UTC
Will, you wroted:
"I have three main questions/comments:

1) We always used to speak the accelerators of menu items.  With this patch, we
now only speak them if the new checkbox is checked.  Is this an issue?"
Yes, because if this checkbox is checked, Orca says the accelerator keys for menues (Ctrl+Shift+t for example) and if not have an accelerator key but have mnemonic key (for example Alt+f in file), Orca says this informations. Oldest method this mnemonic information not spokening all objects, but accelerator information spokening for menues (for example Ctrl+Shift+t).

"2) I'm puzzling over what this piece of code does in speechgenerator.py:

+        else:
+            if result[1] == "":
+                utterances.extend(". " + result[0])
+            else:
+                utterances.append(result[1])

Attila, can you provide some insight?"
Some situations the result[1] string is empty, for example the applications menu items (Alt+f1 menues: accessories, sound and video, places menu items etc).
The result[0] return this missing mnemonic keys for this special situation.
My patch bug: if the user going for example the places menu and moving menu items or a pulling down main menu (for example the file menu items all applications), my patch spokening main menu shortcut and after this the actual menu item mnemonic key (for example the open menu spokened shortcut is alt+f shortcut and after this the o mnemonic letter). This is not good, if the actual rolename a menu item, only enough spokening the accelerator key information (if have) and themnemonic key (for example Ctrl+o for open menu item).

So, what can I change?
Sorry my bad english.

Attila
Comment 24 Hammer Attila 2009-01-14 13:14:35 UTC
Created attachment 126419 [details] [review]
This is the next release of this patch. Bug fixes see the comment.

1. Fixing the submenu bug. For example if the user select a submenu, prewious version of patch says the main menubar shortcut information and the actual mnemonic key, this is unneeded, if situation only spokening the mnemonic key.

2. Main menubar shortcut does'nt contains the alt word, because Jaws For Windows 9.0 says only the mnemonic key with main menu.

3. If the actual menu item have an acceleration key and mnemonic key, first saying the acceleration key (for example Ctrl+Shift+t), and after this spokening the mnemonic key.

If have another suggestions, please write comment.

Attila
Comment 25 Willie Walker 2009-01-14 19:08:33 UTC
Created attachment 126444 [details] [review]
A patch that seems to regression test well and provide close to what Attila wants

This patch fixes the regression where menu object accelerator information was being omitted.  It also clarifies things in the code -- this bug is really about speaking the mnemonic for an item.

Attila, the main difference between this and your patch, however, is that the accelerator is placed *after* the mnemonic in the event there is an accelerator and a mnemonic (i.e., in the case of menu items).  Is this OK with you?
Comment 26 Hammer Attila 2009-01-14 19:30:14 UTC
Will, I think yes. Tomorrow I will be trying the last patch and writing comment.

Thank you.

Attila
Comment 27 Hammer Attila 2009-01-15 09:00:58 UTC
Trying latest version of this function.
I found some realy little problems:
1. If the verbosity is set to brief, Orca does'nt say accelerator key combinations (for example Ctrl+c, Ctrl+x etc). If the verbosity level setted verbose, says correct.

2. When checked the check box and reloading the preferences, the mnemonics sayes correct all verbosity level, but found a little problem.
The menues, submenues, radio and check menues mnemonic say is right, enough the mnemonic letter.
But not right another dialog controls my openion need the alt tag before the mnemonic letter, because where am I command says this form. Some trivial example what problem reproducable when the user try pressing only the mnemonic letter with controls (not menues, submenues etc, only the normal dialog controls):
When we going the Orca preferences and tabbing the keyboard laiout radiobutton group, Orca now says only the mnemonic letter the radiogroup elements. When I trying change the value desktop to laptop only mnemonic letter, not work, only work whwen the user pressing alt+mnemonic letter.
Another example:
the disable Orca main window check box mnemonic letter in hungarian translated version is k. When I tabbed the check box and trying type k letters, no happen nothing. When I press Alt+k, the checkbox is changed (checked or unchecked right).

So, need modification with following controls:
-Check box
- Radio button
- Edit box
- Button
- Toggle button
- Combo box
Etc.

Attila
Comment 28 Willie Walker 2009-01-15 16:39:41 UTC
Created attachment 126511 [details] [review]
Patch to use the full 'Alt+x' form for non-menu objects

(In reply to comment #27)
> Trying latest version of this function.
> I found some realy little problems:
> 1. If the verbosity is set to brief, Orca does'nt say accelerator key
> combinations (for example Ctrl+c, Ctrl+x etc). If the verbosity level setted
> verbose, says correct.

The current design is that accelerators are currently only spoken at the verbose level.  At some point, we will address the verbosity granularity in Orca, but that will be done in a separate bug.

> But not right another dialog controls my openion need the alt tag before the
> mnemonic letter, because where am I command says this form. Some trivial
> example what problem reproducable when the user try pressing only the mnemonic
> letter with controls (not menues, submenues etc, only the normal dialog
> controls):

This newly attached patch should address this (I hope!).
Comment 29 Hammer Attila 2009-01-16 09:10:45 UTC
Looking the last attached patch, I think this function is done. All previous found problems is fixed.
Little suggestions, but not need doing if you would'nt like doing, because not big problems:
Now, the function says the required informations when the check box is checked. this order after all informations spokened before the shortcut informations:
1. The mnemonic information (if the user activating a menu only the letter, non menus the alt tag and letter). This is good.
2. Iff have the acceleration key (Ctrl+x, Ctrl+c etc) if the verbosity setted up verbose. This is good.
3. The tutorial message if tutorial checkbox is checked.

If you would like need modify, possible modify this order with:
1. Iff have the acceleration key (Ctrl+x, Ctrl+c etc) if the verbosity setted up verbose.
2. The tutorial message if the enable tutorial checkbox is checked.
3. The mnemonic letter (if the user activating any menu only the letter, else the alt tag and the mnemonic letter.

If you would'nt like modify the patch, this bug is fixed, please put with trunk age.

I would like thank every body help doing this function, very nice feature.

Attila
Comment 30 Willie Walker 2009-01-17 02:23:00 UTC
Created attachment 126631 [details] [review]
Final patch that was committed.

I made a small adjustment to the last patch.  Attila - I didn't get a chance to do your last minor request, but I think we can be safe and call this bug done.  Yeah!  Thanks for your work.