GNOME Bugzilla – Bug 354470
] Contracted Braille
Last modified: 2008-07-22 19:33:53 UTC
Orca should support grade 2 braille and provide an option for turning it on and off. (NOTE: this needs to be written up better in the Orca doc set).
Tracking.
Add accessibility keyword. Apologies for spam.
Allow grade-1 and grade-2 to be switched quickly using the buttons on the braille display.
From Michael Whapples on the Orca users list: http://mail.gnome.org/archives/orca-list/2007-April/msg00285.html "For doing the contracted braille in orca, it could use something such as liblouis (www.jjb-software.com). This can do braille translation in both directions, has tables for a few braille codes, and also is open source. I have used it in some of my own projects written in python, and it seems to work fine. You would need to write a python API for it, I have done that before using ctypes. The one problem I did have when interfacing with ctypes, which is relevant to mention here, is that if someone has compiled python from source with 32-bit unicode then you will need to convert to 16-bit unicode as that is what liblouis uses. I don't know of any python function to check if python is using unicode32 or unicode16, but I managed to do it by comparing the results from len and sizeof from a ctypes unicode buffer (create_unicode_buffer), the relationship should be a multiple of 2 if unicode16 is used and 4 if unicode32 is used (one gives the number of characters and the other gives number of bytes and then you can calculate number of bytes per character). I may be making more of this than is necessary, but I have found this to be so in one distro of Linux I have used."
Assigning to Eitan as part of the speech/braille refactor.
Created attachment 102075 [details] [review] Proposed UI changes This patch adds a "contracted braille" frame to the preferences interface. I made this so it could be committed into trunk without the options actually being exposed by default. To review the interface simply run the file orca_gui_prefs.py as standalone. All of the new strings have been properly marked for translation.
I just tested this and I'm fine with the strings and layout if everyone else is.
Looks cool to me, though I have a couple comments: 1) It would be much better if we got the braille contraction table strings from the liblouis bindings rather than hard coding them into our stuff. Is this something you plan on doing before the final commit? 2) Mnemonics should be used for the new checkbox and combobox. I'm not sure of great ones, but how about the "E" in Enable and the "T" in Table? I'm also assuming that the default braille contraction table will be chosen based upon the current locale? In any case, it will be great to have this for GNOME 2.22, so I'm psyched to see progress on this. Thanks!
(In reply to comment #8) > Looks cool to me, though I have a couple comments: > > 1) It would be much better if we got the braille contraction table strings from > the liblouis bindings rather than hard coding them into our stuff. Is this > something you plan on doing before the final commit? > You mean the "U.S. English Grade 2" part? I see a couple of problems there: * There is no way in liblouis today to list tables or extract a description out of a table, liblouis simply uses the file that you point at it. * Liblouis currently is not localized, so even if it could give human readable names for the tables, they would always be in English. * Liblouis is not a GNOME module, so it will never get as much translation as a GNOME module gets, assuming it becomes localizable. So that hard coding is ugly, but I figured we should get as much strings as possible in before the freeze. Is there another approach you would suggest? > 2) Mnemonics should be used for the new checkbox and combobox. I'm not sure of > great ones, but how about the "E" in Enable and the "T" in Table? > Yeah, I'll add those. > I'm also assuming that the default braille contraction table will be chosen > based upon the current locale? Yep, sounds right.
> * Liblouis currently is not localized, so even if it could give human readable > names for the tables, they would always be in English. > * Liblouis is not a GNOME module, so it will never get as much translation as a > GNOME module gets, assuming it becomes localizable. Ah...bummer. Yeah, my thought was to put the strings in the Python bindings for liblouis and defer the translation problem. A possible interim workaround to this might be to put the liblouis Python bindings under the orca module (e.g., parallel to the brl directory). With that, the translations for the tables would happen as part of the Orca translation, but be more tightly coupled with the liblouis stuff. Do you know what the status of liblouis is these days? It is still under development? Does it need a home for source code control and bug management? If so, do you think the maintainers might be amenable to proposing it as a module for GNOME 2.24?
Yea contracted braille!!! This is going to be awesome to have! Seeing as how we're trying to get in strings before the freeze.... A setting that other screen readers have is to expand/uncontract the word that's at the caret. I think this is handy because sometimes you want to see how something is spelled. (As a related aside, I have always wanted to do a study of highly literate folks and compare the spelling ability of braille readers versus print readers.) Anyhoo, is this a feature we want to include and if so, should it be added to the prefs dialog now?
> Seeing as how we're trying to get in strings before the freeze.... A setting > that other screen readers have is to expand/uncontract the word that's at the > caret. I think this is handy because sometimes you want to see how something > is spelled. I've never found this setting to be all that useful because I think any braile user will normally want to see the word under the caret expanded. This is currently the intended plan. That said, if you really think we need the feature I'm fine with it.
(In reply to comment #10) > Ah...bummer. Yeah, my thought was to put the strings in the Python bindings > for liblouis and defer the translation problem. A possible interim workaround > to this might be to put the liblouis Python bindings under the orca module > (e.g., parallel to the brl directory). With that, the translations for the > tables would happen as part of the Orca translation, but be more tightly > coupled with the liblouis stuff. John offered to have the Python bindings ship with liblouis. I thought that was great news, but I am thinking now that perhaps it would be better if they were in Orca for now. Liblouis does not have any schedule commitments, while Orca does. Not sure, we need to make these decisions quickly so that the distros don't get too confused. > > Do you know what the status of liblouis is these days? It is still under > development? Does it need a home for source code control and bug management? I am in contact with John, so far it needs all of that, I suggested to him sf, launchpad or the google code thing. > If so, do you think the maintainers might be amenable to proposing it as a > module for GNOME 2.24? > I'm not sure if that would be the right decision for liblouis, it would require a certain commitment from the maintainers, scheduled releases, etc. Plus liblouis has more uses than just screen readers, for example embossers.
Created attachment 102154 [details] [review] Proposed UI changes This patch adds the mnemonics. Could I submit this to trunk?
(In reply to comment #12) > > Seeing as how we're trying to get in strings before the freeze.... A setting > > that other screen readers have is to expand/uncontract the word that's at the > > caret. I think this is handy because sometimes you want to see how something > > is spelled. > I've never found this setting to be all that useful because I think any braile > user will normally want to see the word under the caret expanded. This is > currently the intended plan. That said, if you really think we need the > feature I'm fine with it. > I'm fine either way too. My thought now is to put down as many strings as possibly needed, so if we think we might want it, I'll put it in. Worst case we could take it out, I think it is better than the other way around, it's better if we have one extra translated string than a string that we use but is not as widely translated.
> I've never found this setting to be all that useful because I think any braile > user will normally want to see the word under the caret expanded. This is > currently the intended plan. I didn't realize it was the intended plan. I *believe* in other screen readers the default is not to expand it (which I think is silly). I just want it to be expanded <smile>. So never mind my suggestion.
(In reply to comment #14) > Created an attachment (id=102154) [edit] > Proposed UI changes > > This patch adds the mnemonics. > Could I submit this to trunk? I'm OK with committing this for 2.21.5. I still wish we could figure out a way to keep Orca from needing to maintain knowledge of the liblouis table strings. Those really should come from liblouis. The comments for translators should probably include some reference to what "Grade 1", "Grade 2", and "Grade 3" refer to. A pointer to http://en.wikipedia.org/wiki/Braille might be sufficient. > John offered to have the Python bindings ship with liblouis. I thought that was > great news, but I am thinking now that perhaps it would be better if they were > in Orca for now. Liblouis does not have any schedule commitments, while Orca > does. How about we go for keeping the bindings with Orca for now and migrate them out to a more 'public' space after we've had a chance to work with them and let them mature? With this, we could also put the table name strings in the Python bindings, yet still benefit from the GNOME translations. Thoughts? > I am in contact with John, so far it needs all of that, I suggested to him sf, > launchpad or the google code thing. > > > If so, do you think the maintainers might be amenable to proposing it as a > > module for GNOME 2.24? > > > > I'm not sure if that would be the right decision for liblouis, it would require > a certain commitment from the maintainers, scheduled releases, etc. > Plus liblouis has more uses than just screen readers, for example embossers. Sounds fair enough. I guess treating it similar to how we treat brltty is fine (i.e., if it's there, we use it. if it's not, we don't). I think the main end goal is just to get liblouis in a more 'open source' state so it can evolve more easily and we can track bugs and enhancement requests.
Lets not include the expand current word checkbox in the UI. I really think users want the word containing the caret expanded so this checkbox will just be extra UI clutter to tab through.
Created attachment 102613 [details] [review] UI changes plus new louis module This is what I am committing for the 2.21.5 release.
Thank you! Please make sure to send out a string change announcement (see 17-Dec entry of http://live.gnome.org/TwoPointTwentyone) for the new strings: +TABLE_NAMES = {'Cz-Cz-g1': _('Czech Grade 1'), + 'Es-Es-g1': _('Spanish Grade 1'), + 'Fr-Ca-g2': _('Canada French Grade 2'), + 'Fr-Fr-g2': _('France French Grade 2'), + 'Lv-Lv-g1': _('Latvian Grade 1'), + 'Nl-Nl-g1': _('Netherlands Dutch Grade 1'), + 'No-No-g0': _('Norwegian Grade 0'), + 'No-No-g1': _('Norwegian Grade 1'), + 'No-No-g2': _('Norwegian Grade 2'), + 'No-No-g3': _('Norwegian Grade 3'), + 'Pl-Pl-g1': _('Polish Grade 1'), + 'Pt-Pt-g1': _('Portuguese Grade 1'), + 'Se-Se-g1': _('Swedish Grade 1'), + 'ar-ar-g1': _('Arabic Grade 1'), + 'cy-cy-g1': _('Welsh Grade 1'), + 'cy-cy-g2': _('Welsh Grade 2'), + 'de-de-g0': _('German Grade 0'), + 'de-de-g1': _('German Grade 1'), + 'de-de-g2': _('German Grade 2'), + 'en-GB-g2': _('U.K. English Grade 2'), + 'en-gb-g1': _('U.K. English Grade 1'), + 'en-us-g1': _('U.S. English Grade 1'), + 'en-us-g2': _('U.S. English Grade 2'), + 'fr-ca-g1': _('Canada French Grade 1'), + 'fr-fr-g1': _('France French 1'), + 'gr-gr-g1': _('Greek Grade 1'), + 'hi-in-g1': _('Hindi Grade 1'), + 'it-it-g1': _('Italian Grade 1'), + 'nl-be-g1': _('Belgium Dutch Grade 1')}
Created attachment 103107 [details] Repackaged version of liblouis This is a repackaged version of liblouis (1.3.5). It has all of the pkg-config magic needed to build Orca smoothly with liblouis support. When configuring use --enable-ucs4, if you won't do that you will get gibberish.
Created attachment 103108 [details] [review] Initial grade 2 support This patch gives Orca initial grade 2 support. When you do "configure" when building orca use --enable-liblouis=yes. If liblouis is not installed properly configure will fail, but generally configure should autodetect liblouis. Some known issues: - The radio and checkbutton indicators are mangled, we eed to have a minor refactor so that we don't contract those indicators. - Uncontract under cursor does not work yet, this seems to be a liblouis bug, I could make a workaround, but maybe John could fix it. - The U.S. Grade 2 table is chosen by default, I need to implement smarter default table selection. - No routing key support yet. - Barely tested in general.
> When configuring use --enable-ucs4, if you won't do that you will get > gibberish. As Eitan and I discovered last week, you don't need this flag on Solaris Express. If you use this flag on Solaris, you get gibberish. :-)
(In reply to comment #22) > Created an attachment (id=103108) [edit] > Initial grade 2 support This is looking pretty cool (THANKS!). I think some bullet proofing and defensive code might be needed, however. When I select the 'compressed' table, for example, Orca ends up hanging. :-( I wonder also if we need to update the braille monitor to check for strings that contain non-printable characters? I notice the braille output sometimes ends up causing the braille monitor to complain. > Some known issues: > - The radio and checkbutton indicators are mangled, we eed to have a minor > refactor so that we don't contract those indicators. Eitan and I talked about this last week as well. One of the ideas was to create braille.py:Component subclasses named CheckBoxIndicator and RadioButtonIndicator (or something similar). These subclasses would present the indicators (e.g., one of settings.brailleCheckBoxIndicators or setting.brailleRadioButtonIndicators) and would resist contraction. The braillegenerator would use these classes to build up a sequence of braille regions, where there would be a region for the indicator and a region for the label, with both being backed by the same accessible checkbox/radiobutton. I also noticed that cursor tracking in flat review doesn't work very well. In any case, this is something to play with and is a good start. Thanks Eitan!
Created attachment 103681 [details] Another repacked liblouis Ignore the identical version name, this is an updated package.
Created attachment 103682 [details] [review] Grade 2 support This adds expand under cursor support. Also a bit of cleanup.
(In reply to comment #26) > Created an attachment (id=103682) [edit] > Grade 2 support > > This adds expand under cursor support. Also a bit of cleanup. > This patch ends up with rejections for settings.py and _louis.c when applied to trunk (svnversion 3509). In looking at it, I'm guessing the UI code that is changing isn't really presenting a UI layout change, but is more about enabling the buttons? Is that correct?
Created attachment 103988 [details] [review] Grade 2 support This is a patch that applies cleanly to trunk (rev 3523).
(In reply to comment #28) > Created an attachment (id=103988) [edit] > Grade 2 support > > This is a patch that applies cleanly to trunk (rev 3523). > Thanks! I'm building/running this on a system w/o louis, and it fails because of the "import louis" in braille.py. :-( Can you work up a patch that gracefully recovers if liblouis support is not available (please :-))?
Created attachment 104053 [details] [review] Grade 2 support This patch has the beginning of router key support (only tested in gedit). It also fails safely when liblouis is not present. I would suggest throwing this in to trunk, so that testing could resume more easily. Also, liblouis is now available as a code.google.com project: http://code.google.com/p/liblouis/
Every one of these patches look better and better. thanks a lot Here are just a few bugs I've noticed so far. 1. turning off grade2 in terminal doesn't seem to be working. To test do the following: A. enable contracted braille B. open gnome-terminal C. press orca+CTRL+space to bring up the orca prefs for terminal. D. on the braille tab uncheck contracted braille and press OK Notice that any time you open terminal after the change contracted braile is still turned on. I took a look and noticed that the setting in .orca/app-settings is correct ie set to false. 2. On any web page or HTML message in thunderbird the word under the orca cursor/caret is not uncontracted as I would expect. Just to confirm this should behave exactly the way we are doing in text documents.
Created attachment 104089 [details] [review] Grade 2 support This patch addresses none of Mike's input :) I'll get to that tomorrow. In the meantime these are the changes: 1. gnome-terminal caret behavior fixed. 2. Fixed cursor behavior with labeled entries.
This is the 6th patch. It still needs plenty of work, but I am hoping that now that it is safe with installations without liblouis, we could put it in trunk.
Created attachment 104176 [details] [review] Grade 2 support Further cleanup, and new stuff. 1. Settings should work per-app, like in Mike's gnome-terminal example above. 2. Quick grade 2 toggling from the braille display, right now I set it to the "A" button that appears in the X11 driver, but I think ideally it should be bound to CMD_SIXDOT, if that actually maps to an actual key on the display.
Created attachment 104350 [details] [review] Grade 2 support This patch adds attribute and selection support.
(In reply to comment #35) > Created an attachment (id=104350) [edit] > Grade 2 support > > This patch adds attribute and selection support. > I just took a quick look at this patch....there might be some stuff in there that works with Python 2.5, but not 2.4. Pylint doesn't like this line in braille.py (thinks it is invalid syntax), and my version of Python (2.4) also doesn't like it: + self.label = label+ ' ' if label else '' In addition, pylint also complained about this line in orca_gui_prefs.py, where the complaint is that it is redefining the built-in 'iter': + iter = combobox.get_active_iter() Just renaming iter to myIter or something like that would be fine. Thanks!
Created attachment 104617 [details] [review] Grade 2 support This patch fixes the issues that Will pointed out above. Also expand on cursor should work with Firefox. Also components with indicators, like radio and checkboxes should appear correctly.
Created attachment 104679 [details] [review] Patch to remove a print, fix some pylint issues, and work with Python 2.4 This builds upon the previous patch and does the following: 1) Eliminates some pylint warnings 2) Removes a debug 'print' that was in the code 3) Supports Python 2.4 The supporting of Python 2.4 surrounds the use of rpartition and partition in __init__.py. These methods are new for Python 2.5. The workaround defaults to trying to use these methods first, but then falling back to stuff that works with Python 2.4 I've only tested it with gedit.
(In reply to comment #38) > Created an attachment (id=104679) [edit] > Patch to remove a print, fix some pylint issues, and work with Python 2.4 > > This builds upon the previous patch and does the following: > > 1) Eliminates some pylint warnings > 2) Removes a debug 'print' that was in the code > 3) Supports Python 2.4 > > The supporting of Python 2.4 surrounds the use of rpartition and partition in > __init__.py. These methods are new for Python 2.5. The workaround defaults to > trying to use these methods first, but then falling back to stuff that works > with Python 2.4 > > I've only tested it with gedit. In testing this more, some work needs to be done in flat review so that the current word being reviewed (i.e., the word with the character we are reviewing, the word we are reviewing, etc.) is uncontracted. Right now, the "uncontraction" seems to be biased towards caret offset.
Obsoleting the other patch - Eitan tested the new one and verified it works with Python 2.5. Between Eitan and me, we have 2.4 and 2.5 covered. :-) We still need to fix the flat review uncontraction support, though.
Created attachment 104748 [details] [review] Grade 2 support This patch has the following addition: Make text regions always expand on cursor. Fixed component expansion in flat review. Fixed expand on cursor for flat review.
(In reply to comment #41) > Created an attachment (id=104748) [edit] > Grade 2 support > > This patch has the following addition: > Make text regions always expand on cursor. > Fixed component expansion in flat review. > Fixed expand on cursor for flat review. > This is looking much better! Thanks! I see some strangeness with expansion when reviewing the status bar in flat review: the "Ln x, Col y" stuff is all expanded when reviewing it, but this is much better. Mike - I think it's time for you to take this for a ride again.
> Mike - I think it's time for you to take this for a ride again. PS - when applying this patch use -p1 instead of -p0.
Yikes! Bummer. This patch seems to introduce some odd regressions. Here's a typical one when running the following test: ./runone.sh ../keystrokes/gtk-demo/role_text_multiline_navigation.pytk-demo 0 ... Test 29 of 100 FAILED: ../keystrokes/gtk-demo/role_text_multiline_navigation.py:KP_5 to flat review ' ' EXPECTED: "BRAILLE LINE: ' $l'", " VISIBLE: ' $l', cursor=1", "SPEECH OUTPUT: 'white space'", ACTUAL: "BRAILLE LINE: ' ", " $l'", " VISIBLE: ' ", " $l', cursor=1", "SPEECH OUTPUT: 'white space'", ...
(In reply to comment #44) > Yikes! Bummer. This patch seems to introduce some odd regressions. OK - that regression was easy to fix. The patch introduced a change that was not quite right: - if string[-1:] == "\n": - string = string[:-1] - self.string = string.encode("UTF-8") + string.strip('\n') + self.rawLine = string.encode("UTF-8") Changing it to this made it work (i.e., save the result of string.strip('\n')): - if string[-1:] == "\n": - string = string[:-1] - self.string = string.encode("UTF-8") + string = string.strip('\n') + self.rawLine = string.encode("UTF-8") There's still some regressions in flat review, however, where braille is being omitted: Test 69 of 100 FAILED: ../keystrokes/gtk-demo/role_text_multiline_navigation.py: KP_7 to flat review toolbar EXPECTED: "BRAILLE LINE: 'Open & y toggle button Quit panel GTK! $l'", " VISIBLE: 'Open & y toggle button Quit pane', cursor=1", "SPEECH OUTPUT: 'Open not pressed toggle button Quit panel GTK!'", ACTUAL: "SPEECH OUTPUT: 'Open not pressed toggle button Quit panel GTK!'", In looking at the debug, it looks like it's being caused by a traceback: Traceback (most recent call last):
+ Trace 188739
n processInputEvent consumed = self.function(script, inputEvent)
eviewNextItem self._reviewCurrentItem(inputEvent)
reviewCurrentItem self.updateBrailleReview(targetCursorCell)
pdateBrailleReview [regions, regionWithFocus] = context.getCurrentBrailleRegions()
regionWithFocus.expandRegion()
pandRegion self.cursorOffset = self.inPos[self.cursorOffset]
I'll dig a little more.
> In looking at the debug, it looks like it's being caused by a traceback OK - seems as though expandRegion was being called for a region even if contracted braille was not enabled. I resolved this by modifying braille.py:expandRegion to check for this. Not sure if this was the right thing to do, but this and the previous fix address all the regressions in role_text_multiline_navigation.py. def expandRegion(self): if not self.contracted: return self.string = self.rawLine try: self.cursorOffset = self.inPos[self.cursorOffset] except IndexError: self.cursorOffset = len(self.string) PS - new patch to follow.
Created attachment 104860 [details] [review] Patch to address regressions in role_text_multiline_navigation.py This addresses the role_text_multiline_navigation.py regressions as described in the previous comments. There are still additional regressions when running the whole suite of gtk-demo tests. The problem seems to be an extra space in the braille output when we're working with tables. Here's a typical failure -- look for the extra space before "Fixed?": Test 7 of 8 FAILED: /export/home/wwalker/orca/trunk.354470/test/harness/../keystrokes/gtk-demo/role_column_header.py:Checkbox cell EXPECTED: "BRAILLE LINE: 'gtk-demo Application GtkListStore demo Frame ScrollPane Table Fixed? ColumnHeader < > Fixed?'", " VISIBLE: '< > Fixed?', cursor=1", "SPEECH OUTPUT: ''", "SPEECH OUTPUT: 'Fixed? column header'", "SPEECH OUTPUT: 'check box not checked '", ACTUAL: "BRAILLE LINE: 'gtk-demo Application GtkListStore demo Frame ScrollPane Table Fixed? ColumnHeader < > Fixed?'", " VISIBLE: '< > Fixed?', cursor=1", "SPEECH OUTPUT: ''", "SPEECH OUTPUT: 'Fixed? column header'", "SPEECH OUTPUT: 'check box not checked '",
(In reply to comment #47) > look for the extra space before "Fixed?": Actually, it might be an extra space after the "< >" indicator. I'm not sure, though. Just a hunch since the patch does work in the area of indicators and all the regressions are in the area of extra spaces after indicators.
Created attachment 104870 [details] [review] Patch to eliminate all the gtk-demo regressions The problem with the extra space was, well, an extra space. :-) A space was always being added after the indicator in braille.py:Component:__init__ even if there was no string. In the regressions we were seeing, we were working with table cells that were just check boxes. As a result, there was no label for the check box and we were just getting the indicator. This patch checks and adjusts for that condition. I ran the entire gtk-demo regression test suite with this patch and it's now regression free. :-)
Created attachment 104887 [details] [review] Same patch, but with simple pylint fix (remove a tab) This is the same as the previous patch, but it pylints better. Made against svnversion 3556.
Talked with the team - committing this patch for GNOME 2.21.91. Mike will still work on identifying remaining issues (e.g., text selection indication for contracted text), so leaving this bug open. Thanks for the hard work so far, Eitan!
One small issue I've just discovered with this functionality is as follows. 1. Open gnome-terminal 2. type ls in a directory with several files. 3. flat review the output. Notice that the line you are flat reviewing is expanded. I would expect only the word under the flat review focus to be expanded.
The gnome-terminal behavior is expected, as the contraction table in liblouis tries to use computer braille on detected file names. I am closing this bug. Any further grade 2 bugs should be files separately.