GNOME Bugzilla – Bug 150073
Speech Does not Work with Open Office, Mozilla and other applications
Last modified: 2004-12-22 21:47:04 UTC
Gnompernicus speech does not work with menus within the applications (i.e. Open Office, Mozilla, etc.) 1. Bring up gnopernicus 2. enable speech 3. listen and verify speech is working 4. bring up Open Office or Mozilla or any applications 5. Click on "File" do a open file 6. listen for speech NO speech for any menus within applications ... System configurations: Gnopernicus 0.9.7 Gnome Desktop 2.7.4 Fedora Core 2
Additional Steps info : 1. Open Term window 2. Run gnopernicus & 3. OPen 2nd term window 4. Bring up and applications and run in background (i.e. OPenOfice or Mozilla &) from 2nd term window opened. 5. Once application is up, click on file menu and listen for speech 6. Go back to 1 gnome-term window (gnopernicus), you should see the follow srcore Log: ---- (srcore:3927): gnopernicus-WARNING **: Unable to find parameter gnopernicus-Message: speech initialization succeded (srcore:3927): SRSXML-CRITICAL **: internal error (srcore:3927): SRSXML-CRITICAL **: Extra content at the end of the document gnopernicus-ERROR **: file srs-xml.c: line 323 (srs_endElement): should not be r eached aborting... (gnopernicus:3926): gnopernicus-WARNING **: srcore exited.
Created attachment 30742 [details] [review] patch to see what is sent to speech I followed your scenario but I can not reproduce this bug on my computer. Please apply this patch in order to obtain a log with the informations sent to speech. It will be usefull to see what is sent to speech when it crash with these errors.
It was and still is a problem in accessibility in java. Java access (jaccess) didn't work with java 1.5.x (I don't remember x value). Check if you are able to see any information from same aplication using at-poke.
Remus: I don't think your comment has anything to do with this bug report. Speech and accessibility work fine with Java 1.5.0. It looks as though the reporter may have been seeing one of the recent srcore crasher bugs. Tony, did you get _any_ speech output from gnopernicus, from any applications?
Bill, the only speech i got is from gnome-terminal. Yet the terminal will eventually crash (see bug 141191 for detail), but that's another problem.
The symptoms look a lot like 150464. Tony, what version of gnopernicus are you running? I wonder if these crashers were fixed by the recent changes in gnopernicus 0.9.9, or if they are regressions...
marking AP1, though bug is unconfirmed. Test team, are you seeing this behavior?
I do not see this behavior. It will be usefull for us to identify the cause of this bug if Tony will apply the patch attache at comment #2. Thanks!
I tested using different mozilla versions and I managed to optain this gnopernicus crash when the shortcut and accelerator are not correct encoded. In this case the information sent to speech looks like: <SRSOUT markers="out-started:out-ended" id="8120338p"><TEXT voice="name">File</TEXT><TEXT voice="role">Menu</TEXT><TEXT voice="system">shortcut</TEXT><TEXT voice="shortcut">ð@ð¢@</TEXT><TEXT voice="system">14</TEXT><TEXT voice="system">items</TEXT></SRSOUT> After sending this information to speech, gnopernicus crashes. See the shortcut!!! Please check with at-poke to see what are the keybindings for your mozilla version menus. (Bug: http://bugzilla.mozilla.org/show_bug.cgi?id=246190 was filed against mozilla to describe this problem).
Bill - I am using Gnopernicus 0.9.7 Dana - I am trying to get my team to apply the patch. Alexandra - Glad that you see gnopernicus crash. But this problem occured in Other Applications as well like OpenOffice (beside mozilla), where speech would stop working as soon as any application brought up. Please see orginal steps ...
Tony: gnopernicus 0.9.7 had/has some serious problems. We don't expect it to work properly, unfortunately. Can you try gnopernicus 0.9.9? The known crashers in 0.9.7 were reportedly fixed in 0.9.9.
Do you see this bug only when you try to access the menus? Try to see those applications using at-poke. Do you see something accessible?
We test using gnopernicus and an independent at-spi tester : - for the same menu item at-spi sends to gnopernicus (for example: string2) different string that the one at-spi tester receives (for example: string1), also there is a difference between string1' length and string2' length. - testing using just the independent at-spi tester there are different strings generated for the same menu item (the string generated when the item is focused for the first time is different that the one generated when the same item is focused again, after same navigation between menu items) - testing using just with gnopernicus, for every shortcut, we used 'g_utf8_validate' function that should verify a string if it is correct encoded or not. If the string is correct encoded it will be send to speech. For some reasons, some times, this function returns a valid value for the string even if it is not correct encoded. Other thing is that the tests could be made only on JDS build 14 with default mozilla 1.7. This bug can't be reproduced on later builds, there the shortcuts seems to be correct encoded with the same default mozilla 1.7.
Ada said: "We test using gnopernicus and an independent at-spi tester : - for the same menu item at-spi sends to gnopernicus (for example: string2) different string that the one at-spi tester receives (for example: string1), also there is a difference between string1' length and string2' length." Ada, it seems very unlikely that at-spi is sending different strings to different AT clients. Can you check to make certain that gnopernicus and your independent tester are getting the string from the event _in exactly the same way_, and using the same APIs to present this string? In other words, make sure that your diagnostic fprintf(stderr, "...) or whatever are the same in the two programs, and that you are reading the menu item string from exactly the same at-spi call. The fact that g_utf8_validate() sometimes succeeds even for "garbage" or incorrectly encoded strings is not a surprise - since even random strings will sometimes happen to be "legal" UTF-8 even if they contain "incorrect" information. - Bill
Created attachment 31443 [details] [review] proposed patch ...
Created attachment 31444 [details] ... and its results
As it can be seen, to the SAX parser a valid UTF-8 string is sent, but the parser is not able to parse the XML string.
Remus: valid UTF-8 isn't the same as valid XML. Are you specifying an encoding for the XML fragment or the parser? In other words, does the parser know that it's supposed to be expecting UTF-8 instead of some other encoding? In some cases the platform default encoding for the parser might be something other than UTF-8.... just a thought.
also Remus: srcore should not be exiting in this situation, since the error is recoverable. I think you've got an assertion in there that should not be an assertion, it should be just an if() block with, possibly, a warning.
"64, 127, 64, 32, 32, 25, 9, 16," These appear to be the illegal text content. These characters are, in UTF-8: @ (at sign), (delete), @ (at sign), (space), (space), (negative acknowledge), (horizontal tabulation), (data link escape). Of these, delete, negative-acknowledge, horizontal-tabulation, and data-link-escape are NOT valid XML (see the XML spec). In short, XML cannot include control characters. So before passing a string to the XML parser, rather than checking to see that it's valid UTF-8, you should check to make sure that it's valid XML. You could either use the parser itself to tell you this, and just omit any substrings that were not valid, or you (probably) could find some glib utilities for checking this. In any case this is a gnopernicus bug (as well as a bug in the applications) since gnopernicus should not crash if it's given bad input from some other application. It's up to gnopernicus to either handle an error return from the parser, or prevent the parser error by validating the string first. g_utf8_validate, as you have seen, is not the correct validation in this case, or at least is not sufficient. - Bill
I have found a glib function called g_markup_escape_text, but it doesn't solve the problem. I haven't found a function to validate an XML string. The only possibility seems to be the parsing.
Created attachment 31467 [details] [review] proposed patch
Created attachment 31468 [details] [review] a new proposed patch
Previous patch checks also if the string sent to gnopernicus devices is a valid utf8 one and if a key binding contains signs "<" and ">" as a pair.
remus, I think the above patch doesnt' quite do what we want, though it would detect many errors. Perhaps we need to check the string with g_unichar_isctrl, one unicode character at a time... but it seems to me that there ought to be something available in libxml2 to test this.
Remus, the patch definitely won't work, since it will fail if the "garbage" string happens to contain '>'. What we need is a validation method in gnopernicus which checks for valid UTF-8 CDATA, meaning test the string for utf-8 validity and also make sure all characters are valid CDATA according to the XML spec.
Bill, every string is checked if it contains \>, \<, \&, \', \". If does, then these characters are converted. This is already implemented in gnopernicus. So, it _never_ may happen ho have these characters in an XML, as a part of the information. The only problem is with special characters, (eg: 25, 127, 16). I don't see as solution to check only for these, because other chars my be in other cases and situations. So, a solution is to check _every_ char using g_unichar_isctrl or, much better in my oppinion using g_unichar_isalnum. But, for last function (g_unichar_isalnum) I am not sure about the result in case of language specific chars. Also, in this case, gnopernicus _should_ check _every_ string received from at-spi, or from po files (when "_" macro is used) if it is a valid utf-8 string, and also if it contains control characters. This will be a VERY serious _performance_ issue for gnopernicus. Gnopernicus may check only strings received for keybinding, but, in future other surprises like this one (with menus in mozilla) may appear. So, to avoid such situations with other info (name, description, etc) gnopernicus has to check all results from at-spi. Then, the question is why this isn't done in at-spi. In this way, other clients of at-spi will not be forced to implement same validations. The validation is really useful only when information received is bad. As I can read from http://developer.gnome.org/doc/API/2.0/at-spi/at-spi-cspi-AccessibleAction-Interface.html, for AccessibleAction_getKeyBinding function, the result SHOULD be an UTF-8 (Returns : a UTF-8 string which can be parsed to determine the i-th invokable action's keybindings.). So, why the checks are _not_ made in at-spi? _What_ kind of information are then returned by at-spi?
Created attachment 31543 [details] [review] a new proposed patch
Remus: at-spi returns UTF-8 strings; but realize, it's up to the ATK implementation to provide these strings. The ATK implementation is application-specific, so these bugs are in the applications themselves (if the UTF-8 is not valid), not in at-spi. This is not really important to this bug however; regardless of whether the string is valid UTF-8, gnopernicus must check to see if it is also valid for use as XML CDATA or not. Any client which uses the g_utf8.... functions should test the string with g_utf8_validate before using the string. This is documented in the GLIB API documentation. So even if the string from at-spi has a problem, gnopernicus should test and detect the problem. - Bill
Patch applied to cvs head and gnome-2-8 branch.
*** Bug 150464 has been marked as a duplicate of this bug. ***
please revert the patch, it needs work.
Comment on attachment 31543 [details] [review] a new proposed patch Comments inline: >Index: gnopernicus/srcore/srbrl.c >=================================================================== >RCS file: /cvs/gnome/gnopernicus/srcore/srbrl.c,v >retrieving revision 1.53 >diff -u -r1.53 srbrl.c >--- gnopernicus/srcore/srbrl.c 12 Aug 2004 08:11:21 -0000 1.53 >+++ gnopernicus/srcore/srbrl.c 14 Sep 2004 12:45:30 -0000 >@@ -527,7 +527,10 @@ > { > if (monoutput) > { >- src_brl_chop_output (monoutput); >+ if (g_utf8_validate (monoutput, -1, NULL)) >+ src_brl_chop_output (monoutput); >+ else >+ sru_warning ("you are about o send an invalid UTF-8 string to brlmonitor"); > } > } OK so far, but since the warnings are not for the end-user (they aren't translated), I think they should be shorter. Just sru_warning ("brlmon_output: invalid UTF-8 received") would do. >@@ -574,7 +577,10 @@ > { > if (brloutput) > { >- brl_xml_output(brloutput, strlen(brloutput)); >+ if (g_utf8_validate (brloutput, -1, NULL)) >+ brl_xml_output(brloutput, strlen(brloutput)); >+ else >+ sru_warning ("you are about o send an invalid UTF-8 string to braille"); Again, I would use a shorter string. "you are about o send" is a typo anyway. > #ifdef _BRAILLE_DEBUG_ > fprintf (stderr, "\n%s", brloutput); > #endif >Index: gnopernicus/srcore/srmag.c >=================================================================== >RCS file: /cvs/gnome/gnopernicus/srcore/srmag.c,v >retrieving revision 1.58 >diff -u -r1.58 srmag.c >--- gnopernicus/srcore/srmag.c 15 Jul 2004 14:27:39 -0000 1.58 >+++ gnopernicus/srcore/srmag.c 14 Sep 2004 12:45:36 -0000 >@@ -94,8 +94,10 @@ > #ifdef SRMAG_DEBUG > fprintf (stderr,"\nXML:%s",magoutput); > #endif >- mag_xml_output (magoutput, >- strlen (magoutput) ); >+ if (g_utf8_validate (magoutput, -1, NULL)) >+ mag_xml_output (magoutput, strlen (magoutput)); >+ else >+ sru_warning ("you are about o send an invalid UTF-8 string to magnifier"); Cut-and-paste of the typo. Replace with sru_warning ("mag_output: invalid UTF-8 received."); > } > busy = FALSE; > } >Index: gnopernicus/srcore/srmain.c >=================================================================== >RCS file: /cvs/gnome/gnopernicus/srcore/srmain.c,v >retrieving revision 1.142 >diff -u -r1.142 srmain.c >--- gnopernicus/srcore/srmain.c 6 Aug 2004 08:16:18 -0000 1.142 >+++ gnopernicus/srcore/srmain.c 14 Sep 2004 12:45:40 -0000 >@@ -2715,10 +2715,9 @@ > gchar* > src_xml_process_string (gchar *str_) > { >- gint len, i, pos; >- gchar *rv, *crt; >- gchar *str; >- >+ gchar *crt, *str; >+ GString *rv; >+ > if (!str_ || !str_[0]) > return NULL; > >@@ -2727,45 +2726,57 @@ > if (!str) > return NULL; > >- len = strlen (str); >- /* 6 = maximum lengt of xml_ch din translate table */ >- crt = rv = (gchar*) g_malloc ((len * 6 + 1) * sizeof (gchar)); >- if (!rv) >+ if (!g_utf8_validate (str, -1, NULL)) > return NULL; This leaks 'rv', doesn't it? >- >- for (i = 0, pos = 0; i < len; ++i) >+ >+ rv = g_string_new (NULL); >+ crt = str; >+ while (*crt) > { >- static struct >+ gchar *next = g_utf8_next_char (crt); >+ gunichar uni = g_utf8_get_char (crt); >+ gboolean special = FALSE; >+ >+ if (g_unichar_iscntrl (uni)) >+ { >+ g_string_free (rv, TRUE); >+ g_free (str); *THIS* is where you should print the warning. >+ return NULL; >+ } >+ else >+ sru_warning ("string contains control characters. Skiping it from presentation"); Doesn't this mean the warning will get printed once for every *valid* character? This can't be right... >+ if (next - crt == 1) > { >- gchar ch; >- gchar *xml_ch; >- }translate[] = { >+ static struct >+ { >+ gchar ch; >+ gchar *xml_ch; >+ }translate[] = { > {'<', "<" }, > {'>', ">" }, > {'&', "&" }, > {'\'', "'"}, > {'\"', """}, > }; >- gint j; >- gboolean special = FALSE; >- >- for (j = 0; j < G_N_ELEMENTS (translate); j++) >- { >- if (str[i] == translate[j].ch) This may be dangerous, you are testing 8 bits of a UTF-8 character. You should be testing the gunichar, shouldn't you? >+ gint j; >+ >+ for (j = 0; j < G_N_ELEMENTS (translate); j++) > { >- crt = g_stpcpy (crt, translate[j].xml_ch); >- special = TRUE; >+ if (*crt == translate[j].ch) >+ { >+ g_string_append (rv, translate[j].xml_ch); >+ special = TRUE; >+ } > } > } > if (!special) >- { >- *crt = str[i]; >- crt++; >- } >+ g_string_append_len (rv, crt, next - crt); >+ >+ crt = next; > } >- *crt = '\0'; >+ > g_free (str); >- return rv; >+ return g_string_free (rv, FALSE); > } > > gchar* >Index: gnopernicus/srcore/srpres.c >=================================================================== >RCS file: /cvs/gnome/gnopernicus/srcore/srpres.c,v >retrieving revision 1.45 >diff -u -r1.45 srpres.c >--- gnopernicus/srcore/srpres.c 27 Aug 2004 07:57:49 -0000 1.45 >+++ gnopernicus/srcore/srpres.c 14 Sep 2004 12:45:44 -0000 >@@ -1163,6 +1163,16 @@ > sru_assert_not_reached (); > break; > } >+ if (*(gchar**)info) Should you be testing 'info' also to make sure it's not a NULL pointer? >+ { >+ if (!g_utf8_validate (*((gchar**)info), -1, NULL)) >+ { >+ sru_warning ("Invalid info queried (%d) ", info_type); >+ g_free (*(gchar**)info); >+ *(gchar**)info = NULL; >+ rv = FALSE; >+ } >+ } > return rv; > } > >Index: gnopernicus/srcore/srspc.c >=================================================================== >RCS file: /cvs/gnome/gnopernicus/srcore/srspc.c,v >retrieving revision 1.83 >diff -u -r1.83 srspc.c >--- gnopernicus/srcore/srspc.c 6 Aug 2004 09:57:44 -0000 1.83 >+++ gnopernicus/srcore/srspc.c 14 Sep 2004 12:45:48 -0000 >@@ -470,7 +470,10 @@ > if (busy) > return FALSE; > busy = TRUE; >- srs_output (spcoutput, strlen(spcoutput)); >+ if (g_utf8_validate (spcoutput, -1, NULL)) >+ srs_output (spcoutput, strlen(spcoutput)); >+ else >+ sru_warning ("you are about o send an invalid UTF-8 string to speech"); Shorter version please, without the cut-and paste typo. > rv = TRUE; > > /* fprintf (stderr, "\n%s", spcoutput); */ >Index: gnopernicus/srlow/libsrlow/SRObject.c >=================================================================== >RCS file: /cvs/gnome/gnopernicus/srlow/libsrlow/SRObject.c,v >retrieving revision 1.113 >diff -u -r1.113 SRObject.c >--- gnopernicus/srlow/libsrlow/SRObject.c 27 Aug 2004 07:57:49 -0000 1.113 >+++ gnopernicus/srlow/libsrlow/SRObject.c 14 Sep 2004 12:46:04 -0000 >@@ -4873,6 +4873,8 @@ > for (tmp = strstr (key, "<"); tmp; tmp = strstr (key, "<")) > { > gchar *tmp2 = strstr (tmp, ">"); >+ if (!tmp2) >+ return NULL; > *tmp2 = '\0'; > tmp_key = g_stpcpy (tmp_key, tmp + 1); > tmp_key = g_stpcpy (tmp_key, " ");
Created attachment 31648 [details] [review] proposed patch to correct Remus's patch form comment 28 (attachment 31543 [details] [review])
>- if (!rv) >+ if (!g_utf8_validate (str, -1, NULL)) > return NULL; >This leaks 'rv', doesn't it? No, because rv is allocated after that. >- if (str[i] == translate[j].ch) >This may be dangerous, you are testing 8 bits of a UTF-8 character. You should >be testing the gunichar, shouldn't you? No, because I'am doing that only if curent character is 1-length (next-crt == 1 branch). >+ if (*(gchar**)info) >Should you be testing 'info' also to make sure it's not a NULL pointer? The function is always called, with a non NULL parameter, but a check should be here.
Created attachment 31716 [details] [review] additional patch for 31543
Comment on attachment 31716 [details] [review] additional patch for 31543 there's a typo in the word "skiping" (should be "skipping") and there should be a space after the colon (": ") in the sru_warnings. Otherwise it looks good.
Created attachment 31839 [details] [review] proposed patch I made the changes that you recommended . Because the characters '\n' and '\t' should be ignored on the validation of the string, I made the changes to do that.
The changes regarding '\n' and '\t' don't look right to me; these characters are legal as XML CDATA and I don't see why you would disallow them. Otherwise the patch is OK, but I think that change should be removed.
The 'if' condition is true when the current character is _not_ '\n' or '\t' and when g_unichart_iscntrl() is true. ... if (g_unichar_iscntrl (uni) && (uni != '\n') && (uni != '\t')) { ... sru_warning ("... Skipping it from presentation"); return NULL; } ... So, this condition allows the characters '\n' and '\t' to pass. Without this condition, all the strings containing '\n' or '\t' ware generating warning and they ware _not_ sent to speeech, braille device or brlmon. Without this change, some gedit strings/lines (containing '\n' or '\t') ware not presented on up/down navigation.
Ada, thanks for explaining. You're quite correct of course. Please commit (I am changing the patch status).
Comment on attachment 31839 [details] [review] proposed patch Patch applied to cvs head and gnome-2-8 branch.
Thank you all. With the patch, the problem seems to be resolved in : Fedora Core 2 Gnopernicus : 0.9.12 Gnome Desktop : 2.8 Though it seems to be working for Mozilla, Words, etc., but Open Office applications is not working still. Could you please encoprate the same fix over in OpenOffice applications ?
Tony, are you saying gnopernicus is still crashing with OpenOffice? Or just it doesn't report something for this app? In second case (no report) check if OO is accessible using at-poke.
It does not report something for this app. (OpenOffice) It does not appear to be crashing at this time. But speech for OpenOffice application does not work and does not speak at menus within the application. Steps: 1. Bringup gnopernicus, enable speech 2. Bring up Mozilla, key navigations thru menu 3. Listen for speech, Speech is speaking 4. Bring up OpenOffice application (i.e spreadsheet, etc.) 5. Key Nav thru menus 6. Listen for speech, No Speech
The problem with the OpenOffice menus which are not reported by gnopenricus is also observed in bug #154246 (but the text typed into TextEditor is spoken and displayed on brlmon). Is OpenOffice accessible on your system? Did you check this using at-poke?