GNOME Bugzilla – Bug 157941
Help browser content is inaccessible [REGRESSION]
Last modified: 2009-09-22 01:48:06 UTC
Run gnome-help, using gnome-2.9.X or HEAD. Run gnopernicus, GOK, or at-poke. Note that there is no text visible for the HTML content, no AtkHyperlink interfaces for the links, and no actionable interfaces for the content. Gnopernicus doesn't speak the help text when the user navigates through it, and GOK doesn't show the hypertext link in UI Grab. at-poke shows that these interfaces are not present on the help content pane.
Marking 'AP0' priority as accessibility of the help system is essential to accessibility of the GNOME desktop.
Is this because yelp is no longer using gtkhtml2?
yes, and there's something wrong with the gecko integration and/or the gecko ATK implementation.
Marking as a 2.10 stopper.
I don't know what the difference is between my machine and others', but at-poke is showing text in the Gecko component for me, although it doesn't appear to show links as activatable.
I briefly investigated this, here are the results so far: 1 Text. Works fine here and for shaunm, it may depend on your mozilla build... I tested with the latest code from 1.7 branch. The accessibility gconf pref needs to be enabled or you can export GNOME_ACCESSIBILITY=1 2 Hyperlinks. These are not showed, I'm pretty sure they are expected to work though, building 1.8 to see if it's fixed. Otherwise I'll try to figure out what's up with them (I'd expect this to be fixable) 3 hypertext links in UI Grab https://bugzilla.mozilla.org/show_bug.cgi?id=246770 has a patch... See last comment, maybe the patch is in, not very clear ;)
The hypertext patch probably is not in mozilla 1.7 yet; it would be in the Sun/a11y mozilla builds - though that doesn't help us much for gnome-help in general. Has anyone besides me tested with GOK and gnopernicus? (I haven't been able to reproduce the success above yet)
GOK seem to work fine with SUN mozilla build http://ftp.mozilla.org/pub/mozilla.org/mozilla/accessibility/sun-mozilla-1.7/12-03-2004/ At least links are showed correctly. I cannot test gnopernicus atm, cvs head crash on startup for me. at-poke doesnt show the hyperlinks. In general if it's ok to make yelp full accessibility depending on Sun patches, we can probably manage it for 2.10. Otherwise it seem though... Mozilla 1.8 is not yet on the roadmap, we should try to get the necessary changes in 1.7.x. It would be good if someone with some accessibility clues could get mozilla with accessibility support running, so that we can know exactly what the problems are at this point. As far as I can say so far they are: - gok UI grab links working only with Sun patches - at-poke not showing links at all Bill, as far as I can tell yelp and mozilla has exactly the same level of accessibility. You could try out the mozilla binary at: http://ftp.mozilla.org/pub/mozilla.org/mozilla/accessibility/sun-mozilla-1.7/12-03-2004/fedora3/ Or you could build the source from: http://ftp.mozilla.org/pub/mozilla.org/mozilla/accessibility/sun-mozilla-1.7/12-03-2004/src/ (I can help you on IRC building from source if you need, with that it should be easy to build yelp too) I'm also going to mail the people that are working on mozilla accessibility to see what they think about these two issues.
Marco: you said "Bill, as far as I can tell yelp and mozilla has exactly the same level of accessibility." But this is NOT true; the fact that GOK can't navigate the hyperlinks using the (non-SUN) gecko-plugin version of gnome-help means a major regression for GOK users. There are serious problem for gnopernicus users too. Because the GNOME desktop build won't (for most users/distributors) be using the accessible "Sun" version of mozilla yet, the end result is a much less accessible help system. I don't think GNOME 2.10 should be released with such a regression, and I/we just don't have the resources to fix this before 2.10.0.
The Sun Mozilla accessibility team is working hard to get all of the accessibility patches that are in our 1.7.x branch into Mozilla CVS head. We're getting some assistance from IBM accessibility engineers. This work is ongoing, and many patches have made it into head. However, I don't think this work will be completed in the GNOME 2.10.0 timeframe. Can this not be a user-option as to which HTML rendering engine is used for gnome-help? As this is an AT compatbility issue, could this not reasonably be tied to the AT support gconf flag?
>But this is NOT true; the fact that GOK can't navigate the hyperlinks using the >(non-SUN) gecko-plugin version of gnome-help means a major regression for GOK >users. There are serious problem for gnopernicus users too. Bill, I didnt really say it's not a major problem. I was just saying mozilla and yelp has the same level of accessibility. In facts Gok doesnt work well even with mozilla. I said that because testing mozilla is easier than testing yelp itself... I'm just investigating the problem and providing infos. I don't know what's the best decision in general here, and I'll leave that up to someone else.
thanks Marco - I misunderstood your comment to mean that the gecko-based yelp had 'the same level' of functionality as in the gnome-2.8 branch. It's true that GOK + Mozilla-1.7 isn't so good without the sun patches. I believe the thinking has been that the help system is at least as important as the browser, accessibility-wise. If help isn't accessible, the system is pretty useless. Some people would say that's true of the browser too, but at least things are better with the gtkhtml2 version of yelp. I don't know "how much is too much" when it comes to the accessibility regression. The gecko-yelp situation seems to be better than we first feared, since the ATK-gecko integration seems basically to be working according to reports above. But I am not sure that I personally will have time to complete such an assessment before 2.10 freezes. I think a few minor regressions are tolerable for the sake of the advantages which gecko provides, but we really should have a clear view of whether gecko-yelp remains basically usable by blind users and GOK users, without the Sun patches. I will try to get an assessment from the other AT developers...
If we ship yelp 2.8 (or is it 2.6) with 2.10, then this is not a showstopper, is it?
Yup. Punting to 2.12, I guess...
Bill, what's the status of the a11y patches in mozilla these days? Is it remotely possible that we can ship gecko based yelp in a 2.10.x point release if issues have been sorted out upstream?
Hi kjartan: it depends on "which patches" are upstream; only a minority have made their way to moz trunk AFAIK. I do not believe a 2.10 point release is a reasonable place to make such a big UI change, but for 2.12 I would hope that we could: (1) test yelp+gecko and iron out the remaining integration issues (I still haven't been able to successfully build mozilla with jhbuild gnome-2.23 modules tho :-( ) (2) identify the most critical remaining a11y bugs in 1.7 gecko and either get them committed to gecko 1.7 before gnome 2.12, or else apply the patches ourselves as part of the gnome build process (ugh). regards Bill
FWIW, the bangalore guys working on LDTP are reporting that firefox trunk at the very least works with at-poke.
cc'ing myself in response to: http://mail.gnome.org/archives/gnome-accessibility-list/2005-July/msg00013.html ?
I hope to have a look at this myself in the next week. Thanks to everyone who's kept an eye on this. Bill
Kjartan has recently reported finding some issues with latest-yelp a11y, but the basic a11y structure seems to be present and working. Kjartan, can you confirm that F7 gives you a text caret, gnopernicus either speaks or brailles (if braille monitor is on), and you can basically read the help content that's on the screen? If so, we can close this bug as fixed IMO and open new bugs for the remaining issues (keynav to sidebar, spacebar-doesn't-activate links, etc,) regards Bill p.s. - I can't seem to find kjartan's bugzilla username, can someone add him to the cc list? Thanks.
Your wish is my command.
:blush: I think I was just misspelling it.
I'm on all-bugs so no need to add me explicitly. F7 didn't really do anything that I could see and gnopernicus won't speak here at all so I'm not sure how much more help I can be for now.
If F7 doesn't work, then this is still a high priority stopper. Without F7 the help browser _is_ inaccessible. What can we do to make sure the F7 caret browsing mode works for embedded gecko just as it does for Mozilla and Firefox? This will also be an accessibility blocker for Epiphany... (if F7 is working in Epiphany then perhaps we can figure out how/why it works there, and copy the technique in yelp). Bill
I found it under preferences now. So it is possible to activate caret mode, just not through F7.
F7 activates caret mode in Epiphany.
Thanks Reinout. It sounds as though we just need a global keybinding for yelp to enable caret mode.
F7 is the standard documented means of turning on caret mode. It worked for yelp prior to the gecko transplant (i.e. works for GtkHTML2), so the absence of an F7 binding is still a regression.
Well, it's a change anyway. I'm not sure why it's a regression for me to replace an undocumented and unintuitive keyboard shortcut with a discoverable option in the preferences. But whatever, we can add a global shortcut to turn the text cursor on and off. I'd sure like to put a hint near the check box to that effect, but that'll have to wait until the next release cycle. It's not worth a string freeze break. Anybody feel like cooking up a patch? This is really low-hanging fruit for anyone looking for an easy starter project.
It's a regression because the old, documented (and de facto standard) keybinding went away. The discovereable option in the preferences is a good thing IMO, but the F7 should be retained.
All right, I've added F7 as a global shortcut to toggle the preference.
Hey, thanks Shaun! Once someone can confirm that gnopernicus works with yelp-HEAD once caret mode is on, I think we can close this bug.
I have no idea how to verify this but I really want this closed. Can someone tell me what to do to test, and I will? Thanks.
Created attachment 56943 [details] Screenshot This is a screenshot of the latest yelp HEAD running with gok. The only problem I found is that gok doesn't update (as shown by the screenshot). Selecting back and regrabbing updates and all the clickys show up fine, but when they are first selected nothing changes in GOK. I noticed that when epiphany is run with gok, it manages to update fine, so maybe they can enlighten us on how to make it work... This is using FF 1.5.
When using at-poke to poke epiphany or yelp, links don't show up as links, but only as text nodes. So you can't activate them with a11y... Btw, it looks like you got tons of assertion failures on console, did you try to get a trace from them? (breaking on g_log in case they're from gtk, or export XPCOM_DEBUG_BREAK=trap and running under gdb in case they're from mozilla) ?
The screenshot demonstrates the problem in comment #35 - since GOK would show the links as individual buttons, if they were exposed properly. That problem may be a mozilla/gecko problem, but it still means that yelp HEAD is seriously regressed w.r.t. a11y.
OK, just to clarify things a little as I'm slightly confused. I run gok and start up yelp. In gok, I select "UI Grab". All the links on the front page show up ok in GOK (These are the buttons "Desktop", "Applications", "Other Documentation" etc. that can be seen in the screenshot) This all corresponds nicely with what is in the yelp window. If I then select, in gok, one of the links (Desktop), Yelp changes and shows the contents of the Desktop link (Which is what is shown in the Yelp window of the screenshot) but gok doesn't update its buttons to reflect this (This is the situation when the screenshot was taken). In gok, I then click the top left back button and click "UI Grab" again, all the links show up fine (there are buttons for "Panel Applets", "Accessibility Guide" etc.). The stuff I was refering to about epiphany is that all the links show up nicely and about 1-2 seconds after the page has loaded fully, the links in gok change to show the new links on the page. I was wondering how this was achieved. > When using at-poke to poke epiphany or yelp, links don't show up as links, but > only as text nodes. So you can't activate them with a11y... When I run yelp through at-poke, they do indeed appear to be text, but they have an anchor property that doesn't appear in normal text if that means anything to anyone? In gok, I can activate the links by clicking on the corresponding buttons. About the assertion failures, they're generated by gok, afai remember. I'll investigate them a bit more when I get home. I'll also add a couple ore screenies to try and show more about what I mean. All this was done using a default firefox 1.5 install and a jhbuild Yelp in a jhbuild session.
I see, Don - I was misreading >The stuff I was refering to about epiphany is that all the links show up nicely >and about 1-2 seconds after the page has loaded fully, the links in gok change >to show the new links on the page. I was wondering how this was achieved. Since 'Hypertext' inherits from 'Text' in the ATK/AT-SPI interfaces, this explains your observation about "text with anchors". The problem seems to be that, as you point out, GOK isn't refreshing its UI Grab display using yelp, whereas it does when running epiphany. GOK recognizes certain kinds of events as cues that the UI Grab display should be refreshed; mostly it listens for "object:children-changed:add" events from objects it deems "primary containers". At the moment "primary container-ness" is determined by looking at roles; roles HTML_CONTAINER, SCROLL_PANE, DIALOG, FRAME, ROOT_PANE, and APPLICATION are recognized as "primary" in GOK HEAD. What is the 'role' name of the object (as seen in at-poke) which contains the new links? SOunds to me as though yelp is sending a different event stream from epiphany, when the HTML contents are replaced. Bill
Right. I've looked more into this. First, the roles: Top is the Application Inside that is the window, Role frame Inside that is a filler Then a split pane Then a scroll pane and a filler inside that. Then comes the html view itself. Its role is a frame (Same as epiphany). This is where the text appears. One thing I noticed about this is that there are links in text, but there are also list item objects that contain links. These appear as "<no name>" inside which is the text item with the anchor. Now for the slightly weird bit... I can make gok change its buttons. In one case. No others. In the user-guide, section "Basic Skills->Mouse Skills->Conventions", there is a link at near the bottom of the text, something like "Section 9.12 - Mouse Preferences". If I UI grab in GOK on that screen and click the link, GOK changes to the new page. I've tried several other linkies but none that I could find do the same. I'm off to examine the docbook to see what I can see.
Created attachment 57058 [details] at-poke Right, having read through my last comment, its not exactly clear what I'm taking about (Well, thats true in general but...) This screenie shows at-poke running on yelp. The "Help Topics" is the html frame The <no name> list items are all hyperlinks on the front page of yelp with the text in them containing an anchor. Hopefully that'll clear things up a little ;)
If you run a (possibly slightly hacked) version of at-spi/test/event-listener-test (include the -m flag), you can see which object is emitting "object:children-changed:add" events here (some high-level object should be, hopefully). That will help diagnose, since it will show the notifications GOK is getting when the help content changes.
Maybe the difference between epiphany and yelp is due to how yelp uses gtkmozembed which doesn't fire the onload event? [https://bugzilla.mozilla.org/show_bug.cgi?id=293670]
OK. I found out why that link fires correctly but no others do. Its to do with the tree in the left panel. Selecting that link expands the tree in a different node and selects one of the entries in it. Its the tree in the left that is emitting the signal. One thing I was wondering about. Is it possible to fake a children-changed:add event for something (window or GtkMozEmbed)? Just an idea, if it is the mozilla problem in comment #42?
Don, in the case you describe, it seems that children-changed should still be fired, because the right hand panel should be emitting events to indicate that its content has changed. I would expect that the html containing panel would be replaced, as opposed to being re-filled with new content (the latter would be evil from the point of view of GOK or other assistive technologies, as it would be very inefficient and result in lots of noise).
It's also possible, it seems to me, that the problem is due to the HTML view (role FRAME) being inside an object of ROLE_FILLER; if the frame gets destroyed and replaced by a new frame, then the object emitting the "children-changed:add" event would be ROLE_FILLER, and at the moment GOK ignores "children-changed:add" events from such objects (since it doesn't know that they are "primary containers" by its logic). Just another possibility. Perhaps GOK should check the role of the child being added, as well? In this case the added child is ROLE_HTML_CONTAINER which should flag it as important.
Hi again. I've done a little more digging and heres what I have found out. If a framed html is loaded (using yelp from CVS as that is the only place where frames work ;) ), mozilla is left to handle the loading of it by itself (through gtk_moz_embed_load_url). In all other cases, it is using gtkmozembeds streams. When a framed html is loaded, gok is correctly updated. As long as I stay within the document (i.e. internal links, loaded using gtk_moz_embed_load_url), gok keeps up to date. I'm not exactly a mozilla expert, but I think this is probably related to the mozilla bug mentioned in comment #42 (embed stream API doesn't fire onload event). Since this is a mozilla bug, is there a way to work around it? Is it possible to fire off a children-changed:add event for the window or html frame? If not, is there any other way around it? It would be REALLY nice to get this fixed up fully for gnome 2.14.
Created attachment 57893 [details] [review] workaround for Mozilla bug#293670 Since there is no plan to fix the Mozilla bug#293670 before GNOME 2.14's release, I proposed a workaround patch for yelp.
Created attachment 57954 [details] [review] Another approach Hi, I tried the patch (D'oh. I tried doing something similar, but couldn't get it working). It seems to update GOK ok, but it does seem to introduce a new problem. When showing a docbook manual, no links show up. Using at-poke, the frame seems to have no children, but for info pages it seems to work ok (All the links show up fine and GOK updates). When showing a docbook though, the links arn't shown as buttons in GOK. Using a debug build of firefox and at-poking yelp, when it gets to the html frame, the following is printed into the terminal: WARNING: NS_ENSURE_TRUE(NS_SUCCEEDED(rv)) failed, file nsAccessibleWrap.cpp, line 761 (Don't know if its relevant, but it never happened before) I suspect this is due to gecko not having finished drawing everything to the screen when the signal is fired. If the signal emission is instead done in a timeout function (with a timeout of 500ms - any lower and the links don't show up all the time), this problem is avoided (at least, I can't reproduce it) and at-poke shows all children as expected. This is done in the patch.
Don, not sure what you mean by: "It seems to update GOK ok, but it does seem to introduce a new problem. When showing a docbook manual, no links show up. " Does Ginn's patch solve the GOK problem? Looks to me as though it should. Ginn, does that patch work for you? Bill
Hi, Before: goks window wouldn't change at all. No updating took place. gok would show the hyperlinks available in the page before (Attachment to comment #34). With Ginn's patch: When a TOC page / info page is loaded, gok updates file, all the hyperlinks show up in gok as expected. When a docbook is loaded, gok updates, but only displays non-hyperlink buttons (eg. Back, Forward, Help Topics etc.). This is an improvement in that gok is recognizing that an event took place and updates itself however, gecko hasn't drawn all its elements (specifically, the hyperlinks arn't created / shown yet) and so gok, can't "see" them, hence it can't create buttons for them. With my patch: Gecko gets 0.5 seconds[1] from the time the stream is closed to get all the elements ready / shown before the signal is fired. This, for me, gives gecko enough time to get ready and after that, gok picks up all the hyperlinks correctly and shows them all as buttons as expected. Basically, Ginns patch should work, but gecko just needs a little time to sort itself out before the signal should be fired. [1] This gives enough time for things to work for me, but it might need tweaking a little to accomodate slower computers. I noticed epiphany doesn't update gok for ~1-2 seconds, maybe thats more reasonable.
Hi Don: I think I understand what is happenning now (and you probably do, too). Ginn's patch emits the right event, but since the docbook loading is asynchronous, it hasn't completed when GOK gets the notification. Thus, GOK doesn't see the new content. In any case, as Ginn notes, this is just a temporary fix while we wait for the real fix to get into mozilla. Probably the correct event to fire, in this case, is not "children-changed:add", but instead, "document-load-complete". That last event hasn't really even been defined yet, it's still in the draft AT-SPI Document interface spec (http://gnome.org/~billh/at-spi-new-idl/html/html). Bill
I reverted this due to bug 329407
Brent, what did you revert? Please add 'accessibility' keyword to relevant bugs. I really don't want us to ship an inaccessible help system in 2.14, that would be a major regression, so IMO we really need to get Ginn's patch, or something like it, into cvs.
Bill, My patch in comment #48 was applied and released in yelp 2.13.4 (with a timeout of 2 seconds). It was reverted in 2.13.5 due to bug #329407. I suspect bug #329407 is a more general problem than yelp, but it affects us even with accessibility disabled, hence rendering yelp useless with the patch applied. I'm also affected by the problem, and have reported it downstream at: https://launchpad.net/distros/ubuntu/+source/gnome-session/+bug/30248 I'll add some comments to the other bug when I get a chance to test some more.
"I suspect bug #329407 is a more general problem than yelp, but it affects us even with accessibility disabled,..." I don't see how that can possibly be, unless you have a serious distro problem, because the code being called isn't even loaded if the assistive tech support gconf key is false. Thus the problem you report looks like a distro bug to me. I'd really like to see your patch reinstated in gnome cvs, because it seems to me that the problem that led you to revert it may not be a general one.
Don, I'm going to pull CVS HEAD again and try to make sure I'm not seeing this problem using the stock gnome components...
Ok, I've recommitted to HEAD. Leaving this issue open though as I understand this is still just a workaround.
Brent and Don, thanks a bunch!
Lowering severity since workaround is in place.
Apologies for spam... ensuring Sun a11y folks are cc'ed on all current accessibility bugs.
Sorry for the spam. Marking patch as obsolete
removing the ancient gnome target milestone as a workaround has been committed to CVS.
*** Bug 355696 has been marked as a duplicate of this bug. ***
Testing against gecko 1.9 (XULRunner trunk build as of 19th November), it appears the "children_changed::add" signal is correctly fired and everything is working as expected (i.e. gok updates nicely and all links in all pages I tested showed up properly) :)
don, so should we close this as obsolete?
I reckon close as Fixed, even though the issue was partly 'notgnome'
The last comment from Bill in December 2006 reckons that this bug should be closed as FIXED. Doing so to get it off our radar. If anybody thinks this should still be open, then please reopen and state your case. Thanks.