GNOME Bugzilla – Bug 789340
should update caret position while speaking entire document in firefox
Last modified: 2019-02-20 10:14:21 UTC
Created attachment 362084 [details] [review] proposed fix Hello, With a webpage opened in firefox, the KP_Add shortcut allows to "speak entire document", just like in other application. One difference, however, is that in other applications, during the speaking of the whole document, the cursor position is updated regularly, to show the progress of the speak, which is convenient for users, but in Firefox this does not happen, the cursor position is only updated at the end of the speak (at speak stop position). I don't know whether that was intentional, but users have expressed they would like the feature to be enabled in firefox too. The attached patch fixes this by calling setCaretOffset instead of setCaretContext in __sayAllProgressCallback. Samuel
Thanks for the patch. I remember that we used to set the caret physically and that it had unwanted side effects. I'd have to do some investigation to determine what they were.
I thought about something like that, but didn't find traces of this in the git history: acedcca172988997f85c342b1eedf2a28f13de0b introduced it as setCaretContext already.
(In reply to Joanmarie Diggs (IRC: joanie) from comment #1) > Thanks for the patch. I remember that we used to set the caret physically > and that it had unwanted side effects. I'd have to do some investigation to > determine what they were. What is the state of this bug? Best regards.
If you set the caret position, you can trigger unwanted changes in the document content. When navigating/interacting with the content, that is reasonable; it's not when doing a SayAll. I think a better solution would be for us to be able to scroll the items into view. This would, of course, require new API added to ATK and AT-SPI2. Samuel mentioned this on the GNOME Accessibility list. Once that is in place, let me know and all add support to it in Orca. Thanks!
Right, on the long run using scrollto will be better.
(In reply to Samuel Thibault from comment #5) > Right, on the long run using scrollto will be better. ScrollTo should be used but it isn't sufficient to have synchronization between Orca and a screen magnifier. @joanie: Do you know if we could be aware of the Orca position in the page when sayAll is used? Best regards.
I think that as part of the new API (to be added to ATK and AT-SPI2), an event could be emitted that magnifiers could listen to. In other words, Orca would use this API to tell Firefox what to scroll into view. Firefox would scroll it into view. Then Firefox would emit an accessibility event to indicate that this object was scrolled into view. Magnifiers would listen for that accessibility event, and when they receive it they would ensure that the object was contained within the magnified view.
(In reply to Joanmarie Diggs (IRC: joanie) from comment #7) > I think that as part of the new API (to be added to ATK and AT-SPI2), an > event could be emitted that magnifiers could listen to. In other words, Orca > would use this API to tell Firefox what to scroll into view. Firefox would > scroll it into view. Then Firefox would emit an accessibility event to > indicate that this object was scrolled into view. Magnifiers would listen > for that accessibility event, and when they receive it they would ensure > that the object was contained within the magnified view. Right. We hope more granularity, we expect to be aware of the Orca position for each sentence for example. Are you aware of such feature on IAccessible2 or on macOS? In the future we hope we could have a granularity by word, it's the index marking feature of some speech synthesis and that is available into speech-dispatcher (see https://devel.freebsoft.org/doc/speechd/ssip.html#Message-Events-Notification-and-Index-Marking). That's feature is available on Windows with ZoomText Fusion (combination of Jaws and ZoomText). Best regards.
One issue is that as discussed in https://bugzilla.gnome.org/show_bug.cgi?id=792820 , one can want to scroll paragraph by paragraph (case of non-sighted user working with sighted user), or word by word (case of screen magnification). Perhaps that would be an option in Orca for configuring which object gets the ScrollTo call? (paragraph or word)
Yeah, what the scrollTo is would be configurable in Orca. Regarding the granularity, the to-be-created API in ATK and AT-SPI2 might actually be two related sets of API: One in the Component interface and one in the Text interface. The associated signals would be the same. So if Orca scrolled to offset 12 in a paragraph, Firefox could scroll that into view and then emit an event for interested magnifiers to tell them the object and offset.
Mmm, as I have just mentioned on https://bugzilla.gnome.org/show_bug.cgi?id=792820 , IAccessible2's ScrollTo doesn't have such notion of offset.
Hello, Just to mention that the ScrollTo API has been added to AtSpi and firefox (will be available in 62), so it can be used to fix this.
Created attachment 372550 [details] [review] proposed implementation Hello, Here is a patch that makes orca use scrollTo during readall, tested with firefox to-be-62. I added the probe for availability for scrollTo, so that it does not have to already depend on the next pyatspi release. Samuel
Thanks for the patch. However, I plan to look at the new API and see where else we might want to apply it. Rather than commit something I will later remove, I would like to wait until I have more time to consider it. Thanks for your understanding and patience.
Hello Joanie, What's about this one? We would like to see this feature appears in Orca 3.30 as it is highly requested by low-vision people. Best regards, Alex.
I'll try to add support next week.
(In reply to Joanmarie Diggs (IRC: joanie) from comment #16) > I'll try to add support next week. What's about this issue Joanie? Are you waiting an action from Hypra? Best regards, Alex.
I am waiting until I have some more time to look at the API and possibly implement across the board support including related to Chrome/Chromium (as that becomes more accessible). At this time, I do not need anything from Hypra.
Hello Joanie, FYI, as it is the most requested feature for our users (For example, it is one of the initial questions people ask them before switching to GNU/Linux with Hypra), we'll be forced to implement the feature downstream. Also, users ask me, are you aware we don't see what there is on the screen with say all? I'm forced to reply we're working on it. This is one of the reasons why users chose to buy ZoomText. This message is just to let you know that we will work on it, and hope that this work will be available not only to Hypra's users. Best regards, Alex.
Thanks for the patch. I've just committed an alternative version to master. I hope it will accomplish what you want.
It seems to be doing what we want indeed, thanks!