GNOME Bugzilla – Bug 686622
Lack of accessibility support on clutter-gtk
Last modified: 2021-06-10 11:17:44 UTC
This is an old one, but not a problem until now due the lack of core GNOME applications using it. Recently Zeeshan Ali pinged me because he mentioned that Boxes lacked accessibility support. So some background here. clutter-gtk provides two containers (a gtk one and a clutter one) that requires ATK support. Providing it shouldn't be complex. ATK right now doesn't works fine with multiple-toolkits enviroments. Firefox add some hacky workarounds. My old plan was solving this in a general way, that probably would lead to an API breakage (bug 611507). But we have a real need now, so probably some kind of workarounds are required, as waiting for a definitive solution can be too long. Some proposals to this problem soon.
As I was not sure if clutter-gtk was a library with real usage on the GNOME ecosystem (after gnome-documents droppage of his use) I asked on d-d-l: https://mail.gnome.org/archives/desktop-devel-list/2013-February/msg00059.html Although the feedback was not big, at least the conclusion was that Boxes doesn't plan to change that. After a long chat on IRC today with Bastien Nocera, totem and cheese (obviously), and gnome-control-center doesn't plan to drop that library in the short term.
(In reply to comment #1) > As I was not sure if clutter-gtk was a library with real usage on the GNOME > ecosystem (after gnome-documents droppage of his use) I asked on d-d-l: > > https://mail.gnome.org/archives/desktop-devel-list/2013-February/msg00059.html > > Although the feedback was not big, at least the conclusion was that Boxes > doesn't plan to change that. Actually its more like Boxes *can't* change that. We have animations that docs does not have and they can't be done with Gtk+ only: http://static.fi/~zeenix/tmp/boxes-prop-animation.webm
Thread about clutter-gtk usage, and how key events (my main concern) are being implemented. Feb 14 17:03:27 <API> cosimoc, hi, when I find some spare moments, I'm trying to take a look to clutter-gtk a11y Feb 14 17:03:37 <API> as gnome-documents is one of the modules right now using clutter-gtk Feb 14 17:03:46 <rishi> API: It dropped clutter-gtk. Feb 14 17:03:52 <API> how gnome-documents uses clutter-gtk? Feb 14 17:03:54 <cosimoc> API, I actually dropped clutter-gtk this week Feb 14 17:03:54 <API> ah Feb 14 17:04:11 <cosimoc> API, that doesn Feb 14 17:04:11 <API> well, so it seems that the answer was easy Feb 14 17:04:14 <mccann> hmm known_diagonals[] might be a little out of date :) Feb 14 17:04:23 * c-a_ has quit (Read error: 131 (Connection reset by peer)) Feb 14 17:04:25 <cosimoc> API, that doesn't mean it's worth making it work :) Feb 14 17:04:26 <API> in that case sorry for asking ;) Feb 14 17:04:35 <API> I saw clutter-gtk on the dependencies list this monday Feb 14 17:04:51 <cosimoc> API, but I expect other similar apps to gradually move away from clutter-gtk in the future Feb 14 17:05:04 <API> well yeah, but it is easier (imho) looking at that using real apps Feb 14 17:05:10 <API> well yes Feb 14 17:05:19 <API> in fact Company mentioned that Feb 14 17:05:23 <API> some weeks ago Feb 14 17:05:25 <cosimoc> Boxes is still on clutter-gtk, don't know if it's going to be ported for 3.8 Feb 14 17:05:38 <API> that people shouldn't use clutter-gtk as his dead is already being planned Feb 14 17:05:54 <API> the thing is that in theory, one could use clutter-gtk Feb 14 17:05:56 * c-a_ (~c-a@195.176.94.190) has joined #gnome-os Feb 14 17:05:56 <cosimoc> yeah Feb 14 17:06:09 <API> to have full gtk widgets and full clutter widgets Feb 14 17:06:15 <API> but afaik, most people are just using gtk widgets Feb 14 17:06:23 <API> and use the clutter stuff for effects Feb 14 17:06:37 <cosimoc> if you use clutter-gtk you're mostly interested in clutter for animations Feb 14 17:06:41 <cosimoc> not as a toolkit Feb 14 17:06:51 <API> if that is the case, I think that having something working from a11y pov would be easier Feb 14 17:07:11 <API> in fact the main reason of me pinging you was about asking if you were doing that Feb 14 17:07:15 <API> "just effects" Feb 14 17:07:19 <cosimoc> so essentially there are some clutter containers in a GTK hierarchy Feb 14 17:07:21 <cosimoc> yes Feb 14 17:07:23 <API> cosimoc, do you know how boxes is using clutter-gtk? Feb 14 17:07:31 <cosimoc> same way as Documents Feb 14 17:07:36 <API> cosimoc, yeah I took a look to the library, and I say the containers Feb 14 17:07:52 <API> so adding a11y stuff on each container should (in theory) be easy Feb 14 17:07:56 <API> just adding some ATK stuff Feb 14 17:08:09 <API> the complex thing would be the global stuff, like events and key events Feb 14 17:08:15 <API> anyway, I will not bother you too much Feb 14 17:08:19 <API> thanks for the info Feb 14 17:08:34 <cosimoc> API, we never use key events from Clutter in those scenarios Feb 14 17:09:12 <cosimoc> API, the plan sounds good to me, as long as ATK can generate a mixed Clutter/GTK hierarchy of containers, it should just work for the typical use Feb 14 17:09:24 <API> cosimoc, that is what I hope Feb 14 17:09:34 * fmuellner has quit (Ping timeout: 600 seconds) Feb 14 17:09:40 <API> the other tricky part is ensure that the root object is the gtk one Feb 14 17:09:52 <API> other day I asked ebassi and he told me that the root object is a gtkwindow Feb 14 17:10:17 <cosimoc> yes, the toplevel is always a GtkWindow, or a subclass of Feb 14 17:10:32 * stefw is now known as stefw_afk Feb 14 17:11:47 <API> hmm doing a jhbuild list gnome-documents, it still list clutter-gtk as a dependency Feb 14 17:11:59 * calvaris (~calvaris@12.98.117.91.dynamic.mundo-r.com) has joined #gnome-os Feb 14 17:12:02 <cosimoc> let's fix the moduleset Feb 14 17:12:16 <API> cosimoc, ok, thanks Feb 14 17:13:07 <cosimoc> fixed now Feb 14 17:13:36 * doshitan has quit (Ping timeout: 600 seconds) Feb 14 17:17:41 <ebassi> API: the top-level in an app using clutter-gtk cannot be anything else but a GtkWindow; gtk does not react very well at being embedded inside another toolkit's top-level Feb 14 17:18:08 <API> yeah, I already understood that last time we talked Feb 14 17:18:13 <API> one of the issues right now is that Feb 14 17:18:22 <ebassi> API: so it's basically guaranteed that everything inside a clutter-gtk app is using gtk as the top-level, even if it's not using gtk widgets embedded inside clutter :-) Feb 14 17:18:23 <API> when clutter-gtk is initialized Feb 14 17:18:39 <API> is the clutter a11y root object the one used Feb 14 17:18:53 <API> atk ask for a root object when it starts Feb 14 17:19:03 <API> so thats one of the things to fix Feb 14 17:19:15 <API> about the previous comment about my concerns on key events Feb 14 17:19:23 <API> one question Feb 14 17:19:46 <ebassi> API: right now, we initialize gtk first, and clutter second, so the gtk root a11y node should be created first, if it's done at init time Feb 14 17:20:00 * fmuellner (~fmuellner@nat-pool-mad-t.redhat.com) has joined #gnome-os Feb 14 17:20:12 <API> yeah I looked to the code Feb 14 17:20:18 <API> but when clutter is loaded Feb 14 17:20:24 <ebassi> API: if you add API to defer the initialization of the a11y root node from clutter, I can definitely use it in clutter-gtk Feb 14 17:20:25 <API> it overrides the root object Feb 14 17:20:33 <API> yes I was thinking about that Feb 14 17:20:39 <API> or just adding an API to set the root object Feb 14 17:20:49 <API> right now there are a atk_get_root Feb 14 17:20:50 <ebassi> API: for instance, a cally_set_root (gtk_get_a11y_root ()) Feb 14 17:20:55 <API> so probably a atk_set_root Feb 14 17:21:25 <API> yes something like that, I would need to think a little about that to see what would be the saner API Feb 14 17:21:38 <API> and about my previous concern on key events Feb 14 17:21:42 <API> on a clutter-gtk app Feb 14 17:22:17 <API> do all the events go thought the gtk_main_do_event? Feb 14 17:22:27 <API> or only the ones affecting gtk widgets? Feb 14 17:24:58 <ebassi> API: events in clutter-gtk are just gtk events; key events are a bit special because of the impedence mismatch between gdk and clutter in determining which widget should get the events Feb 14 17:25:26 <ebassi> API: so for key events we just take the GDK event on the GtkClutterEmbed widget and we repackage them into the equivalent Clutter events Feb 14 17:26:02 <alex> API: gnome-boxes is currently using clutter-gtk, and will likely keep doing so for a while. However, a11y wise its a bit fucked up anyway, since there is now way to connect the gtk a11y stack to the a11y stack running in the vm, so I assume its not gonna be super useful with a11y anyway Feb 14 17:26:02 <ebassi> API: in practice, though, people have started disabling that kind of event handling, and just disable key event delivery to Clutter actors Feb 14 17:26:16 <mccann> rtcm, cosimoc, halfline better? http://fpaste.org/6xyJ/ Feb 14 17:26:42 <API> ebassi, disabling key event delivery? Feb 14 17:26:43 <API> ah Feb 14 17:26:43 <alex> ebassi: yeah, it doesn't work for gtk+ as the clutter event delivery is delayed, so we can't report if key events were handled Feb 14 17:26:49 <API> ok, so using it just for effects Feb 14 17:27:14 <API> so in that sense the key events are just relevant for the gtk widgets, right? Feb 14 17:27:14 <ebassi> API: yes, key events are just delivered to embedded widgets, not to the actors Feb 14 17:27:15 <alex> API: yeah, basically treat GtkClutterEmbedd as a fancy Gtk container Feb 14 17:27:42 <API> in that sense, I assume that the key snooping would should be still working on those gtk widgets Feb 14 17:27:43 <API> right? Feb 14 17:28:07 <ebassi> alex: there's also something else; for instance, if I install a filter function, I don't get key events from GDK/X11 anyway Feb 14 17:28:11 <ebassi> API: yes Feb 14 17:28:49 <ebassi> alex: that's why there's the workaround with the GDK → Clutter event translation Feb 14 17:28:50 <alex> ebassi: although with the new gtk frame stuff we could maybe get the key event emission right Feb 14 17:28:51 <API> ebassi: ok, this is good, is what I told before to cosimoc. If the usage of clutter-gtk was mostly focused for effects, I hoped about being able to get a11y working Feb 14 17:28:54 <API> more or less easy Feb 14 17:29:13 <API> it would be trickier if having a real mixed clutter-widgets and gtk-widgets environment Feb 14 17:29:32 <API> alex, about your comment "I assume its not gonna be super useful with a11y anyway" Feb 14 17:29:32 <ebassi> the problem with saying "clutter-gtk is going to die" is that it's going to die when gtk starts getting based on clutter, which won't happen soon :-) Feb 14 17:29:42 * fcrozat has quit (Read error: 131 (Connection reset by peer)) Feb 14 17:30:01 <API> it something I have been thinking since cosimoc said that clutter-gtk will be dropped by most modules Feb 14 17:30:15 <API> in the sense of "it is worth to really worry about a11y here?" Feb 14 17:30:33 <API> where worry==use my time here Feb 14 17:30:33 <ebassi> gtk can get fancy layout management and CSS transitions, but there's no animation framework, so you can simply do a transition in code without reimplementing half of Clutter's API anyway Feb 14 17:30:37 <cosimoc> mccann, looks good to me Feb 14 17:31:03 <alex> API: Like, you can use a11y to give focus to the VM, but once you're there the VM is just some pixels from the Gtk+ perspective Feb 14 17:31:03 <mccann> thanks I'll push it Feb 14 17:31:07 <API> but as ebassi said, although there are that clutter gtk, it is not planned to happen *now*, so clutter-gtk will be around for a while Feb 14 17:31:18 <alex> API: I guess if a11y is set up correctly inside the VM it might work... Feb 14 17:31:35 <API> alex, well, but it is more a general thought, not just focused on boxes Feb 14 17:31:41 <alex> yeah
(In reply to comment #2) > (In reply to comment #1) > > As I was not sure if clutter-gtk was a library with real usage on the GNOME > > ecosystem (after gnome-documents droppage of his use) I asked on d-d-l: > > > > https://mail.gnome.org/archives/desktop-devel-list/2013-February/msg00059.html > > > > Although the feedback was not big, at least the conclusion was that Boxes > > doesn't plan to change that. > > Actually its more like Boxes *can't* change that. Yes true. Benjamin Otte explained me that today on IRC. Sorry for the vague explanation. > We have animations that docs > does not have and they can't be done with Gtk+ only: > http://static.fi/~zeenix/tmp/boxes-prop-animation.webm Ok, thanks.
I have been taking a look to this bug during the afternoon using gnome-control-center and basic clutter-gtk example. Here my discoveries: * About the root object: As Bastien mentioned on IRC, for gnome-control-center and basic clutter-gtk examples, the root was correct (the gtk one). This was because when gtk is initialized, at-spi2-atk calls for the root, and uses it. When I made my tests, I was always from the server side, and when clutter loads its own implementation, it overrides too atk_get_root. * Key events: but as the clutter accessibility is initialized, it loads its own AtkUtil implementation. So also loads clutter key event registering implementation, as as concluded on comment 3, we want the gtk one. So it was somewhat tricky, as was using Gtk root object but Clutter AtkUtil implementation. Anyway, as explained on comment 3, right now clutter-gtk is basically used for effects, and for the long talk I had today on IRC with Bastien and Benjamin, in order add video/audio widgets. And in that sense, for those cases, we just want the a11y support for gtk. I think that for those cases we don't need the ATK objects for the containers at clutter-gtk for that case. As a conclusion, an option for gnome-control-center could be just disable clutter accessibility. We need to test it. For the case of boxes that would be somewhat more tricky, as Alex (larrson?) said.
Any progress on this? We'll now have a lot of clutter-gtk in Maps too and essential UI components will have to be inside clutter actors so this is becoming more and more important. Also see the bug I'm adding to 'blocks' here that I assume is symptom of this bug(?).
*** Bug 706622 has been marked as a duplicate of this bug. ***
(In reply to comment #6) > Any progress on this? Unfourtunately not. On the case of gnome-control-center and empathy, the only thing that we fixed is ensure that clutter accessibility is not activated for those cases, as in those cases clutter was used only to get some effects (fwiw, I'm talking about bug 691468). > We'll now have a lot of clutter-gtk in Maps too and > essential UI components will have to be inside clutter actors so this is > becoming more and more important. Will take a look to gnome-maps. Anyway I will repeat something I already said on the past. Some people in GNOME tells me that clutter-gtk will die soon, and is in minimum-maintainance status. In one specific case, one person says that clutter-gtk shouldn't have existed at all. In most cases, it was told that was a lesser evil in order to add some effects on gtk apps + gstreamer improved support. But now and then people says that clutter-gtk is becoming more and more important. Is really complex to prioritize our scarce resources without a clear message. FWIW, right now our priorities (using an oversimplified summary) are providing the proper accessibility support for: a) gtk and gtk applications b) gnome-shell c) accessibility tools (including infrastructure and tools per-se) needed for a) and b) > Also see the bug I'm adding to 'blocks' here > that I assume is symptom of this bug(?). Probably not. As far as I understand bug 706622, reporter is talking about buttons, so I understand that is still about gtk (unless those buttons were drawn by clutter, and include on the clutter-stage).
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version of clutter-gtk, then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a ticket at https://gitlab.gnome.org/GNOME/clutter-gtk/-/issues/ Thank you for your understanding and your help.