GNOME Bugzilla – Bug 727453
New API to get set size, position, and level
Last modified: 2021-06-10 11:26:46 UTC
Currently there are a number of overlapping ways to get numeric and hierarchical details: Set size: 1. the parent object's child count 2. the accessible relation targets from NODE_PARENT_OF 3. counting all the accessibles with NODE_CHILD_OF with the same target 4. an object attribute Position in set: 1. the object's index in parent 2. getting the set of logical children (using 2 or 3 above), and using the index with respect to the logical children 3. an object attribute Hierarchical level: 1. Ascending the accessible hierarchy looking at nesting within real ancestors 2. Ascending the accessible hierarchy via NODE_PARENT_OF targets 3. an object attribute To complicate matters, for any given application or toolkit, the expected way might vary and multiple ways might be in place (and holding contradictory information). One must also take into account things like the accessible state set (e.g. presence or absenve of STATE_MANAGES_DESCENDANTS) and role (ROLE_LIST_ITEM vs ROLE_TABLE_CELL vs ROLE_HEADING), along with and toolkit-specific hierarchical variations (e.g. table cell children of table cells as found in Gtk+). All of this just so Orca can reliably present things like "Foo, item 3 of 5" or "Bar, expanded 7 items, tree level 2". ;) Thus it would be extremely helpful to be able to stop doing all of the above and have something like: * atk_object_get_set_size () * atk_object_get_position_in_set () * atk_object_get_level ()
(In reply to comment #0) > Currently there are a number of overlapping ways to get numeric and > hierarchical details: > > Set size: > 1. the parent object's child count > 2. the accessible relation targets from NODE_PARENT_OF > 3. counting all the accessibles with NODE_CHILD_OF with the same target My first reaction after reading current ATK documentation for NODE_PARENT_OF and NODE_CHILD_OF was about removing them, as I didn't understand why if they are children, they don't appear as part of the parent object children set (so count). In summary, I didn't see them useful. Taking into account that IA2 also have that relation, and after reading its documentation [1] (that probably we should re-use) I moved from "sure to remove them" to "not sure about remove them". Why do we want a relation parent-child out that are not part of the hierarchy? That means that implementors need to expose some children as part of the children set, and then mark other elements as children. Shouldn't be easier to just expose all them as children? Probably we should ask about that to implementors, as I never needed to use that relation. As far as I see, it is implemented on both gtk+ and firefox. But, for example, it is not used at WebKitGtk. And after googling a little, I don't see a equivalent relation on Macos. > 4. an object attribute Could you provide an example of this? Sounds like a kind of hack (put stuff under the carpet), but I can be wrong. > All of this just so Orca can reliably present things like "Foo, item 3 of 5" or > "Bar, expanded 7 items, tree level 2". ;) Thus it would be extremely helpful to > be able to stop doing all of the above and have something like: > > * atk_object_get_set_size () > * atk_object_get_position_in_set () Ideally, if we can get rid of those relations, that API would not be needed, making the API (and our life) more simple. > * atk_object_get_level () Makes sense, but a question, is the level inside a specific container or starting to count since the top-level (application) container? [1] http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/group__grp_relations.html#ga2c0cecaf4accc079f9a78a6c413d040a
(In reply to comment #1) > My first reaction after reading current ATK documentation for NODE_PARENT_OF > and NODE_CHILD_OF was about removing them And in doing so, breaking quite a bit. So I'm glad you didn't. > hierarchy? That means that implementors need to expose some children as part of > the children set, and then mark other elements as children. Shouldn't be easier > to just expose all them as children? You'd have to ask the implementors of tree views and other hierarchical objects which manage their descendants. But given all the stuff we currently have, my guess is that the answer is "no." What if the number of children is in the millions (example a Calc spreadsheet)? We don't want all those exposed as children. It would be worse in the case of a hierarchical table -- more so in a sortable hierarchical table: Just because you resort a table doesn't mean the accessible hierarchy should be completely dumped and redone. > > 4. an object attribute > > Could you provide an example of this? Sounds like a kind of hack (put stuff > under the carpet), but I can be wrong. Look at http://www.w3.org/TR/wai-aria-implementation in the mapping table for: * aria-level * aria-posinset * aria-setsize > Ideally, if we can get rid of those relations, that API would not be needed, > making the API (and our life) more simple. I personally do not see this happening. Huge data tables, at the very least, will likely continue to have this need. Tables which can be dynamically manipulated will too I think. > > * atk_object_get_level () > > Makes sense, but a question, is the level inside a specific container or > starting to count since the top-level (application) container? It will depend on the object. For a heading it will just be the assigned heading. For a Gtk+ tree view it will be based on node level w.r.t. that tree view. In the case of ARIA, it will be whatever aria-level says it is.
Some drive-by comments, so I haven't fully thought them through. (1) This doesn't belong at the AtkObject level really. An object is not guaranteed to be part of a set, so calling atk_object_get_pos_in_set() is invalid on 99% of cases. (2) Can objects belong to more than 1 set? (3) The naming is ugly. Sets are generally understood to be unordered and ordered sets are called a list. I suppose if we just mirror ARIA the naming makes sense though (in that case ARIA uses ugly naming and they deserve the shame). (4) What's this intended to be for in the real world? Only treeviews? In short, it seems like one of these "ugh" APIs that tries to be overly broad to defines something that really has only very few (maybe just one) use case.
(In reply to comment #3) > (1) This doesn't belong at the AtkObject level really. An object is not > guaranteed to be part of a set, so calling atk_object_get_pos_in_set() is > invalid on 99% of cases. The set is the collection of functional siblings. For the average AtkObject, that coincides with all of the children of the parent of that object. But there are a number of cases where that is not true. > (2) Can objects belong to more than 1 set? Hmmmm.... At any given time I would think/hope not. Objects could become reparented, but then they'd change set; not join a new one. > (4) What's this intended to be for in the real world? Only treeviews? No, ideally all objects. From the AT side of things (my world view), one should be able to identify that a given object is the nth "child" out of a total of x "children" -- and do so via a single method call for each of those two pieces of information, and without having to know if an object manages its descendants, or is ARIA, or comes from a particular toolkit, etc. > In short, it seems like one of these "ugh" APIs that tries to be overly broad > to defines something that really has only very few (maybe just one) use case. No the problem, as I see it, is the opposite. Orca has a bunch of checks based on toolkit, role, state, ARIA or not, etc. as described in the opening report.
Would you expect a button in a dialog box to report it's the 3rd of 5 buttons? Or a toolitem in a toolbar to report it's 7 of 9? (Though that'd be tricky semantics for toolbuttons because separators are toolitems, too...)
(In reply to comment #2) > > hierarchy? That means that implementors need to expose some children as part of > > the children set, and then mark other elements as children. Shouldn't be easier > > to just expose all them as children? > > You'd have to ask the implementors of tree views and other hierarchical objects > which manage their descendants. But given all the stuff we currently have, my > guess is that the answer is "no." > > What if the number of children is in the millions (example a Calc spreadsheet)? Yes, that was the idea behind ATK_STATE_MANAGES_DESCENDANT state. > We don't want all those exposed as children. It would be worse in the case of a > hierarchical table -- more so in a sortable hierarchical table: Just because > you resort a table doesn't mean the accessible hierarchy should be completely > dumped and redone. But there are an inherent list of children with a specific order, that this new API proposed (AFAIU) would expose, and need to be recomputed somehow if resort. Additionally, when you pinged Benjamin at IRC, you mentioned this: "Apr 03 13:25:15 <joanie> Company: you have opinions on bug 727453? Apr 03 13:25:16 <Services> Bug http://bugzilla.gnome.org/show_bug.cgi?id=727453 enhancement, Normal, ---, atk-maint, UNCONFIRMED, New API to get set size, position, and level Apr 03 13:25:37 <joanie> This might, at least in part, help solve the AtkRelation-create-all-tree-view-children-at-runtime issue Apr 03 13:25:41 <joanie> not sure though" So I'm somewhat confused. If you are saying that this API will now replace the need for those relations, how this API proposed would help to solve part of the problem with adding those relations? > > > 4. an object attribute > > > > Could you provide an example of this? Sounds like a kind of hack (put stuff > > under the carpet), but I can be wrong. > > Look at http://www.w3.org/TR/wai-aria-implementation in the mapping table for: > * aria-level > * aria-posinset > * aria-setsize Ok, thanks. > > Ideally, if we can get rid of those relations, that API would not be needed, > > making the API (and our life) more simple. > > I personally do not see this happening. Huge data tables, at the very least, > will likely continue to have this need. Tables which can be dynamically > manipulated will too I think. > > > > * atk_object_get_level () > > > > Makes sense, but a question, is the level inside a specific container or > > starting to count since the top-level (application) container? > > It will depend on the object. I'm concerned about that "I will depend". I would prefer something more cohesive. We already have problems due loosely defined concepts (like the already mentioned ATK_STATE_MANAGES_DESCENDANT), that implementors need to guess how to implement. In some cases that leads to different toolkits implementing a different behaviour for the same thing. In short, I would prefer a definition that clearly says to implementors what they should expose. > For a heading it will just be the assigned > heading. For a Gtk+ tree view it will be based on node level w.r.t. that tree > view. In the case of ARIA, it will be whatever aria-level says it is. So in the case of aria is a just a copy of the value, but in other cases depends? See my previous concern.
(In reply to comment #5) > Would you expect a button in a dialog box to report it's the 3rd of 5 buttons? No. Because in that case there's not a conceptual set per se. So the default fallback would make sense. So the set size would be the child count for the parent of that button. The position would be the index in parent. > Or a toolitem in a toolbar to report it's 7 of 9? (Though that'd be tricky > semantics for toolbuttons because separators are toolitems, too...) Separators in menus are too. So looking at the cases in the opening report, this is yet another one. For menus we have to iterate through all the children just to find any separators that might be there and prune them from the "set." Oh, and in a GtkNotebook, we have to check for and prune hidden/not-showing pages. It really sucks to be an AT. :P But I think that we can come up with a list of these sorts of things and address them through developer documentation. (Which would be much clearer and well-thought out than what I jotted down above.) I could add that to my to-do list if we can agree on the API.
(In reply to comment #6) > > > Makes sense, but a question, is the level inside a specific container or > > > starting to count since the top-level (application) container? > > > > It will depend on the object. > > I'm concerned about that "I will depend". I would prefer something more > cohesive. At the risk of echoing what I said in my previous comment, we will document it. The ARIA documentation could be used as a starting point perhaps. And we can flesh it out with some concrete examples: http://www.w3.org/TR/wai-aria/states_and_properties#aria-level
(In reply to comment #6) > Additionally, when you pinged Benjamin at IRC, you mentioned this: > > "Apr 03 13:25:15 <joanie> Company: you have opinions on bug 727453? > Apr 03 13:25:16 <Services> Bug > http://bugzilla.gnome.org/show_bug.cgi?id=727453 enhancement, Normal, ---, > atk-maint, UNCONFIRMED, New API to get set size, position, and level > Apr 03 13:25:37 <joanie> This might, at least in part, help solve the > AtkRelation-create-all-tree-view-children-at-runtime issue > Apr 03 13:25:41 <joanie> not sure though" > > So I'm somewhat confused. If you are saying that this API will now replace the > need for those relations, how this API proposed would help to solve part of the > problem with adding those relations? As the words you quote indicate, I'm not sure. But the reason we need those AtkRelations are to accomplish the following: 1. Get the set size. The set size of a child node is gotten by looking at the child node for the AtkRelation node-child-of. The target of that relation is the parent node. Then you look at that parent node for its AtkRelation node-parent-of. That relation will point to all the child nodes. Thus the number of targets in that relation is the set size. 2. Get the position of the child in the set. See the process for item 1 above. The position is the index of the child with respect to the child nodes. This may or may not be 100% accurate, but it's all we've got. 3. Get the level of a node. The level of a node is based on its depth with respect to the tree, so given: -> Foo -> Bar -> Baz If Foo is level 0, then Bar is level 1, and Baz is level 2. The way we figure that out is by ascending the hierarchy: Baz has AtkRelation node-child-of pointing to Bar. Bar has node-child-of pointing to Foo. Foo doesn't have node-child-of ergo it's the top of the tree. The only reason we care about node-child-of and node-parent-of is to do those calculations. If I can just ask for the level, position, and set size, I do not need those relations.
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version of atk, then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a ticket at https://gitlab.gnome.org/GNOME/atk/-/issues/ Thank you for your understanding and your help.