GNOME Bugzilla – Bug 584970
instrument the shell to measure efficiency and effectiveness
Last modified: 2021-07-05 14:22:33 UTC
Might be interesting to instrument the shell to measure efficiency and effectiveness as well as usage patterns. May be useful during user testing. Maybe measure things like: * Time spent in overlay * Time between visits to overview * Number of open windows (peak and mean) * Number of windows visible * Number of workspaces * Use of button, hotcorner, or keyboard to activate overview * Time before second workspace is created * Display form-factor/resolution * Single/multiple monitor * Use of alt-tab vs. overview * etc.
I second that idea! It'd be useful in overall UI tuning in other applications also. Maybe another GNOME infrastructure module ? P.S. But you also need people who will process that huge amount of collected data into usability improvement ideas. :)
(In reply to comment #1) > I second that idea! > It'd be useful in overall UI tuning in other applications also. > Maybe another GNOME infrastructure module ? I like the idea of an extension as Filipe proposed here: http://mail.gnome.org/archives/gnome-shell-list/2012-January/msg00077.html Though, I'm not sure that gnome extension users are representative sample of gnome users. > P.S. But you also need people who will process that huge amount of collected > data into usability improvement ideas. :) Sure. But it would be as well a good resource for anyone working an bug/extension/feature, if the data is easily accessible. I think it's quite common to run into something that would benefit from such data. I'm be interested in some demographics, too. Users could answer some optional questions on extension installation I'm not sure what exactly. Something like: computer experience, profession, age..
There could be a couple of approaches to carrying out this testing. One would be to have the extension openly available, and wait for people to install it and send us their reports. This has the problem that it would give us results from a very narrow kind of people, who are already technical experts. We do not need hundreds of participants as much as we need that they are representative of typical intermediate users. For instance, Nielsen recommends testing with 20 users when collecting quantitative usability metrics: http://www.useit.com/alertbox/quantitative_testing.html So, the other option would be to develop this extension, and then have some GNOMErs recruit participants (e.g. friends, relatives) and set up the experiment for them. I guess that it should not be hard to recruit a couple dozen people this way. The experiment itself would need to be as automated as possible, so one of us could simply go to our friend's computer, download and install the extension, and upload the results a couple of weeks later (together with some general demographic info on the participant). We can discuss if it would make sense to make the data publicly available (after anonimizing, of course), or if only some selected people should have access to it. As for the technical part, I don't really know much about the detailed implementation of the Shell and how we could write this experimental extension (maybe it could be built on top of the a11y infrastructure???).
(In reply to comment #3) > > As for the technical part, I don't really know much about the detailed > implementation of the Shell and how we could write this experimental extension > (maybe it could be built on top of the a11y infrastructure???). Well, an extension have direct access to some parts of GNOME Shell (AFAIK). Current accessibility infrastructure on GNOME Shell (still a WIP) just expose GNOME Shell internals to accessibility technologies. So I feel that it would be more direct to solve this problem writing an extension and forget the accessibility nits. accessibility infrastructure could have sense if the proposal was create a external tool to implement that feature. ie (crazy idea): accerciser has support for plugins. Create a accerciser plugin recording while you interact with the GNOME Shell. And remember that in this case, accerciser or any other accessibility-based solution, will receive information in a accessibility abstract way (that information would be based on ATK signals, ATK roles etc). Taking into account that there are already a procedure to interact directly with GNOME Shell internals (extensions), I don't see any advantage by using accessibility infrastructure.
(In reply to comment #4) > > Taking into account that there are already a procedure to interact directly > with GNOME Shell internals (extensions), I don't see any advantage by using > accessibility infrastructure. Thank you, good point. Let's investigate the extension way, it seems the best approach in the case of GNOME Shell.
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a new ticket at https://gitlab.gnome.org/GNOME/gnome-shell/-/issues/ Thank you for your understanding and your help.