GNOME Bugzilla – Bug 693769
tests/a11y/pickers.ui fails very often
Last modified: 2013-09-17 04:11:43 UTC
"make check" in GTK often fails on this test: /home/ubuntu/gnome/checkout/gtk+/tests/a11y/pickers.ui: FAIL GTester: last random seed: R02Se31d29409c70da73e2953ad1fae49575 E. g. in the current log of jhbuild on Jenkins: https://jenkins.qa.ubuntu.com/view/Raring/view/JHBuild%20Gnome/job/jhbuild-amd64-gtk-/141/artifact/gtk+.log This already failed on the same test case in earlier runs, then has worked for a while, and now fails again (see build history on https://jenkins.qa.ubuntu.com/view/Raring/view/JHBuild%20Gnome/). I can reproduce the very same failure in my local jhbuild on my workstation.
That's the verbose output on my workstation: $ jhbuild run ./accessibility-dump -p /home/martin/upstream/gtk+/tests/a11y/pickers.ui --verbose GTest: random seed: R02S88194ad77fe9e4884c155df35e5e02e3 GTest: run: /home/martin/upstream/gtk+/tests/a11y/pickers.ui (MSG: Contents don't match expected contents: --- /home/martin/upstream/gtk+/tests/a11y/pickers.txt 2013-02-06 10:58:53.000000000 +0100 +++ /tmp/.MTU8RW 2013-02-14 09:45:47.699538730 +0100 @@ -80,7 +80,7 @@ "menu item" parent: unnamed-GtkMenuAccessible-2 index: 1 - name: Desktop + name: File System state: enabled selectable sensitive visible toolkit: gtk <AtkComponent> @@ -91,18 +91,15 @@ action 0 description: Clicks the menuitem <AtkSelection> unnamed-GtkMenuItemAccessible-5 - "menu item" + "separator" parent: unnamed-GtkMenuAccessible-2 index: 2 - name: File System state: enabled selectable sensitive visible toolkit: gtk <AtkComponent> layer: popup alpha: 1 <AtkAction> - action 0 name: click - action 0 description: Clicks the menuitem <AtkSelection> unnamed-GtkMenuItemAccessible-6 "separator" @@ -116,20 +113,9 @@ <AtkAction> <AtkSelection> unnamed-GtkMenuItemAccessible-7 - "separator" - parent: unnamed-GtkMenuAccessible-2 - index: 4 - state: enabled selectable sensitive visible - toolkit: gtk - <AtkComponent> - layer: popup - alpha: 1 - <AtkAction> - <AtkSelection> - unnamed-GtkMenuItemAccessible-8 "menu item" parent: unnamed-GtkMenuAccessible-2 - index: 5 + index: 4 name: Other? state: enabled selectable sensitive visible toolkit: gtk @@ -184,7 +170,7 @@ action 0 name: press action 0 description: Presses the combobox <AtkSelection> - unnamed-GtkMenuAccessible-9 + unnamed-GtkMenuAccessible-8 "menu" parent: button1 index: 0 ) GTest: result: FAIL Abgebrochen (Speicherabzug geschrieben)
It depends on what devices show up via gvfs. We've tried to nail that down by adding GIO_USE_VFS=local to the environment. Not quite good enough, it seems...
(In reply to comment #2) > It depends on what devices show up via gvfs. > We've tried to nail that down by adding GIO_USE_VFS=local to the environment. Do we? a "git grep GIO_USE_VFS" doesn't turn up anything in gtk+ nor jhbuild. But anyway, I always get exactly the same diff in all those cases: env -u DBUS_SESSION_BUS_ADDRESS jhbuild run ./accessibility-dump --verbose env GIO_USE_VFS=local jhbuild run ./accessibility-dump --verbose jhbuild run dbus-launch ./accessibility-dump --verbose The latter won't help much of course as it will just respawn gvfs on the new local bus, but the other two should at least help if it's really a gvfs issue? When I run the verbose mode on the jenkins jhbuild machine, where there is no system nor session D-BUS, no local gsettings, etc, the diff is slightly different: --- /home/ubuntu/gnome/checkout/gtk+/tests/a11y/pickers.txt 2013-02-08 18:29:09.800606486 +0000 +++ /tmp/.UA6LSW 2013-02-14 15:51:26.188309918 +0000 @@ -116,20 +116,9 @@ <AtkAction> <AtkSelection> unnamed-GtkMenuItemAccessible-7 - "separator" - parent: unnamed-GtkMenuAccessible-2 - index: 4 - state: enabled selectable sensitive visible - toolkit: gtk - <AtkComponent> - layer: popup - alpha: 1 - <AtkAction> - <AtkSelection> - unnamed-GtkMenuItemAccessible-8 "menu item" parent: unnamed-GtkMenuAccessible-2 - index: 5 + index: 4 name: Other? state: enabled selectable sensitive visible toolkit: gtk @@ -184,7 +173,7 @@ action 0 name: press action 0 description: Presses the combobox <AtkSelection> - unnamed-GtkMenuAccessible-9 + unnamed-GtkMenuAccessible-8 "menu" parent: button1 index: 0 ) In particular, the - name: Desktop + name: File System bit is not there.
Oh, I found something interesting: On my local workstation, the test works if I redirect $XDG_CONFIG_HOME: env XDG_CONFIG_HOME=/tmp jhbuild run ./accessibility-dump --verbose So maybe it does look at some gsetting of mine? However, that doesn't work on the jenkins server, I still get the very same diff with that. If that's so hard to get right, would it maybe be better to just check if the file has a few key items, instead of comparing it against a full "expected" one?
Sorry, not GIO_USE_VFS, we set GIO_USE_VOLUME_MONITOR: http://git.gnome.org/browse/gtk+/tree/tests/a11y/Makefile.am#n22
a11y tests also fail because we get font setup differences between a naked Xvfb and a full gnome session, and those affect the reported values of some adjustments.
not sure there's anything to do here. suggestions would be welcom