GNOME Bugzilla – Bug 703647
Running tests in specific order causes various failures in test_object_marshaling
Last modified: 2013-07-06 18:31:14 UTC
I am seeing a testsuite regression with pygobject 3.8.3 on Fedora i686 builders, compared to the previous 3.8.2 release. On x86_64 the same test passes. test_uninitialized_object (test_gobject.TestReferenceCounting) ... ok ====================================================================== FAIL: test_vfunc_in_object_transfer_full_with_floating (test_object_marshaling.TestVFuncsWithFloatingArg) ---------------------------------------------------------------------- Traceback (most recent call last):
+ Trace 232191
self.assertEqual(ref_count, 0)
====================================================================== FAIL: test_vfunc_in_object_transfer_none_with_floating (test_object_marshaling.TestVFuncsWithFloatingArg) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 268, in test_vfunc_in_object_transfer_none_with_floating self.assertTrue(vfuncs.in_object_is_floating) AssertionError: None is not true ====================================================================== FAIL: test_vfunc_in_floating_transfer_full_with_held_floating (test_object_marshaling.TestVFuncsWithHeldFloatingArg) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 547, in test_vfunc_in_floating_transfer_full_with_held_floating self.assertTrue(held_object_ref() is None) AssertionError: False is not true ====================================================================== FAIL: test_vfunc_in_floating_transfer_none_with_held_floating (test_object_marshaling.TestVFuncsWithHeldFloatingArg) ---------------------------------------------------------------------- Traceback (most recent call last): File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 513, in test_vfunc_in_floating_transfer_none_with_held_floating self.assertTrue(vfuncs.in_object_is_floating) AssertionError: None is not true ---------------------------------------------------------------------- Ran 879 tests in 12.956s FAILED (failures=4, expected failures=4, unexpected successes=2) ('source id:', 113L) make[2]: *** [check-local] Error 1 make[2]: Leaving directory `/builddir/build/BUILD/pygobject-3.8.3/tests'
I am unable to reproduce this in either a 32bit chroot environment or a 32 bit VM running Fedora 19. Can you provide more specifics on how this is being run, i.e. using jhbuild or even a link to information on how Fedora builders work? Thanks!
Compiler flags would also be useful.
Meh, it seemed to be reliably failing yesterday, but today I managed to get 7 successful runs and only the 8th one failed. I suppose it's a race somewhere. The build environment is the official Fedora koji builders. The builders use a tool called 'mock' to install a minimal chroot environment based on the build dependencies from rpm spec files, which makes the build environment reproducible. This is the link to a failed koji i686 task: http://koji.fedoraproject.org/koji/taskinfo?taskID=5579932 http://kojipkgs.fedoraproject.org//work/tasks/9932/5579932/build.log The build.log there has all the details, the build flags used and the commands executed. And for comparison, another i686 task that completed successfully, in the same environment: http://koji.fedoraproject.org/koji/taskinfo?taskID=5579930 http://kojipkgs.fedoraproject.org//work/tasks/9930/5579930/build.log The compiler flags used are: -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables And linker flags: -Wl,-z,relro The tests are run under xvfb, with the following command: xvfb-run make DESTDIR=/builddir/build/BUILDROOT/pygobject3-3.8.3-1.fc19.i386 check V=1
That actually helps a lot, thanks. If you look at the successful vs. failing, the order in which the tests are being run is different which may help reproduce this consistently.
Indeed running the tests in the order shown with the failing suite will reproduce this. The problem is not actually specific to 32 bits or the 3.8.3 release (it is also in 3.8.2...) For the comedy: TEST_NAMES=$(wget -q -O- http://kojipkgs.fedoraproject.org//work/tasks/9932/5579932/build.log | sed -n 's/^test_.* (\(.*\)\..*) .*$/\1/p' | uniq | tr '\n' ' ') make check And for the record: TEST_NAMES="test_thread test_properties test_gio test_internal_api test_overrides_glib test_overrides_gdk test_gi test_signal test_option test_gtype test_gdbus test_iochannel test_mainloop test_generictreemodel test_glib test_overrides_pango test_atoms test_everything test_interface test_subprocess test_object_marshaling" make check
Simplified: make check TEST_NAMES="test_thread test_object_marshaling" This is problematic not because of some threading issue, but because test_thread imports testhelper before GObject. https://git.gnome.org/browse/pygobject/tree/tests/test_thread.py?id=3.8.3#n4 The module C extension attempts to import "gi._gobject._gobject" which is the static bindings without the overrides. https://git.gnome.org/browse/pygobject/tree/tests/testhelpermodule.c?id=3.8.3#n618 I'm guessing the problem here is there is some sort of type caching going on which causes the issue. The test helper C extension should be changed to import "gi.repository.GObject".
The following fix has been pushed: 075a11a testhelper: Fix import requirement for GObject
Created attachment 248530 [details] [review] testhelper: Fix import requirement for GObject Replace the importing of gi._gobject._gobject with gi.repository.GObject in tests/testhelpermodule.c The testhelper module was only importing the static bindings (gi._gobject._gobject) and not the overrides (gi.repository.GObject). This was causing some tests to fail when test_thread was the first test to run in the suite due to it registering new types based on PyGObject_Type.
Pushed to 3.8 branch here: https://git.gnome.org/browse/pygobject/commit/?h=pygobject-3-8&id=075a11ad29f04
Awesome, thanks for the investigation and the fixes, Simon!