After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 703647 - Running tests in specific order causes various failures in test_object_marshaling
Running tests in specific order causes various failures in test_object_marsha...
Status: RESOLVED FIXED
Product: pygobject
Classification: Bindings
Component: general
3.8.x
Other Linux
: Normal normal
: ---
Assigned To: Nobody's working on this now (help wanted and appreciated)
Python bindings maintainers
Depends on:
Blocks:
 
 
Reported: 2013-07-05 11:29 UTC by Kalev Lember
Modified: 2013-07-06 18:31 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
testhelper: Fix import requirement for GObject (1.23 KB, patch)
2013-07-06 16:55 UTC, Simon Feltman
committed Details | Review

Description Kalev Lember 2013-07-05 11:29:14 UTC
I am seeing a testsuite regression with pygobject 3.8.3 on Fedora i686 builders, compared to the previous 3.8.2 release. On x86_64 the same test passes.

test_uninitialized_object (test_gobject.TestReferenceCounting) ... ok
======================================================================
FAIL: test_vfunc_in_object_transfer_full_with_floating (test_object_marshaling.TestVFuncsWithFloatingArg)
----------------------------------------------------------------------
Traceback (most recent call last):
  • File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 287 in test_vfunc_in_object_transfer_full_with_floating
    self.assertEqual(ref_count, 0)
AssertionError: 1L != 0
======================================================================
FAIL: test_vfunc_in_object_transfer_none_with_floating (test_object_marshaling.TestVFuncsWithFloatingArg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 268, in test_vfunc_in_object_transfer_none_with_floating
    self.assertTrue(vfuncs.in_object_is_floating)
AssertionError: None is not true
======================================================================
FAIL: test_vfunc_in_floating_transfer_full_with_held_floating (test_object_marshaling.TestVFuncsWithHeldFloatingArg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 547, in test_vfunc_in_floating_transfer_full_with_held_floating
    self.assertTrue(held_object_ref() is None)
AssertionError: False is not true
======================================================================
FAIL: test_vfunc_in_floating_transfer_none_with_held_floating (test_object_marshaling.TestVFuncsWithHeldFloatingArg)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builddir/build/BUILD/pygobject-3.8.3/tests/test_object_marshaling.py", line 513, in test_vfunc_in_floating_transfer_none_with_held_floating
    self.assertTrue(vfuncs.in_object_is_floating)
AssertionError: None is not true
----------------------------------------------------------------------
Ran 879 tests in 12.956s
FAILED (failures=4, expected failures=4, unexpected successes=2)
('source id:', 113L)
make[2]: *** [check-local] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/pygobject-3.8.3/tests'
Comment 1 Simon Feltman 2013-07-06 10:43:27 UTC
I am unable to reproduce this in either a 32bit chroot environment or a 32 bit VM running Fedora 19. Can you provide more specifics on how this is being run, i.e. using jhbuild or even a link to information on how Fedora builders work?

Thanks!
Comment 2 Simon Feltman 2013-07-06 10:45:32 UTC
Compiler flags would also be useful.
Comment 3 Kalev Lember 2013-07-06 12:45:32 UTC
Meh, it seemed to be reliably failing yesterday, but today I managed to get 7 successful runs and only the 8th one failed. I suppose it's a race somewhere.

The build environment is the official Fedora koji builders. The builders use a tool called 'mock' to install a minimal chroot environment based on the build dependencies from rpm spec files, which makes the build environment reproducible.

This is the link to a failed koji i686 task:
http://koji.fedoraproject.org/koji/taskinfo?taskID=5579932
http://kojipkgs.fedoraproject.org//work/tasks/9932/5579932/build.log

The build.log there has all the details, the build flags used and the commands executed.

And for comparison, another i686 task that completed successfully, in the same environment:
http://koji.fedoraproject.org/koji/taskinfo?taskID=5579930
http://kojipkgs.fedoraproject.org//work/tasks/9930/5579930/build.log

The compiler flags used are: -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -grecord-gcc-switches  -m32 -march=i686 -mtune=atom -fasynchronous-unwind-tables

And linker flags: -Wl,-z,relro

The tests are run under xvfb, with the following command:
xvfb-run make DESTDIR=/builddir/build/BUILDROOT/pygobject3-3.8.3-1.fc19.i386 check V=1
Comment 4 Simon Feltman 2013-07-06 14:11:28 UTC
That actually helps a lot, thanks. If you look at the successful vs. failing, the order in which the tests are being run is different which may help reproduce this consistently.
Comment 5 Simon Feltman 2013-07-06 15:47:55 UTC
Indeed running the tests in the order shown with the failing suite will reproduce this. The problem is not actually specific to 32 bits or the 3.8.3 release (it is also in 3.8.2...)

For the comedy:

TEST_NAMES=$(wget -q -O- http://kojipkgs.fedoraproject.org//work/tasks/9932/5579932/build.log | sed -n 's/^test_.* (\(.*\)\..*) .*$/\1/p' | uniq | tr '\n' ' ') make check

And for the record:

TEST_NAMES="test_thread test_properties test_gio test_internal_api test_overrides_glib test_overrides_gdk test_gi test_signal test_option test_gtype test_gdbus test_iochannel test_mainloop test_generictreemodel test_glib test_overrides_pango test_atoms test_everything test_interface test_subprocess test_object_marshaling" make check
Comment 6 Simon Feltman 2013-07-06 16:48:00 UTC
Simplified:

make check TEST_NAMES="test_thread test_object_marshaling"

This is problematic not because of some threading issue, but because test_thread imports testhelper before GObject.

https://git.gnome.org/browse/pygobject/tree/tests/test_thread.py?id=3.8.3#n4

The module C extension attempts to import "gi._gobject._gobject" which is the static bindings without the overrides. 

https://git.gnome.org/browse/pygobject/tree/tests/testhelpermodule.c?id=3.8.3#n618

I'm guessing the problem here is there is some sort of type caching going on which causes the issue. The test helper C extension should be changed to import "gi.repository.GObject".
Comment 7 Simon Feltman 2013-07-06 16:55:05 UTC
The following fix has been pushed:
075a11a testhelper: Fix import requirement for GObject
Comment 8 Simon Feltman 2013-07-06 16:55:07 UTC
Created attachment 248530 [details] [review]
testhelper: Fix import requirement for GObject

Replace the importing of gi._gobject._gobject with gi.repository.GObject
in tests/testhelpermodule.c

The testhelper module was only importing the static bindings
(gi._gobject._gobject) and not the overrides (gi.repository.GObject).
This was causing some tests to fail when test_thread was the first
test to run in the suite due to it registering new types based on
PyGObject_Type.
Comment 9 Simon Feltman 2013-07-06 17:00:00 UTC
Pushed to 3.8 branch here:
https://git.gnome.org/browse/pygobject/commit/?h=pygobject-3-8&id=075a11ad29f04
Comment 10 Kalev Lember 2013-07-06 18:31:14 UTC
Awesome, thanks for the investigation and the fixes, Simon!