After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 694487 - 2.35.6 reorders tests passed to g_test_add_func
2.35.6 reorders tests passed to g_test_add_func
Status: RESOLVED NOTABUG
Product: glib
Classification: Platform
Component: general
2.35.x
Other Linux
: Normal normal
: ---
Assigned To: gtkdev
gtkdev
Depends on:
Blocks:
 
 
Reported: 2013-02-22 22:55 UTC by Cole Robinson
Modified: 2013-03-18 02:22 UTC
See Also:
GNOME target: ---
GNOME version: ---


Description Cole Robinson 2013-02-22 22:55:10 UTC 
Previously, test cases were invoked in the same order they were passed to g_test_add_func. However that changes after this commit:

commit 89aa9dbd9871d6f6cffc25ae8cb02e3b764a25b6
Author: Matthias Clasen <mclasen@redhat.com>
Date:   Sun Jan 20 03:14:24 2013 -0500

    Fix g_test_add_vtable
    
    This function was creating a test suite for each added
    testcase, when it should have grouped tests according to
    their paths.

Sounds like an intentional change, but at least this broke tests in the qemu unit test suite, where a destructive test was specifically specified last to not interfere with other tests.

Maybe qemu is doing it wrong, if so please indicate proper usage. Thanks.
Comment 1 Matthias Clasen 2013-02-24 20:59:56 UTC 
Sorry that this causes problems for you. But I do indeed think that if qemu is having destructive tests that are interfering with the rest of the testsuite, it is doing it wrong. To keep this working, it should be enough to make sure that the destructive test is in a suite of its own:

/tests/test1
/tests/test2
...

/destructive-tests/this-comes-last

should work
Comment 2 Allison Karlitskaya (desrt) 2013-02-25 13:46:40 UTC 
fwiw, "destructive tests" is not the only case here where reordering is causing people trouble.  I had the opposite problem: I had a test that needed to run in a clean environment before global state of GLib had been modified by other testcases.

Also: Tristian was having trouble with this change because order of test failures mattered to him: you want to find out about lower-level failures before they cause higher-level tests to fail as a side-effect (and in a way that's harder to debug).

All that said, the change is kinda sane...
Comment 3 Matthias Clasen 2013-03-18 02:22:27 UTC 
I think I'll declare this notabug. If your tests depend on ordering, your testsuite has a bug. If your tests require stronger isolation that we can provide, put them in separate binaries.