GNOME Bugzilla – Bug 709894
g_test_run () should return 77 (SKIP status) when all its tests were skipped.
Last modified: 2014-01-22 17:43:42 UTC
Currently if you skip all test cases in a test by adding the "/subprocess" string in the name, g_test_run() returns the test as a success. If you choose rather to use g_test_skip() from the test case function itself, and return; the test shows as a failure. So first of all, that is not consistent. And in any case, it should be neither a failure, nor a success. It should be a "skip" return. Automake doc says: « When no test protocol is in use, an exit status of 0 from a test script will denote a success, an exit status of 77 a skipped test, an exit status of 99 an hard error, and any other exit status will denote a failure. » http://www.gnu.org/software/automake/manual/automake.html#Tests It does not answer to: what should be the finale test exit value when some of the test case are skipped and some succeeds? I am not sure, but I guess, it should still be a success. Ideally I would like the output to my `make check` to be: /gimp-tools/crop_tool_can_crop: skipped /gimp-tools/crop_set_width_without_pending_rect: skipped SKIP: test-tools
As someone with an interest in this being resolved (bug 722651) I'd say that when all tests are skipped GTest should return 77, but if there are fails or successes as well as skips then 0 or 99 are returned.
oops, this was fixed as part of bug 720263 (In reply to comment #0) > Currently if you skip all test cases in a test by adding the "/subprocess" > string in the name, g_test_run() returns the test as a success. This is still the case though; the /subprocess tests are not "skipped" in the sense of g_test_skip(). It's more like they're treated like you didn't actually register them. (In reply to comment #1) > As someone with an interest in this being resolved (bug 722651) I'd say that > when all tests are skipped GTest should return 77, but if there are fails or > successes as well as skips then 0 or 99 are returned. 99 is different. That's for when your test fails because of something unrelated to the thing that you're trying to test (eg, if fork() returns -1). gtestutils doesn't currently have any support for that though. The current (git master) status is: - if any test fails, g_test_run() returns 1 (assuming it returns at all, rather than the test program just aborting) - if there is at least 1 test, and all tests call g_test_skip(), it returns 77 - otherwise (if there are 0 tests, or if all tests pass), it returns 0 *** This bug has been marked as a duplicate of bug 720263 ***