Created attachment 185988 [details] [review]
API: gtester: Add g_test_fail()

I like the idea of non-fatal failures, when I think of validating function output with multiple input values where it's useful to compare which of those failed and which didn't.

How does g_test_fail () play with g_assert_strcmp and friends? An example of how this should be used would be appreciated.

As pointed out on IRC, g_assert_strcmp() is one layer above this function. This is pretty much the nonfatal equivalent to abort(), which is the way how the assertions messages indicate fatal failures currently.

It would not be hard to add a nonfatal layer using the g_test_fail() if we wanted to do that. But that'd be a separate bug.

An example test function for this is http://git.gnome.org/browse/gtk+/tree/tests/css/parser/test-css-parser.c?h=http://git.gnome.org/browse/gtk+/commit/?h=parser&id=91d1d29d91ec35a4d9a5bfa0b70141e769a320fe#n165

The function takes a CSS file, parses it and then calls css_provider_to_string() to check that the parsed CSS matches a reference css file. This is a nice way to check that the parser's implementation of the CSS grammer and its error detection and recovery work as expected. Because a broken check is not fatal to or damaging to the integrity of the test-runner, it is fine to just indicate test failure and continue with the next test instead of restarting the whole test-running application.

Also, it improves performance by an order of 10, as we don't have to gtk_init() just to check a 100bytes css file.

Makes sense to me.

After fixing the patch to actually make the test runner fail (oops), I committed it to master.