After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 653386 - Test framework does not support skipped tests?
Test framework does not support skipped tests?
Status: RESOLVED FIXED
Product: glib
Classification: Platform
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: gtkdev
gtkdev
Depends on:
Blocks:
 
 
Reported: 2011-06-25 03:00 UTC by Behdad Esfahbod
Modified: 2013-08-18 00:52 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Behdad Esfahbod 2011-06-25 03:00:11 UTC
I can't seem to find any support in GTester for skipped tests.  In my case, in harfbuzz, user has to have certain fonts to run certain tests.  If the font is not available, I don't want to pass the test, and I don't want to fail...
Comment 1 Matthias Clasen 2011-06-27 14:15:18 UTC
you can just do something like

if (!font_exits)
  return;

in your test function, maybe ?
Comment 2 Behdad Esfahbod 2011-06-27 17:31:03 UTC
That would pass the test.  Running a test suite, getting "100 out of 100 passed" while in reality none of the tests were run is less than ideal to say the least.
Comment 3 Emmanuele Bassi (:ebassi) 2011-06-27 20:06:46 UTC
in Clutter's conformance test suite I have this macro:

#define TEST_CONFORM_SKIP(CONDITION, NAMESPACE, FUNC)   G_STMT_START {  \
  if (CONDITION) {                                                      \
    g_test_add ("/skipped" NAMESPACE "/" #FUNC,                         \
                TestConformSimpleFixture,                               \
                shared_state, /* data argument for test */              \
                test_conform_simple_fixture_setup,                      \
                test_conform_skip_test,                                 \
                test_conform_simple_fixture_teardown);                  \
  } else { TEST_CONFORM_SIMPLE (NAMESPACE, FUNC); }     } G_STMT_END

http://git.gnome.org/browse/clutter/tree/tests/conform/test-conform-main.c#n63

with test_conform_skip_test() being just an empty function. this guarantees that the final report still mentions the skipped test units, under the /skipped path.

it could be good to have the same simple macro in GTest.
Comment 4 Behdad Esfahbod 2011-06-28 15:25:40 UTC
Thanks Emmanuele, moving them into /skipped/ is useful.
Comment 5 Emmanuele Bassi (:ebassi) 2011-06-28 18:07:41 UTC
from an API standpoint, it might be interesting to have "regions" in the test suite; something like:

  /* creates a "skip" region in the test suite; all the test functions
   * added inside this region will be executed only if @condition
   * evaluates to %TRUE; if it evaluates to %FALSE, the tests will not
   * be executed, and the will be appended under @prefix.
   */
  void
  g_test_push_skip_region (gboolean    condition,
                           const char *prefix);

  /* creates a "todo" region in the test suite; all test functions
   * added inside this region will be expected to fail . if they do
   * not fail, they will be appended under @prefix. in both cases,
   * they will not stop the execution of the test suite.
   */
  void
  g_test_push_todo_region (const char *prefix);

  /* terminates the innermost region */
  void
  g_test_pop_region (void);
Comment 6 Behdad Esfahbod 2011-06-28 18:56:30 UTC
I rather we just add g_test_skip().
Comment 7 Matthias Clasen 2011-06-30 17:56:20 UTC
I would happily accept a patch to add g_test_skip
Comment 8 Emmanuele Bassi (:ebassi) 2011-07-01 09:45:27 UTC
cool, I'll have a go at it today, then.
Comment 9 Matthias Clasen 2013-08-18 00:52:25 UTC
I just added g_test_skip and g_test_incomplete