GNOME Bugzilla – Bug 775205
Use Automake test suite runner
Last modified: 2016-11-30 06:26:39 UTC
Here are some patches that convert the test suite to use Automake's TAP test harness. This gives clearer output and allows running the test suite in parallel.
Created attachment 340877 [details] [review] build: Use Automake test driver with TAP output For better integration with the build system and nicer output, use Automake's TAP driver and gtester's TAP output mode. There are a few things going on in this commit: - We don't put logs in test_user_data/logs anymore. That really assumed in the first place that all tests would be run serially, and we want them to be run in parallel. So we'll let Automake take care of the stderr and debug outputs and redirect them to a per-test log file. - We use the significant-to-Automake variable TESTS instead of TEST_PROGS. TESTS contains whatever is in check_PROGRAMS plus whatever else we put into it (our test script.) - TESTS_ENVIRONMENT is also significant to Automake, but should not be set within Automake, instead the AM_TESTS_ENVIRONMENT variable should be used so that the developer can override TESTS_ENVIRONMENT from the command line. This variable consists of a sequence of commands and needs to end with a semicolon. (So we can stuff XVFB_START in there too.) - We need to put the test invocation in a separate script since gtester doesn't pass the --tap option on to its test binary.
Created attachment 340878 [details] [review] build: Convert testCommandLine.sh to output TAP It's nice to be able to see in the test output what exactly is failing.
Created attachment 340879 [details] [review] build: Remove privlibdir in favour of pkglibdir Elsewhere in the makefiles this is referred to as pkglibdir, so it would have been broken anyhow if they were not the same.
Created attachment 340880 [details] [review] build: Reorganize testing makefiles These were a bit hard to navigate before, it was difficult to determine where to look if you wanted to write a test for new functionality. Now everything devoted to making "make check" work is in Makefile-test.am. The file is delineated into sections and comments are added. All the rules for converting some of the tests into installed tests, and installing them in the right place, are in Makefile-insttest.am now.
Review of attachment 340877 [details] [review]: Nice, looks good.
Review of attachment 340878 [details] [review]: OK
Review of attachment 340879 [details] [review]: OK
Review of attachment 340880 [details] [review]: I trust that this is just moving rules around -- nothing obviously wrong jumps out but I did not look in detail.
Attachment 340877 [details] pushed as 8feb2e4 - build: Use Automake test driver with TAP output Attachment 340878 [details] pushed as 5be2bb7 - build: Convert testCommandLine.sh to output TAP Attachment 340879 [details] pushed as 8cda645 - build: Remove privlibdir in favour of pkglibdir Attachment 340880 [details] pushed as 838a7f3 - build: Reorganize testing makefiles
(In reply to Cosimo Cecchi from comment #9) > Review of attachment 340880 [details] [review] [review]: > > I trust that this is just moving rules around -- nothing obviously wrong > jumps out but I did not look in detail. Yes, just moving around and adding a few comments.