GNOME Bugzilla – Bug 789171
Regression on tests
Last modified: 2017-10-21 19:58:27 UTC
Recently there has been a change[0], which makes the `installed-tests` to fail once they are installed. The output is as follows: Running test: glib-networking/connection.test # random seed: R02Sd32181099be5042880b36d998b42a050 1..30 # Start of tls tests # Start of connection tests ok 1 /tls/connection/basic ok 2 /tls/connection/verified ok 3 /tls/connection/verified-chain ok 4 /tls/connection/verified-chain-with-redundant-root-cert ok 5 /tls/connection/verified-chain-with-duplicate-server-cert ok 6 /tls/connection/verified-unordered-chain ok 7 /tls/connection/verified-chain-with-alternative-ca-cert ok 8 /tls/connection/invalid-chain-with-alternative-ca-cert ** GLib-Net:ERROR:connection.c:975:test_client_auth_connection: assertion failed (error == NULL): Failed to open file ?/tmp/gn/libexec/installed-tests/glib-networking/files/client2-and-key.pem?: No such file or directory (g-file-error-quark, 4) # GLib-Net:ERROR:connection.c:975:test_client_auth_connection: assertion failed (error == NULL): Failed to open file ?/tmp/gn/libexec/installed-tests/glib-networking/files/client2-and-key.pem?: No such file or directory (g-file-error-quark, 4) FAIL: glib-networking/connection.test (Child process killed by signal 6) Although the solution looks easy just by copying `client2-and-key.pem` file, it produces another error. The test is successful if it's run in the source code tree. [0] https://git.gnome.org/browse/glib-networking/commit/?id=8da92fd64d3c321f8591bea6a9516ac10f8abcb7
(In reply to Iñigo Martínez from comment #0) > Although the solution looks easy just by copying `client2-and-key.pem` file, > it produces another error. Thanks for the heads-up. There's just a couple more files to copy.
(In reply to Michael Catanzaro from comment #1) > (In reply to Iñigo Martínez from comment #0) > > Although the solution looks easy just by copying `client2-and-key.pem` file, > > it produces another error. > > Thanks for the heads-up. There's just a couple more files to copy. The issue isn't solved. Actually this is my fault as I've seen the following error before, when I tried to solve the issue, but I haven't included the output here. This is the output now: # random seed: R02S1b91957700f93dd1bb7dd1f4e5a1ae2b 1..30 # Start of tls tests # Start of connection tests ok 1 /tls/connection/basic ok 2 /tls/connection/verified ok 3 /tls/connection/verified-chain ok 4 /tls/connection/verified-chain-with-redundant-root-cert ok 5 /tls/connection/verified-chain-with-duplicate-server-cert ok 6 /tls/connection/verified-unordered-chain ok 7 /tls/connection/verified-chain-with-alternative-ca-cert ok 8 /tls/connection/invalid-chain-with-alternative-ca-cert ** GLib-Net:ERROR:connection.c:989:test_client_auth_connection: assertion failed (g_tls_certificate_is_same (peer, cert)) # GLib-Net:ERROR:connection.c:989:test_client_auth_connection: assertion failed: (g_tls_certificate_is_same (peer, cert)) FAIL: glib-networking/connection.test (Child process killed by signal 6) Running test: glib-networking/pkcs11-pin.test Hope this is more useful now.
How exactly are you running the tests?
The command was exactly as follows: gnome-desktop-testing-runner -d /tmp/gn/share glib-networking The prefix used to install `glib-networking` was `/tmp/gn`. However, thanks to Emmanuele Bassi, I've fixed the meson build port[0], so they can be executed as `unit tests` with the following command: ninja -C builddir test I have been looking at the source code, but I'm not very familiar with it yet and I still don't have a conclusive answer, so I hope any of these work. [0] https://bugzilla.gnome.org/attachment.cgi?id=361904
I didn't know about gnome-desktop-testing-runner (which is amazingly not anywhere in our modulesets). But the test passes for me even when running with gnome-desktop-testing-runner. At this point, I have no clue why it's failing for you. But I'll guess: perhaps, after all the fun you've had with the meson porting, you've somehow wound up with an old version of glib-networking installed in your prefix, but also the latest version of the tests. Is there any chance at all this is possible? (Probably not. It's not a very good guess.)
Once you figured out what was causing the problem, the tests not using the GIO modules just built, I have tried a quick and dirty workaround: 14 envs = [ 15 'G_TEST_SRCDIR=' + meson.current_source_dir(), 16 'G_TEST_BUILDDIR=' + meson.current_build_dir(), 17 'GIO_MODULE_DIR=' + join_paths(meson.build_root(), 'tls', 'gnutls') 18 ] Once `GIO_MODULE_DIR` is added as an environment variable, pointing to the directory where `libgiognutls.so` is, fixes the problem, at least when executing the tests by using meson`s `unit tests` feature: ninja -C _build test I've been very busy for the last couple of days, but I can try to fix it properly as soon as I've some time.
That actually looks like the proper fix to me. The only way to get it to not use the system GIO modules is with the GIO_MODULE_DIR environment variable. Now, if you install into a custom build prefix and try to run the installed tests there, like you were doing when you reported this bug, then it's probably not going to ever work. But what I discovered today is that simple 'make check' or 'ninja test' style tests have been improperly using the system module instead of the one in the source directory, which is bad, and we can fix that.
The following fix has been pushed: 04d4bf3 Ensure tests run against the GIO modules in the build root
Created attachment 362024 [details] [review] Ensure tests run against the GIO modules in the build root Currently, our tests run against the GIO modules that are installed on the system. So if we 'ninja test' before running 'ninja install', we'll be testing the wrong code. And this is actually the default case when running 'jhbuild buildone -n --check', so that's bad. This was actually broken in the Autotools build, too. We just never noticed before. I discovered this after testing a build without TLS support: after reenabling TLS support and rebuilding glib-networking, but before installing it, all the TLS tests failed.
(In reply to Michael Catanzaro from comment #7) > That actually looks like the proper fix to me. The only way to get it to not > use the system GIO modules is with the GIO_MODULE_DIR environment variable. > > Now, if you install into a custom build prefix and try to run the installed > tests there, like you were doing when you reported this bug, then it's > probably not going to ever work. That's probably actually a good thing, though. After all, the point of installed tests is to test whatever you actually have installed. So you can actually use the glib-networking tests to test glib-openssl if you want. (I wonder how well that would go....)