After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 789171 - Regression on tests
Regression on tests
Status: RESOLVED FIXED
Product: glib
Classification: Platform
Component: network
unspecified
Other Linux
: Normal normal
: ---
Assigned To: gtkdev
gtkdev
Depends on:
Blocks:
 
 
Reported: 2017-10-18 19:32 UTC by Iñigo Martínez
Modified: 2017-10-21 19:58 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Ensure tests run against the GIO modules in the build root (3.33 KB, patch)
2017-10-21 19:56 UTC, Michael Catanzaro
committed Details | Review

Description Iñigo Martínez 2017-10-18 19:32:00 UTC
Recently there has been a change[0], which makes the `installed-tests` to fail once they are installed.

The output is as follows:

  Running test: glib-networking/connection.test
  # random seed: R02Sd32181099be5042880b36d998b42a050
  1..30
  # Start of tls tests
  # Start of connection tests
  ok 1 /tls/connection/basic
  ok 2 /tls/connection/verified
  ok 3 /tls/connection/verified-chain
  ok 4 /tls/connection/verified-chain-with-redundant-root-cert
  ok 5 /tls/connection/verified-chain-with-duplicate-server-cert
  ok 6 /tls/connection/verified-unordered-chain
  ok 7 /tls/connection/verified-chain-with-alternative-ca-cert
  ok 8 /tls/connection/invalid-chain-with-alternative-ca-cert
  **
  GLib-Net:ERROR:connection.c:975:test_client_auth_connection: assertion failed (error == NULL): Failed to open file ?/tmp/gn/libexec/installed-tests/glib-networking/files/client2-and-key.pem?: No such file or directory (g-file-error-quark, 4)
  # GLib-Net:ERROR:connection.c:975:test_client_auth_connection: assertion failed (error == NULL): Failed to open file ?/tmp/gn/libexec/installed-tests/glib-networking/files/client2-and-key.pem?: No such file or directory (g-file-error-quark, 4)
FAIL: glib-networking/connection.test (Child process killed by signal 6)

Although the solution looks easy just by copying `client2-and-key.pem` file, it produces another error.

The test is successful if it's run in the source code tree.

[0] https://git.gnome.org/browse/glib-networking/commit/?id=8da92fd64d3c321f8591bea6a9516ac10f8abcb7
Comment 1 Michael Catanzaro 2017-10-18 21:00:39 UTC
(In reply to Iñigo Martínez from comment #0)
> Although the solution looks easy just by copying `client2-and-key.pem` file,
> it produces another error.

Thanks for the heads-up. There's just a couple more files to copy.
Comment 2 Iñigo Martínez 2017-10-19 06:27:26 UTC
(In reply to Michael Catanzaro from comment #1)
> (In reply to Iñigo Martínez from comment #0)
> > Although the solution looks easy just by copying `client2-and-key.pem` file,
> > it produces another error.
> 
> Thanks for the heads-up. There's just a couple more files to copy.

The issue isn't solved. Actually this is my fault as I've seen the following error before, when I tried to solve the issue, but I haven't included the output here.

This is the output now:

  # random seed: R02S1b91957700f93dd1bb7dd1f4e5a1ae2b
  1..30
  # Start of tls tests
  # Start of connection tests
  ok 1 /tls/connection/basic
  ok 2 /tls/connection/verified
  ok 3 /tls/connection/verified-chain
  ok 4 /tls/connection/verified-chain-with-redundant-root-cert
  ok 5 /tls/connection/verified-chain-with-duplicate-server-cert
  ok 6 /tls/connection/verified-unordered-chain
  ok 7 /tls/connection/verified-chain-with-alternative-ca-cert
  ok 8 /tls/connection/invalid-chain-with-alternative-ca-cert
  **
  GLib-Net:ERROR:connection.c:989:test_client_auth_connection: assertion failed (g_tls_certificate_is_same (peer, cert))
  # GLib-Net:ERROR:connection.c:989:test_client_auth_connection: assertion failed: (g_tls_certificate_is_same (peer, cert))
  FAIL: glib-networking/connection.test (Child process killed by signal 6)
  Running test: glib-networking/pkcs11-pin.test

Hope this is more useful now.
Comment 3 Michael Catanzaro 2017-10-19 14:08:48 UTC
How exactly are you running the tests?
Comment 4 Iñigo Martínez 2017-10-19 21:46:13 UTC
The command was exactly as follows:

  gnome-desktop-testing-runner -d /tmp/gn/share glib-networking

The prefix used to install `glib-networking` was `/tmp/gn`. However, thanks to Emmanuele Bassi, I've fixed the meson build port[0], so they can be executed as `unit tests` with the following command:

  ninja -C builddir test

I have been looking at the source code, but I'm not very familiar with it yet and I still don't have a conclusive answer, so I hope any of these work.

[0] https://bugzilla.gnome.org/attachment.cgi?id=361904
Comment 5 Michael Catanzaro 2017-10-20 01:24:33 UTC
I didn't know about gnome-desktop-testing-runner (which is amazingly not anywhere in our modulesets). But the test passes for me even when running with gnome-desktop-testing-runner.

At this point, I have no clue why it's failing for you. But I'll guess: perhaps, after all the fun you've had with the meson porting, you've somehow wound up with an old version of glib-networking installed in your prefix, but also the latest version of the tests. Is there any chance at all this is possible? (Probably not. It's not a very good guess.)
Comment 6 Iñigo Martínez 2017-10-21 18:22:01 UTC
Once you figured out what was causing the problem, the tests not using the GIO modules just built, I have tried a quick and dirty workaround:

 14 envs = [
 15   'G_TEST_SRCDIR=' + meson.current_source_dir(),
 16   'G_TEST_BUILDDIR=' + meson.current_build_dir(),
 17   'GIO_MODULE_DIR=' + join_paths(meson.build_root(), 'tls', 'gnutls')
 18 ]

Once `GIO_MODULE_DIR` is added as an environment variable, pointing to the directory where `libgiognutls.so` is, fixes the problem, at least when executing the tests by using meson`s `unit tests` feature:

  ninja -C _build test

I've been very busy for the last couple of days, but I can try to fix it properly as soon as I've some time.
Comment 7 Michael Catanzaro 2017-10-21 18:30:39 UTC
That actually looks like the proper fix to me. The only way to get it to not use the system GIO modules is with the GIO_MODULE_DIR environment variable.

Now, if you install into a custom build prefix and try to run the installed tests there, like you were doing when you reported this bug, then it's probably not going to ever work. But what I discovered today is that simple 'make check' or 'ninja test' style tests have been improperly using the system module instead of the one in the source directory, which is bad, and we can fix that.
Comment 8 Michael Catanzaro 2017-10-21 19:56:29 UTC
The following fix has been pushed:
04d4bf3 Ensure tests run against the GIO modules in the build root
Comment 9 Michael Catanzaro 2017-10-21 19:56:38 UTC
Created attachment 362024 [details] [review]
Ensure tests run against the GIO modules in the build root

Currently, our tests run against the GIO modules that are installed on
the system. So if we 'ninja test' before running 'ninja install', we'll
be testing the wrong code. And this is actually the default case when
running 'jhbuild buildone -n --check', so that's bad.

This was actually broken in the Autotools build, too. We just never
noticed before. I discovered this after testing a build without TLS
support: after reenabling TLS support and rebuilding glib-networking,
but before installing it, all the TLS tests failed.
Comment 10 Michael Catanzaro 2017-10-21 19:58:27 UTC
(In reply to Michael Catanzaro from comment #7)
> That actually looks like the proper fix to me. The only way to get it to not
> use the system GIO modules is with the GIO_MODULE_DIR environment variable.
> 
> Now, if you install into a custom build prefix and try to run the installed
> tests there, like you were doing when you reported this bug, then it's
> probably not going to ever work.

That's probably actually a good thing, though. After all, the point of installed tests is to test whatever you actually have installed. So you can actually use the glib-networking tests to test glib-openssl if you want. (I wonder how well that would go....)