After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 724198 - should use wget --no-check-certificate
should use wget --no-check-certificate
Status: RESOLVED WONTFIX
Product: jhbuild
Classification: Infrastructure
Component: general
unspecified
Other All
: Normal normal
: ---
Assigned To: Jhbuild maintainers
Jhbuild QA
Depends on:
Blocks:
 
 
Reported: 2014-02-12 00:06 UTC by Allison Karlitskaya (desrt)
Modified: 2016-11-22 04:04 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
tarball: don't check certificates with wget (1.41 KB, patch)
2014-02-26 01:41 UTC, Allison Karlitskaya (desrt)
reviewed Details | Review
tarball checkout: Don't check for certificate validity on remote server. (1.42 KB, patch)
2015-11-05 12:00 UTC, Carlos Alberto Lopez Perez
rejected Details | Review
Certificate Errors (7.23 KB, text/plain)
2016-05-22 15:45 UTC, John Ralls
  Details
wget result (6.93 KB, text/plain)
2016-05-22 16:50 UTC, Ting-Wei Lan
  Details

Description Allison Karlitskaya (desrt) 2014-02-12 00:06:45 UTC
It's possible when downloading from https:// mirrors we run into one of two issues:

  - server has a self-signed cert

  - we don't have a certificate list locally

In either case, it's a bit silly to care about https when we do most of our fetching over unencrypted http and use checksums on the result.

wget has a --no-check-certificate option.  We should use it.
Comment 1 John Ralls 2014-02-12 03:38:24 UTC
+1.
Comment 2 Allison Karlitskaya (desrt) 2014-02-26 01:41:32 UTC
Created attachment 270342 [details] [review]
tarball: don't check certificates with wget

Pass the --no-check-certificate argument to wget to prevent it from
checking certificates when fetching tarballs from https.

Checking certificates is problematic when faced with the prospects of
self-signing servers or missing CA lists on the local machine.

It's also redundant, considering most of what we download comes over
plain http and the fact that we always verify the checksum of the
downloaded files when we're done.
Comment 3 John Ralls 2014-02-26 03:19:04 UTC
Comment on attachment 270342 [details] [review]
tarball: don't check certificates with wget

+1!
Comment 4 Carlos Alberto Lopez Perez 2015-11-05 12:00:41 UTC
Created attachment 314908 [details] [review]
tarball checkout: Don't check for certificate validity on remote server.

Here is a patch to ignore certificate validity on the remote server when doing a tarball checkout (either using wget or curl).


tarball checkout: Don't check for certificate validity on remote server.
    
     * This causes problems when downloading a file from a server that
       don't has a proper SSL certificate (self-signed, expired, etc.).
    
     * Checking for the validity of the SSL certificate of the remote server
       is completely overkill and unneeded since hash checks (md5/sha256) are
       done later on the downloaded file.
Comment 5 Frederic Peters 2015-11-05 12:25:28 UTC
Comment on attachment 314908 [details] [review]
tarball checkout: Don't check for certificate validity on remote server.

Fine, thanks.
Comment 6 Carlos Alberto Lopez Perez 2016-03-30 14:23:05 UTC
(In reply to Frederic Peters from comment #5)
> Comment on attachment 314908 [details] [review] [review]
> tarball checkout: Don't check for certificate validity on remote server.
> 
> Fine, thanks.

Seems this is still not committed.

How this works?
Comment 7 Ting-Wei Lan 2016-05-22 07:09:34 UTC
Is this issue still valid? Is there a server we use in jhbuild having a broken certificate?
Comment 8 John Ralls 2016-05-22 13:35:38 UTC
Yes. I had certificate errors yesterday from Sourceforge and Kitware.
Comment 9 Ting-Wei Lan 2016-05-22 14:52:02 UTC
(In reply to John Ralls from comment #8)
> Yes. I had certificate errors yesterday from Sourceforge and Kitware.

I cannot find Kitware URL in modulesets, and all HTTPS sourceforge hosts listed in modulesets can be successfully connected. I test them with gnutls-cli, and the following hosts seem to have HTTPS correctly configured:

download.sourceforge.net
downloads.sourceforge.net
inkscape.svn.sourceforge.net
liferea.svn.sourceforge.net
mission-control.svn.sourceforge.net
opalvoip.svn.sourceforge.net
prdownloads.sourceforge.net
sourceforge.net
svn.sourceforge.net
xchat.svn.sourceforge.net

Do you remember the hostname that caused problems?
Comment 10 John Ralls 2016-05-22 15:45:50 UTC
Created attachment 328349 [details]
Certificate Errors

Attached are the errors excerpted from the build session yesterday.

Kitware means cmake.org. The sourceforge error was on a redirect from tukaani.org for xz, so it's possible that the redirect might have something to do with the problem. I had forgotten the two errors from freedesktop.org.

The modulesets raising the errors are https://git.gnome.org/browse/gtk-osx/tree/modulesets-stable/bootstrap.modules and https://git.gnome.org/browse/gtk-osx/tree/modulesets-stable/gtk-osx-bootstrap.modules. wget is 1.10.2, quite old but what Apple distributes because of their allergy to GPLv3.
Comment 11 Ting-Wei Lan 2016-05-22 16:50:44 UTC
Created attachment 328353 [details]
wget result

I tried wget commands in your attachment with wget 1.16.3 on FreeBSD, and no certificate problem was found.

I also tried them with wget 1.10.2. Only intltool-0.51.0.tar.gz could be successfully downloaded. Other files could not be downloaded because of certificate error.

For intltool-0.51.0.tar.gz, it seems that your wget cannot find the list of trusted CA on your system. Does manually specify the list of CA fix the problem?

For other files, I think it simply means wget 1.10.2 is too old and it doesn't support certificates used by some sites.

As you already use bootstrap.modules, can wget be added to bootstrap.modules, so users can easily get a working version of wget on Apple systems?
Comment 12 John Ralls 2016-05-22 23:37:44 UTC
(In reply to Ting-Wei Lan from comment #11)
> Created attachment 328353 [details]
> wget result
>  
> For intltool-0.51.0.tar.gz, it seems that your wget cannot find the list of
> trusted CA on your system. Does manually specify the list of CA fix the
> problem?

That doesn't seem to be feasible, but it doesn't actually matter, see below.

> 
> For other files, I think it simply means wget 1.10.2 is too old and it
> doesn't support certificates used by some sites.
> 
> As you already use bootstrap.modules, can wget be added to
> bootstrap.modules, so users can easily get a working version of wget on
> Apple systems?

Doing that in a moduleset would produce a chicken-and-egg problem, except that it turns out that Apple doesn't provide wget. I must have built that myself some time ago and forgotten about it.

Apple does provide curl, and rerunning the build after moving wget out of the way and deleting everything allows jhbuild to fall back to curl. That works fine, no certificate issues. Sorry for the noise and causing you extra work.
Comment 13 Michael Catanzaro 2016-11-22 04:02:27 UTC
Review of attachment 314908 [details] [review]:

Nope, we want to switch to using HTTPS wherever possible and it's very unexpected for HTTPS to be insecure.
Comment 14 Michael Catanzaro 2016-11-22 04:04:43 UTC
So best I can tell, there is some bug with wget on macOS causing valid certificate chains to be spuriously rejected, which requires investigation and remediation from macOS folks. Disabling certificate verification is not an OK workaround. It's true that currently we use http:// and git:// for everything so verifying HTTPS connections is relatively pointless, but we plan to change this!