GNOME Bugzilla – Bug 581342
SSLv3 fallback for broken servers
Last modified: 2011-09-05 14:02:35 UTC
I cannot do my online banking anymore with ephy-webkit, because the domain is inaccessible... Firefox and gecko-based ephy works fine. And I assume it is not really your fault, but it would be easier to fix or work around from your side :)
This looks like a libsoup bug to me: kov@abacate ~/s/l/tests> ./get https://www.swedbank.ee/ /index.html: 7 Connection terminated unexpectedly kov@abacate ~/s/l/tests> pwd ~/src/libsoup/tests
crazy... the bank seems to be rejecting the connection during the initial SSL handshake before we even get to doing HTTP... it appears to be happening inside gnutls: danw@desktop:tests (master)> gnutls-cli -V -p 443 www.swedbank.ee Resolving 'www.swedbank.ee'... Connecting to '193.203.196.77:443'... *** Fatal error: A TLS packet with unexpected length was received. *** Handshake has failed GNUTLS ERROR: A TLS packet with unexpected length was received.
OK, apparently the server freaks out if we claim to support TLS1.1. Eg, this works: gnutls-cli --priority 'NORMAL:!VERS-TLS1.1' -p 443 www.swedbank.ee (Firefox/ephy-gecko have no problem because NSS doesn't support TLS1.1 yet. Oops.) Apparently this may be a little bit gnutls's fault, because to work around another set of server bugs, it's ignoring one of the "backward compatibility" suggestions of the TLS1.0 spec... Pidgin ran into this problem a few months ago, see: http://developer.pidgin.im/ticket/3456#comment:20 and following. I've committed a patch based on what they did (trying to use the new gnutls flag for the more backward-compatible behavior first, falling back to disabling TLS1.1 if your gnutls isn't new enough to support that). Hopefully this won't cause other old TLS servers to fail... if so, we may have to revert.
The patch for this broke tests/ssl-test on machines with the latest gnutls. See https://savannah.gnu.org/support/?106756. So I'm reverting part of the patch for now; we'll just disable TLS1.1 support unconditionally.
this problem affect evolution and groupwise/soap support over ssl too. adding myself as CC
*** Bug 584479 has been marked as a duplicate of this bug. ***
gnutls-cli-debug reports: www.swedbank.se: Checking for TLS 1.1 support... no Checking fallback from TLS 1.1 to... failed Checking for TLS 1.0 support... yes Checking for SSL 3.0 support... yes www.is.ut.ee: Checking for TLS 1.1 support... no Checking fallback from TLS 1.1 to... failed Checking for TLS 1.0 support... no Checking for SSL 3.0 support... yes www.paypal.com: Checking for TLS 1.1 support... no Checking fallback from TLS 1.1 to... failed Checking for TLS 1.0 support... no Checking for SSL 3.0 support... yes Note that swedbank, the initial bug report, supports TLS 1.0, just not 1.1, but www.is.ut.ee (bug 584479) and paypal (discussed on IRC) don't even support TLS 1.0. We need to fall all the way back to SSL 3.0. Compare, eg: bugs.freedesktop.org: Checking for TLS 1.1 support... no Checking fallback from TLS 1.1 to... TLS 1.0 Checking for TLS 1.0 support... yes Checking for SSL 3.0 support... yes (which shows that bug 584645 is not this, because while fdo doesn't support TLS 1.1, it *does* fall back to 1.0 correctly...) For 2.26.3, I'm going to just disable TLS1.0 and TLS1.1 support. Apparently what other web browsers do is they first try to connect with the latest and greatest versions, and then if that fails, they fall back to older versions/no extensions/etc. So I'm going to do this for trunk (with a cache to remember which hosts are lame so we don't have to connect, fail, and reconnect every time). But this requires rewriting the handshaking code a bunch, and it doesn't really make sense to do this now since the SSL code will all have to be rewritten again for the GSocket port. So this will happen later. For now, I've committed the gnome-2-26 change to master as well. Let me know if this breaks other sites...
*** Bug 585148 has been marked as a duplicate of this bug. ***
Minor note, gnutls_priority_set_direct() was introduced in gnutls 2.1.7. How about making that version as the least required version for gnutls? I used 1.6.2 and got caught by this.
fixed configure.in to depend on gnutls>=2.1.7
*** Bug 527633 has been marked as a duplicate of this bug. ***
Thanks. Fixed in master using Nikos's suggestion on the mailing list. The patch will apply fine to earlier versions of libsoup as well.
oops, git-bz fail; i mentioned this bug in the commit for 622857 and ended up closing this one accidentally too. Anyway, it was pointed out on the gnutls list that "NORMAL:%COMPAT" makes PayPal work ("%COMPAT" makes gnutls not use randomized padding bytes.) It also works for www.swedbank.se, but not www.is.ut.ee, so for now I've stuck with the existing solution of just disabling the TLS versions.
Is it really a good idea to *disable* TLS v1.{0,1,2} altogether? On libsoup 2.30.2 it seems we still default to SSL 3.0 which isn't exactly recent and doesn't allow the use of SHA2 hash algorithms (while gnutls does support them). Not exactly sure the state of fallback code at the moment, but preventing users to access servers supporting TLS 1.2 with it might not be a good idea and might not help get the servers fixed./J
And another idea from my side -- if we're supposed to disable new versions one after another like that, wouldn't it be better to provide an inclusive list of supported algos instead of excluding new ones?
*** Bug 641081 has been marked as a duplicate of this bug. ***
Please note that #641081 was about the opposite case - a server that supports only TLS 1.0, but not SSLv3. I suppose it's fallout from the "fix" for 2.26.3+.
correct. once this is fixed, we'll request SSLv3, TLS 1.0, TLS 1.1, or TLS 1.2 on the initial handshake (which will succeed for your server) and only fall back to SSLv3-only if that handshake fails.
Sounds good; thanks! I'll just wait for the fix then. :)
I am really confused by this issue and it hampers webkit pretty good. Why is the string not simply set to "NORMAL" until it can be fixed properly? or make it an API call so that it can be over-ridden. Currently there is no solution for websites like https://cdx.epa.gov/ As a stop-gap I'd like to suggest the following: --- soup-gnutls.c.orig Thu Mar 17 13:32:18 2011 +++ soup-gnutls.c Thu Mar 17 15:58:14 2011 @@ -477,7 +477,7 @@ soup_ssl_wrap_iochannel (GIOChannel *sock, gboolean no goto THROW_CREATE_ERROR; /* See http://bugzilla.gnome.org/show_bug.cgi?id=581342 */ - if (gnutls_priority_set_direct (session, "NORMAL:!VERS-TLS1.2:!VERS-TLS1.1:!VERS-TLS1.0", NULL) != 0) + if (gnutls_priority_set_direct (session, "NORMAL", NULL) != 0) goto THROW_CREATE_ERROR; if (gnutls_credentials_set (session, GNUTLS_CRD_CERTIFICATE,
Note that paypal now supports TLS 1.0, as well as www.swedbank.se. www.is.ut.ee still doesn't seem to. Anyway, disabling TLS supports seems like a large drawback to me and not a good way to make those website upgrade to a correct TLS version. Disabling TLS especially means no extension support, and TLS are the only protocols which are defined in IETF RFCs. Some websites start to switch to TLS-only (https://meego.com for example), which won't work with libsoup-based browser. Regards,
does product need to be changed to glib-networking now, or is it a matter of how soup is using it?
glib-networking defaults to supporting all available TLS versions. libsoup passes it a flag to make it only use SSL3.
(In reply to comment #18) > correct. once this is fixed, we'll request SSLv3, TLS 1.0, TLS 1.1, or TLS 1.2 > on the initial handshake (which will succeed for your server) and only fall > back to SSLv3-only if that handshake fails. Could you tell me is there any progress regarding this job? BR, Davy
Paypal looks like it has been fixed (it will accept TLS1.0 connections), but it is not, and it will send corrupt packets once the connection is established. This means even a proper fix will not work for this one, since the TLS1.0 handshake looks like it succeeds.
And the paypal issue should be ported directly to them. Warning gnutls people might help too, since it happens even with gnutls-cli (but not with openssl s_client)
(In reply to comment #26) > And the paypal issue should be ported directly to them. Warning gnutls people > might help too, since it happens even with gnutls-cli (but not with openssl > s_client) I think they know: http://lists.gnu.org/archive/html/gnutls-devel/2011-02/msg00000.html
Fixed in git (also requires latest git glib-networking to fix a crash)
*** Bug 641080 has been marked as a duplicate of this bug. ***
Kudos!
This fix now breaks access to https://launchpad.net/ subdomains. To reproduce, use Epiphany 3.0 and try accessing https://launchpad.net/ and then clicking through "Browse bugs" (takes you to https://bugs.launchpad.net/), "Help translate" (https://translations.launchpad.net) and "Find answers" (https://answers.launchpad.net) in whatever order you want. Third subdomain is going to cause a 404 error, and any other subdomain after that as well (eg. code.launchpad.net). I believe this is due to this bug because debian package since 2.34.2 started exhibiting the same problem and it set use-ssl3 explicitely to false (compared to upstream, which worked fine), and that's basically now changed as well. As a Launchpad developer, I'll ask around about what's wrong with our set-up, but I can imagine there being other sites like this that would start breaking with this change now. Also, should I file a new bug, since this is a new bug resulting from the way this bug was fixed?
(In reply to comment #31) > I believe this is due to this bug because debian package since 2.34.2 started > exhibiting the same problem and it set use-ssl3 explicitely to false (compared > to upstream, which worked fine), and that's basically now changed as well. The Debian 2.34.2 package doesn’t have yet the proper fix that has been committed to git. It intentionnally fails with broken TLS sites, but it was before Dan’s changes.
Right, but the launchpad sites support TLS, so the fallback code wouldn't be getting hit, so for those sites it's essentially the same as Debian's version. Danilo pointed out on IRC that the launchpad subdomains are all on the same server. Thus, it's likely that the bug is a result of the fact that now that we're using TLS 1.0, we can use the Server Name Indication extension, and we're probably messing something up with cached connections or something.
Danilo: I can't reproduce this. Can you attach the output of running epiphany with WEBKIT_DEBUG=Network ? Also, are you using a proxy or anything?
I am not behind a proxy that I know of, but from the tests I just did, it seems likely I am behind a broken transparent proxy or something. WEBKIT_DEBUG=Network was useful in finding actual requests going over the wire, and I tried using 'openssl s_client -connect launchpad.net:443' to reproduce the problem with identical requests, which only happened when I sent them in sequence in the same HTTP session (even much simpler requests with just "GET / HTTP/1.1\nHost: answers.launchpad.net\n\n" in sequence failed on the third request [HTTP/1.1 assumes Connection: keep-alive, so that's not needed either]). I even confirmed it doesn't happen from my home network (different ISP), and when I come home I'll play with my laptop with recent epiphany/libsoup on that network as well. So, most likely not a libsoup problem, though the change did expose what seems to be a network problem on my side. Not sure how I am going to explain this to the ISP (the worst is that all the other browsers work just fine, just like epiphany did before this patch), but I apologize for wasting your time.
Hum, from further testing, I could get this to happen from other networks as well, but not as frequently, and I get exactly the same behaviour when I try "gnutls-cli launchpad.net" (which checks the certificate) and try to fetch a different subdomain, when I get 400 Bad Request. I am not even sure what to think anymore, but it seems as if epiphany/libsoup is reusing the same connection for different https subdomains, which it probably shouldn't. Maybe it works to an extent because all of the *.launchpad.net subdomains are served from two distinct IPs, but if they were all forced to be on a single IP, it'd fail quicker and it'd be easier to reproduce.
Created attachment 193527 [details] WEBKIT_DEBUG=Network log of epiphany run
(In reply to comment #36) > but it seems as if epiphany/libsoup is reusing the same > connection for different https subdomains, which it probably shouldn't. Yeah, that was my initial theory, but looking at the code, it shouldn't do that (it caches connections by hostname, not IP), and it's not doing that in your debug output anyway: each of the "400 Bad Request" responses is in response to a message that was sent on a never-previously-used connection. (Each of the last 3 requests has a unique SoupSocket number in the Soup-Debug header.) weird
The launchpad problem turned out to be due to a glib-networking issue, and it's fixed in master.