GNOME Bugzilla – Bug 684418
bad error reporting on failed CONNECT
Last modified: 2018-09-21 16:11:25 UTC
Using the Soup GNOME features, when a HTTPS proxy is set to something that does not match the HTTP proxy, all HTTPS connections result in a SoupMessage.response_body.data being null. To reproduce: 1. "apt-get install polipo" (or find a proxy supporting both HTTP and HTTPS) 2. In gnome-control-center, set both HTTP and HTTPS proxy to localhost:8123 (or your proxy address) 3. Fetch https://duckduckgo.com/?q=foo using Soup w/ the GNOME feature type (Ephy 3.5.4 works here) 4. Observe the page load succeeds 5. Change the _HTTP_ proxy to be something other than the HTTPS proxy 6. Fetch https://duckduckgo.com/?q=bar 7. Observe the page load fails (in Ephy, results in a download of an empty file) Note that GProxy is returning the correct proxy address in both cases, so I assume Soup's GNOME proxy resolver is at issue here.
Created attachment 224800 [details] Test program
Created attachment 224801 [details] Prints GNOME proxy settings
The attached programs can be used to verify this without needing Ephy. An example session: > # set HTTP/HTTP proxy to be the same in gnome-control-centre > mjg@montbard:~/tmp$ ./gproxy-resolver.py http://duckduckgo.com/?q=hello > http://localhost:8123 > mjg@montbard:~/tmp$ ./gproxy-resolver.py https://duckduckgo.com/?q=hello > http://localhost:8123 > mjg@montbard:~/tmp$ ./soup-get.py https://duckduckgo.com/?q=hello > [snip HTML content] > # change only the HTTP proxy hostname > mjg@montbard:~/tmp$ ./gproxy-resolver.py http://duckduckgo.com/?q=hello > http://blarg:8123 > mjg@montbard:~/tmp$ ./gproxy-resolver.py https://duckduckgo.com/?q=hello > http://localhost:8123 > mjg@montbard:~/tmp$ ./soup-get.py https://duckduckgo.com/?q=hello > None > mjg@montbard:~/tmp$
This is clearly the same bug as bug 679728, but I still can't reproduce it: danw@laptop:/tmp> /tmp/gproxy-resolver.py http://duckduckgo.com/?q=hello http://no-such-host:8888 danw@laptop:/tmp> /tmp/gproxy-resolver.py https://duckduckgo.com/?q=hello http://192.168.0.1:8888 danw@laptop:/tmp> /tmp/soup-get.py http://duckduckgo.com/?q=hello None danw@laptop:/tmp> /tmp/soup-get.py https://duckduckgo.com/?q=hello [HTML] What versions of libsoup and glib-networking do you have installed? Also, what is message.status_code and message.reason_phrase after the failed GET? Also, can you try stracing the failed soup-get.py?
Created attachment 224890 [details] Updated test program Updated test program also prints response status/message/headers.
Created attachment 224891 [details] strace output from updated test program fetching HTTP URL via proxy
Created attachment 224892 [details] strace output from updated test program fetching HTTPS URL via proxy
Created attachment 224893 [details] strace output from updated test program fetching HTTPS URL direct (not via proxy)
libsoup 2.39.92 libsoup-gnome 2.39.91 glib-networking 2.33.14 Attached update test program, as well as strace output from a few different runs. Response for both HTTP and for HTTP direct (i.e. not via a proxy) is 200 OK: > mjg@montbard:~/tmp$ ./gproxy-resolver.py http://duckduckgo.com/?q=hello > http://www-proxy.cse.unsw.edu.au:3128 > mjg@montbard:~/tmp$ ./soup-get.py http://duckduckgo.com/?q=hello > 200 OK > Server: nginx > Date: Fri, 21 Sep 2012 03:06:08 GMT > Content-Type: text/html; charset=UTF-8 > Expires: Fri, 21 Sep 2012 03:06:09 GMT > Cache-Control: max-age=1 > Content-Length: 18798 > Connection: close > Age: 0 > > <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/T... The response for HTTPS via the proxy is 302 Found: > mjg@montbard:~/tmp$ ./gproxy-resolver.py https://duckduckgo.com/?q=hello > http://localhost:8123 > mjg@montbard:~/tmp$ ./soup-get.py https://duckduckgo.com/?q=hello > 302 Found > > [response body is null] However this seems bogus since the message's response headers are empty. I would have expected at least a Location header if not the Nginx advertisement, also the HTTP request does not receive a 302 response initially - it gets the 200 OK immediately (as verified by Wireshark). The HTTPS direct request seems to be the same.
(In reply to comment #9) > The response for HTTPS via the proxy is 302 Found: ... > However this seems bogus since the message's response headers are empty. I > would have expected at least a Location header if not the Nginx advertisement, > also the HTTP request does not receive a 302 response initially The 302 is presumably coming from the proxy itself, in response to the CONNECT request. libsoup then copies the status code to the GET request, but not the body or any of the headers. Presumably this is the same environment as bug 684519, and your proxy server is configured to not speak https? Until 684519 is fixed, the simplest workaround is probably to install a proxy on localhost, and use that as your https proxy. (I've used tinyproxy for testing in the past; it works pretty much right out of the box without needing lots of configuration.)
It's the combination of bug 684519 and this that causes the problem: I can't leave the HTTPS proxy blank because of that bug - I have already been using polipo (similar to tinyproxy but supports HTTP 1.1 and pipelining support) running on localhost for HTTPS to work-around it previously, but can't do that any more because of this bug. So I need either 684519 or this to get fixed so I can use HTTPS in Epiphany 3.6 and elsewhere at the office. I'm not sure that the proxy is giving me a 302 - polipo supports HTTPS connections (From its FAQ: “Polipo is an HTTP proxy that can tunnel HTTPS”) and from the traces above, I don't think the web site is sending it either. I'm on a 64-bit distro, could this be a pointer-as-int bug or something equally subtle? Or related to Polipo's HTTP 1.1 support? I'll try disabling pipelining and/or try tinyproxy when I am on-premises tomorrow.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/libsoup/issues/44.