GNOME Bugzilla – Bug 615274
Encode proxy user/password in proxy URL
Last modified: 2010-09-29 08:14:21 UTC
When I try to use a webcal calendar I get an error dialog: Error on webcal://ical.mac.com/whitematter/Canadian Holidays.ics Cannot resolve proxy hostname But there is no reason that a proxy name should not be resolvable. I can resolve my proxy name at a shell prompt: $ host proxy proxy.example.com is an alias for linux.example.com. linux.example.com has address 192.168.0.3 linux.example.com has IPv6 address 2001:1234:123:1:123:1234:1234:1234
Do you use SSL on the ical.mac.com? Maybe it's a similar issue as yours bug #611539?
Well, I have the patch from bug 611539 applied. Now when I try to add webcal://ical.mac.com/whitematter/Canadian Holidays.ics I get Error on webcal://ical.mac.com/whitematter/Canadian Holidays.ics Proxy Authentication Required In a dialog and indeed, the request was attempted, the 407 returned and no further attempt made to use the proxy credentials that evolution was told about: 1283524447.003 1 <my_ip> TCP_DENIED/407 3708 GET http://ical.mac.com/whitematter/Canadian%20Holidays.ics - NONE/- text/html 1283524447.018 0 <my_ip> TCP_DENIED/407 3708 GET http://ical.mac.com/whitematter/Canadian%20Holidays.ics - NONE/- text/html That "-" before the "NONE" is the proxy user. As you can see, the second request was made without the proxy credentials being sent.
Well, when I try the url from my machine in the web browser then I get "Not Found" page error. I'm not using a proxy, and I guess the url wasn't supposed to be restricted, was it? Maybe it's just a bad error code returned? Also, SSL doesn't work on this url either, so the answer for my question is "no", as indicates your log entry too.
(In reply to comment #3) > Well, when I try the url from my machine in the web browser then I get "Not > Found" page error. Indeed, but that's a red herring in regard to this problem. Evolution isn't even getting to the point of trying to fetch the URL (and finding it invalid) because the proxy is refusing to honor the request because it is unauthenticated. The issue here is that Evolution is not providing the configured proxy credentials when requesting the URL from the proxy server, so the proxy server isn't even trying to fetch it yet. Put another way, this same problem would exist with a fully functioning URL. For example, if I change the URL to webcal://ical.mac.com/ical/Canadian32Holidays.ics (which has calendar info and is not a 404) it's still the same problem: 1283945801.523 1 <my_ip> TCP_DENIED/407 3674 GET http://ical.mac.com/ical/Canadian32Holidays.ics - NONE/- text/html 1283945801.619 0 <my_ip> TCP_DENIED/407 3674 GET http://ical.mac.com/ical/Canadian32Holidays.ics - NONE/- text/html > I'm not using a proxy, and I guess the url wasn't supposed > to be restricted, was it? Maybe it's just a bad error code returned? All orthogonal. > Also, SSL doesn't work on this url either, so the answer for my question is > "no", as indicates your log entry too. Ah, yes, indeed. No SSL being used in this case.
Created attachment 169785 [details] debug eds patch for evolution-data-server; Could you try to apply this eds debug patch and run e-calendar-factory on its own console, ans see what it'll print there? It should print some useful information, like what proxy URL was passed into Soup session and so on. It should shed some light on this issue finally. Thanks in advance for testing this. (By the way, this is for a CalDAV calendar backend, like the google one.)
(In reply to comment #5) > Created an attachment (id=169785) [details] > debug eds patch > > for evolution-data-server; Maybe you could save me some time here and tell me which libraries in particular that this patch ends up changing so that I can simply replace the existing ones with the ones patched by this patch and not have to go through a whole build cycle. I did just patch and make but it seems to have remade all of the libraries instead of just the 1 or 2 that are actually affected by this patch.
OK. So I copied the newly built ./calendar/libecal/.libs/libecal-1.2.so.7.2.2 and ./libedataserver/.libs/libedataserver-1.2.so.13.0.1 to /usr/lib. I seem to be getting the ep_set_proxy() debug and it's reporting: http://userid:password@proxy:proxy_port https://userid:password@proxy:proxy_port Where userid, password and proxy_port are in fact the username, password and port number on the proxy and are correct. I am still noticing that the "Proxy-Authorization: Basic" header has a value of: gmail_userid:gmailpassword rather than the proxy userid/password.
Thanks for the update and testing. Dan, is it possible that the libsoup is misplacing the Proxy-Authentication header with the one used for google calendar authentication? It seems to me, because, as stated in the previous comment, the userid and password for the proxy in the proxy URL set to the SoupSession by > g_object_set (priv->session, SOUP_SESSION_PROXY_URI, proxy_uri, NULL); is correct. Maybe the CalDAV and other backends does this incorrectly? (this ~~ passing proxy URI with proxy credentials)
Hm... this probably counts as bad API on libsoup's part, but the same "authenticate" signal is emitted for both regular and proxy auth, so if you're not checking soup_auth_is_for_proxy() in your authenticate callbacks, you might end up trying to authenticate it with the server username/password rather than the proxy username/password. actually, that definitely counts as bad API now that we have automatic behind-the-scenes proxy resolution...
Created attachment 170634 [details] [review] proposed libsoup patch for libsoup; Please give a try to this patch, whether it works. Thanks in advance. Thanks Dan. Do you think this patch can be acceptable for libsoup? I would do pretty the same change in all backends using libsoup in evolution-data-server and maybe others, but it would be quite much code duplication. I believe the code is backward compatible, except of the 'retrying' parameter when authenticating against proxy, because the first try can be done with credentials from the proxy uri.
Comment on attachment 170634 [details] [review] proposed libsoup patch >+ soup_auth_authenticate (auth, proxy_uri->user, proxy_uri->password); If you are actually passing the username and password in via the proxy URI, then this should already be happening, in soup-auth-manager.c:authenticate_auth(). So something else is going wrong somewhere.
Ouch, I didn't look there, my fault. While looking around, as a wild guess, would it be possible that the soup-auth-manager.c:request_started does the authentication in the reverse order, even without user interaction, that it tries to authenticate against remote server first, and then against the proxy? I would expect just the opposite order, talk to the nearest server first.
Err, no, it cannot know it requires proxy until it tries to connect to the destination server.
Created attachment 170739 [details] [review] proposed eds patch for evolution-data-server; I just tried with libsoup 2.31.92 and with a plain password consisting only of letters from a-zA-Z and it works as expected. When I have some special letters in the proxy password then it fails, thus this patch, to be able to write special passwords into the proxy uri. Could you retest with this patch and the mentioned libsoup version, please? Thanks in advance. (I used --without-gnome in libsoup configuration.)
(In reply to comment #14) > > Could you retest with this patch and the mentioned libsoup version, please? To be clear, no patches for libsoup, right? And is 2.31.92 of libsoup necessary? Have there been changes to the way proxy is handled since 2.30.2 that are meaningful to our use-case. And further, this eds patch is only necessary for special characters in in the proxy password? > Thanks in advance. (I used --without-gnome in libsoup configuration.) I'm just rebuilding the way Ubuntu built libsoup 2.30.2, so we'll see how that goes. :-)
(In reply to comment #15) > (In reply to comment #14) > > > > Could you retest with this patch and the mentioned libsoup version, please? > > To be clear, no patches for libsoup, right? Yup, that's correct. > And is 2.31.92 of libsoup > necessary? Have there been changes to the way proxy is handled since 2.30.2 > that are meaningful to our use-case. To be honest I do not know, I just used the latest "to-be-stable-version-soon" and it works there just fine. I can download my google calendar through caldav accessed through an authenticated proxy without any problem. > And further, this eds patch is only necessary for special characters in in the > proxy password? Yup, that's correct. > > Thanks in advance. (I used --without-gnome in libsoup configuration.) > > I'm just rebuilding the way Ubuntu built libsoup 2.30.2, so we'll see how that > goes. :-) That's how I compiled it. The option disables proxy integration for gnome, if I got it right, and thus can have a little influence on the behaviour we are trying to debug. But I can be wrong, Dan knows libsoup the best here.
If you're setting proxy URIs manually, then building --without-gnome won't hurt anything. (Although really someday evolution should probably port to using SoupProxyResolverGNOME rather than rolling its own.)
(In reply to comment #14) > Created an attachment (id=170739) [details] [review] > proposed eds patch I didn't apply this patch as I don't have special characters in the password I am using. > I just tried with libsoup 2.31.92 After upgrading to this version of libsoup, indeed, yes, it's all working now. Yay!
(In reply to comment #17) > If you're setting proxy URIs manually, then building --without-gnome won't hurt > anything. (Although really someday evolution should probably port to using > SoupProxyResolverGNOME rather than rolling its own.) There is some plan for it, though there are users complaining that they are using different proxy settings for evolution and for the rest of the system. (In reply to comment #18) > > I just tried with libsoup 2.31.92 > > After upgrading to this version of libsoup, indeed, yes, it's all working now. > Yay! Good. I'll commit the patch to evolution later anyway.
Perhaps this is a different bug, but when I use ctrl-I (or View->Load Images) to see the images in an e-mail, a request is sent to the proxy server with my username but no password is included.
(In reply to comment #19) > (In reply to comment #17) > > (Although really someday evolution should probably port to using > > SoupProxyResolverGNOME rather than rolling its own.) > > There is some plan for it, though there are users complaining that they are > using different proxy settings for evolution and for the rest of the system. Any pointers to bugs/discussion about that? If it's stuff that could be handled via the "Ignored Hosts" mechanism, then we should figure out how to make that more understandable / easier to configure. If it can't be handled via "Ignored Hosts" then we should figure out if the proxy capplet needs more configurability for these cases. (Eg, presumably it's not really "different proxy settings for evolution", but actually "different proxy settings for this protocol+host+port combination".)
(In reply to comment #21) > Any pointers to bugs/discussion about that? If it's stuff that could be handled > via the "Ignored Hosts" mechanism, then we should figure out how to make that > more understandable / easier to configure. If it can't be handled via "Ignored > Hosts" then we should figure out if the proxy capplet needs more > configurability for these cases. (Eg, presumably it's not really "different > proxy settings for evolution", but actually "different proxy settings for this > protocol+host+port combination".) Ah, I see. I do not have any pointers, unfortunately, it's quite some time ago I heard about it, and I think it was just on IRC from one user. I do not recall exactly, I'm sorry.
ah. well, i'm sure we'll get bug reports if/when you change it. :-)
(In reply to comment #17) > If you're setting proxy URIs manually, then building --without-gnome won't hurt > anything. (Although really someday evolution should probably port to using > SoupProxyResolverGNOME rather than rolling its own.) Does the lack of using SoupProxyResolverGNOME mean that evolution won't benefit from all of the libproxy work that's been happening in libsoup? Is all of that wonderful libproxy stuff in 2.31.92? I guess kerberos/negotiate proxy support isn't in libsoup though is it? I dream of a day where nothing on my desktop needs a password and I don't have to have an exception list the length of two of my arms in my proxy server for all of the (gnome) apps that don't know how to deal with authenticated proxies.
The recent proxy work was in glib, not libsoup, so depends on using GSocketConnection, etc. Also, there's no support (in libsoup or gio) for Negotiate auth (although there's been a little work on that, bug 587145). The future is still a little ways off.
(In reply to comment #25) > The recent proxy work was in glib, not libsoup, so depends on using > GSocketConnection, etc. Ahhh. Right. Is libsoup leveraging that work (yet)? > Also, there's no support (in libsoup or gio) for > Negotiate auth (although there's been a little work on that, bug 587145). Yeah. I'm on that bug already. > The > future is still a little ways off. ~sigh~ Indeed.
Created commit 4166cec in eds master (2.33.1+) Created commit de16bec in eds gnome-2-32 (2.32.1+)