GNOME Bugzilla – Bug 136292
no Referer: header sent for context menu HTTP downloads
Last modified: 2010-04-16 19:00:45 UTC
Steps to reproduce: 0) Right click a link and choose "Download" or "Save Link As". Expected results: The HTTP headers sent to the server includes the Referer: header Actual results: No Referer: header sent. Consequently, if the server checks for referer, download will fail.
See firefox bug http://bugzilla.mozilla.org/show_bug.cgi?id=236194.
We have a similar bug already filed. Gtkmozembed load_url doesnt support referrer, maybe we should use mozilla api directly, maybe we should fix gtkmozembed (I filed a bug the prob is backward compatibility).
This is not the same problem as the missing referrer on page loads in new tabs. Here we use mozilla API directly which supports referrer, but we pass nsnull instead (3rd param to nsIWebBrowserPersist::SaveURI), in MozDownload.cpp:InitiateMozillaDownload, and in mozilla-embed-persist.cpp:impl_save.
Target: 1.4 -> 1.6
Mass reassigning of Epiphany bugs to epiphany-maint@b.g.o
Target Milestone: 1.6 -> 1.8
See also https://bugzilla.mozilla.org/show_bug.cgi?id=284868 the same bug on firefox.
Mass changing target 2.16 -> 2.18
This is still there with 2.19.5, using the test page on mozilla bug. Also, Firefox 2.0.0.6 does not have this problem, so this is not a gecko bug. I got the no referrer in any case except when i simply click the link. In any other case (open new tab, new page, download) i got no referrer. Any hint on how to fix this?
Updating milestone and version fields.
The fix needs to be make in gtkmozembed.
I hadn't used epiphany in a while, but I thought I'd check if this bug is still present now that epiphany is using WebKit. I can confirm that epiphany 2.28.1 still has the bug (right-click, download linked file, HTTP GET request has no Referer). This is different to Konqueror, which is also based on WebKit.
OK, updating version. For the record, Konqueror uses KHTML which is not quite the same as WebKit.
Confirmed. Still hapenning in Epiphany WebKit 2.30.2 with libsoup 2.31.0 and WebKit 1.1.90. Will take a look on it.
(In reply to comment #14) > Confirmed. Still hapenning in Epiphany WebKit 2.30.2 with libsoup 2.31.0 and > WebKit 1.1.90. Actually, the User-Agent field is also missing, and the Accept field is different: Normal click: ======== > GET /berto/files/fuse-emulator/libspectrum7_0.5.0.1-1_amd64.deb HTTP/1.1 > Soup-Debug-Timestamp: 1271243446 > Soup-Debug: SoupSessionAsync 1 (0x8b13410), SoupMessage 5 (0x8b1f690), SoupSocket 1 (0x8e76820) > Host: people.igalia.com > User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-us) AppleWebKit/531.2+ (KHTML, like Gecko) Safari/531.2+ Epiphany/2.30.2 > Accept: application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5 > Referer: http://people.igalia.com/berto/ > Accept-Encoding: gzip > Accept-Language: en, es, it Right-click => Download Link: =================== > GET /berto/files/fuse-emulator/libspectrum7_0.5.0.1-1_amd64.deb HTTP/1.1 > Soup-Debug-Timestamp: 1271243454 > Soup-Debug: SoupSessionAsync 1 (0x8b13410), SoupMessage 6 (0x8ee6e88), SoupSocket 1 (0x8e76820) > Host: people.igalia.com > Accept: */* > Accept-Encoding: gzip > Accept-Language: en, es, it
Looks like the problem is that the download requests are handled in different ways: In the first case WebKit is "who" takes care of the whole process of building the network request, the sou message and so on... so it bundles the "Referer" fielf in the request and then notifies Epiphany through the "mime-type-policy-decision-requested" signal emitted by the the WebView. In the second case, Epiphany is "who" originates the request with almost nothing else but the URI and the information already set in the SoupSession (such as Accept-Language). My guessing is that we'd need to either add the missing fields directly from epiphany in the SoupMessage before actually making the request, or to find a way to delegate the whole process through WebKit (don't know whether this is possible, though) to let it do all the hard work for us and in a more consistent way.
Created attachment 158727 [details] [review] Send 'Referer' on headers sent for context menu HTTP downloads (In reply to comment #16) > [...] > My guessing is that we'd need to either add the missing fields directly from > epiphany in the SoupMessage before actually making the request, [...] I've implemented this behaviour in the attached patch. From git log: "Make sure the EphyEmbedPersist object is created specifying the EphyEmbed object, and create the network request inside of it using the URL from the associated web view as 'Referer'." It was a bit strange to me to have to create the WebKitNetworkRequest through g_object_new () to be able to set a SoupMessage explicitly. Perhaps it would be better to provide a webkit_network_request_new_from_msg () in WebKitGtk+ for this, unless there's a good reason not to do it that I'd be missing right now (not very unlikely :-)).
Review of attachment 158727 [details] [review]: Did you check that other users of embed_persist are also setting the embed? I think there are two or three in all ephy code. OTOH, I guess there's no easy way to share cookies or that stuff, perhaps some sites depend on that or something. Wouldn't swear it. Patch looks good to me.
(In reply to comment #18) > Review of attachment 158727 [details] [review]: > > Did you check that other users of embed_persist are also setting the embed? I > think there are two or three in all ephy code. It's only used from these functions in src/popup-commands.c: - save_property_url () - popup_cmd_set_image_as_background () - save_temp_source () I've just set the embed in the first case cause I'm not sure whether it makes sense or not in the other two. Anyway, I've added checks in EphyEmbedPersist to only do the new stuff when an embed was already set, so in those two cases the code keeps working as it was doing it so far. > OTOH, I guess there's no easy way to share cookies or that stuff, perhaps some > sites depend on that or something. Wouldn't swear it. No idea either. > Patch looks good to me. Thanks, let's get it in then :-)
Comment on attachment 158727 [details] [review] Send 'Referer' on headers sent for context menu HTTP downloads So, we'll still be missing some things that WebKit would do in a normal request (in FrameLoader::addExtraFieldsToRequest), but I don't think we expose this in any way at the moment, so this is good for now IMHO.
(In reply to comment #18) > OTOH, I guess there's no easy way to share cookies or that stuff, perhaps some > sites depend on that or something. Wouldn't swear it. > The cookies are set by libsoup at the network/HTTP layer, so that shouldn't be an issue.
(In reply to comment #21) > (In reply to comment #18) > > OTOH, I guess there's no easy way to share cookies or that stuff, perhaps some > > sites depend on that or something. Wouldn't swear it. > > > > The cookies are set by libsoup at the network/HTTP layer, so that shouldn't be > an issue. How is this related to this bug? (Just in case you were talking about doing any other change I'm not getting)
(In reply to comment #22) > How is this related to this bug? (Just in case you were talking about doing any > other change > I'm not getting) Just meant that AFAIK we don't need to consider cookies here at all, since libsoup takes care of that automatically.
(In reply to comment #23) > (In reply to comment #22) > > How is this related to this bug? (Just in case you were talking about doing any > > other change > > I'm not getting) > > Just meant that AFAIK we don't need to consider cookies here at all, since > libsoup takes care of that automatically. Ah, Ok. Then, feel free to push the patch, or just ask me if you want me to change anything and I'll gladly do it :-)
Pushed as b30cf3d