This is libsoup's realm.

Nope, it's epiphany. This is basically a bookkeeping issue; you need to keep track of what hosts are HSTS, and only make https requests to them (eg, when the user types "bugzilla.gnome.org" in the address bar, rewrite it to "https://bugzilla.gnome.org/", not "http://bugzilla.gnome.org/").

This is an important fix for epiphany's security support.  Please address it :)

Hey Dan (not Daniel and not David), I was thinking libsoup might be a good place to implement HSTS, but I see in comment #2 from four years ago that you'd prefer soup users to handle this themselves.  Is this still your preference? If so, I'll work on adding it at the WebKit level instead.  (Currently only the Mac port has HSTS, and it's provided by Core Services.)

Well, the issue is that you need to keep a database of information about previously-visited sites, and libsoup mostly doesn't do that. Though, libsoup does unconditionally depend on sqlite3 now, so I guess it could...

Propose an API and let's see if it makes sense...

My first thoughts, before trying to implement anything (or even having used soup before, so this might be terrible):

How about a new SoupSessionFeature, named SoupHSTSEnforcer or something similar, that when added to a SoupSession via the existing APIs will automatically detect and process any HSTS response headers it sees, adding hosts to an sqlite database if the header is valid according to RFC #6797 (comment #0). It probably makes sense for the HSTS database to be shared by all applications using soup. So the first function we'd need is soup_hsts_enforcer_new().

It'd probably be easiest for soup to automatically translate HTTP requests into HTTPS requests for sites in the database, but the application would definitely need to know the transformed URI in order to display it to the user (e.g. in a browser's address bar). An app could do that by setting up a notify on SoupMessage's uri property.

Any certificate validation failure for a domain in the HSTS database would need to be treated as an unrecoverable transport-level failure (in accordance with section 12.1 No User Recourse), so a SoupMessage would need to return SOUP_STATUS_SSL_FAILED regardless of the value of SoupSession's ssl-strict property. Except that if the application sets ssl-strict to FALSE, then it will need to be able to distinguish a validation failure  from an HSTS-enabled domain so that it knows not to allow the user to load the page anyway... maybe the best way to handle that would be to just introduce a new error code, SOUP_STATUS_HSTS_FAILED, though that doesn't seem like a very good name.

I think that's all we'd really need. Am I missing anything obvious?

We might also want to implement section 12.2 User-Declared HSTS Policy, and section 12.3 HSTS Pre-Loaded List, which are optional. Then applications would need a way to add hosts to the list themselves, query the set of hosts they have added, and also remove any hosts that they added, but I do not think they would not need to see all hosts in the database or remove arbitrary hosts, and I don't think an application's changes ought to be effective for other applications. User-declared HSTS policy would not be appropriate for Epiphany, so I would only propose API for this if you request it for libsoup, but it would be nice for Epiphany to use the same HSTS pre-loaded list as Chromium [1] so we might need such functions anyway. An alternative would be to distribute that list with libsoup, and then applications would not need to configure the HSTS database at all, besides the simple on/off of attaching the new HSTS SoupSessionFeature. Maybe better to put the list in a new package, and allow applications to give libsoup the filename of the list with a function like void soup_hsts_enforcer_add_preloaded_list(const char *).

Thoughts on this? I think we do HSTS at the WebKit level too, I just thought soup would make sense since the most important work is parsing the HTTP response header.

[1] http://src.chromium.org/viewvc/chrome/trunk/src/net/http/transport_security_state_static.json

FYI Mozilla maintains a list for firefox too
https://dxr.mozilla.org/comm-central/source/mozilla/security/manager/ssl/nsSTSPreloadList.inc

FYI: Thinking about this more, I think WebKit is really the best place for this. I can't think of a use-case for this in any app that's not using WebKit; such apps surely want to pin a couple certs instead.

(In reply to Michael Catanzaro from comment #8)
> FYI: Thinking about this more, I think WebKit is really the best place for
> this. I can't think of a use-case for this in any app that's not using
> WebKit; such apps surely want to pin a couple certs instead.

fwiw, I think libsoup is a better place for this functionality than WebKit.  HSTS is an HTTPS/TLS feature, not an HTML or rendering/layout feature.

There are lots of tools that fetch from arbitrary URLs that could make use of HSTS pinning but have no use for WebKit's layout/rendering work.  For example, revision control clients that fetch data over https, download managers, etc.

(In reply to Daniel Kahn Gillmor from comment #9)
> For
> example, revision control clients that fetch data over https, download
> managers, etc.

OK, I couldn't think of such examples, but you're right. Failure of imagination.

(In reply to Daniel Kahn Gillmor from comment #9)
> fwiw, I think libsoup is a better place for this functionality than WebKit. 
> HSTS is an HTTPS/TLS feature, not an HTML or rendering/layout feature.

You just might have prevented us from making a big mistake and developing this at the wrong layer. It's almost complete in bug #767160, and the implementation is in libsoup. :)

*** This bug has been marked as a duplicate of bug 767160 ***