GNOME Bugzilla – Bug 637387
Add support for Reverse HTTP
Last modified: 2018-09-21 16:06:59 UTC
Which could be used to implement AirPlay support in GNOME: http://arstechnica.com/apple/news/2010/12/airplayer-hacks-your-mac-to-playback-airplay-video.ars
"Reverse HTTP" protocol description: http://wiki.secondlife.com/wiki/Reverse_HTTP Basically we would need a way to steal the socket from a SoupServer and hand it to a SoupSession to use. We probably need the "steal socket" API to implement the server side of WebSockets too. Basically, anything involving an Upgrade or CONNECT.
[mass-moving all "UNCONFIRMED" libsoup bugs to "NEW" after disabling the "UNCONFIRMED" status for this product now that bugzilla.gnome.org allows that. bugspam-libsoup-20150210]
Created attachment 305385 [details] [review] WIP: Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer). Not that this currently doesn't work, because we can't make SoupServer not listen... The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
Created attachment 305503 [details] [review] WIP: Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer). The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
This works! The server after setup, received an "event" from the set-top box. I need to add reverse HTTP specific errors. Would adding server also be required? It would certainly make creating a test case easier.
By the way, the problems with the lack of socket reuse was that, if the SoupMessage, after being sent, wasn't unref'ed, its socket wasn't put back in the pool for re-use. I'm not sure whether that's something to document, expected, or a plain bug. This won't use the same connection: msg = soup_message_new (...); soup_session_send_message (session, msg); msg = soup_message_new (...); soup_session_send_message (session, msg); This will: msg = soup_message_new (...); soup_session_send_message (session, msg); g_object_unref (msg); msg = soup_message_new (...); soup_session_send_message (session, msg); g_object_unref (msg);
Not quite working, the reverse HTTP connection is closed, apparently by us, after receiving the first event: recvfrom(5, "POST /event HTTP/1.1\r\nContent-Type: text/x-apple-plist+xml\r\nContent-Length: 283\r\nX-Apple-Session-ID: 1bd6ceeb-fffd-456c-a09c-996053a7a080\r\n\r\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"http://www.apple.com/DTDs/PropertyList-1.0.dtd\">\n<plist version=\"1.0\">\n <dict>\n <key>category</key>\n <string>video</string>\n <key>state</key>\n <string>loading</string>\n </dict>\n</plist>\n", 8192, 0, NULL, NULL) = 424 sendto(5, "HTTP/1.1 200 OK\r\nDate: Wed, 17 Jun 2015 23:01:23 GMT\r\nContent-Length: 0\r\n\r\n", 75, MSG_NOSIGNAL, NULL, 0) = 75 close(5) = 0
Created attachment 305555 [details] [review] WIP: Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer). The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
Patch in comment 8 fixes the problem from comment 7. I've done: if (completion == SOUP_MESSAGE_IO_COMPLETE && soup_socket_is_connected (sock) && soup_message_is_keepalive (msg) && priv->listeners) { start_request (server, client); + } else if (priv->is_reverse_http) { + /* Don't close reverse HTTP connections */ + start_request (server, client); } else { soup_socket_disconnect (client->sock); soup_client_context_unref (client); But maybe something like: if (completion == SOUP_MESSAGE_IO_COMPLETE && soup_socket_is_connected (sock) && soup_message_is_keepalive (msg) && - priv->listeners) { + (priv->listeners || priv->is_reverse_http)) { start_request (server, client); Is better.
Created attachment 305575 [details] [review] soup-version.h: add 2.52 macros
Created attachment 305576 [details] [review] WIP: Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer). The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
Created attachment 305625 [details] [review] WIP: Add reverse-HTTP server support This allows an HTTP server (SoupServer) to become an HTTP client (SoupSession). This doesn't currently work, as I'd need a way to inject a SoupConnection into a SoupSession, and make sure it's kept alive.
Comment on attachment 305575 [details] [review] soup-version.h: add 2.52 macros need additions to libsoup/soup-version.c and docs/reference/libsoup-2.4-sections.txt too
Created attachment 305746 [details] [review] soup-version.h: add 2.52 macros
Comment on attachment 305576 [details] [review] WIP: Add reverse-HTTP client support >+++ b/libsoup/soup-message-server-io.c >@@ -120,6 +120,14 @@ parse_request_headers (SoupMessage *msg, char *headers, guint headers_len, > soup_uri_set_host (uri, soup_address_get_physical (addr)); > soup_uri_set_port (uri, soup_address_get_port (addr)); > soup_uri_set_path (uri, req_path); >+ } else if (g_object_get_data (G_OBJECT (msg), "is-reverse-http")) { >+ SoupAddress *addr = soup_socket_get_local_address (sock); This should probably just be merged with the HTTP/1.0 case (and in particular needs to handle both "http" and "https" URIs). OTOH, the I-D (http://tools.ietf.org/html/draft-lentczner-rhttp-00) suggests that "the Host field in request messages from B SHOULD contain an empty Host field", in which case we wouldn't end up in this clause anyway (because req_host would be "" rather than NULL). Also, you should make "is-reverse-http" be a real SoupMessage property rather than GObject data whose name is used in multiple different files. >+ * SoupReverseHttpError: "SoupReverseHTTPError" for consistency with other libsoup types >+ g_signal_handlers_disconnect_by_func (msg, G_CALLBACK (reverse_http_connect_async_stop), task); Ugh. I see that you just copied this from the websockets code, but it's wrong there too; disconnect_by_func() doesn't work with signal handlers attached via soup_message_add_status_code_handler(). >+ SOUP_REVERSE_HTTP_ERROR_BAD_HANDSHAKE, >+ _("Server ignored reverse-HTTP handshake")); If the server ignored the handshake then isn't it SOUP_REVERSE_HTTP_ERROR_NOT_REVERSE_HTTP ? (in both cases) >+ server = soup_server_new (NULL, NULL); >+ soup_server_set_is_reverse_http (server); I think I'd just add a private SoupServer *soup_server_new_reverse_http (GIOStream *stream, GSocketAddress *local_addr, GSocketAddress *remote_addr, GError **error); instead. >+ socket = g_object_get_data (G_OBJECT (stream), "GSocket"); You should add a comment to soup_session_steal_connection() where this gets set, pointing out that reverse_http_connect_async_stop() is expecting it. (Previously it wasn't setting it so that other people could use it; it was only doing it to ensure that the GSocket didn't get disposed and close the underlying fd until the caller was done with the stolen connection.) >+ * Asynchronously creates a #SoupServer to communicate >+ * with a remote server. Hm... I realize now that the websockets documentation which you copied from is a bit sparse... you should clarify that the method and path to use in @msg depend on the specific application you're connecting to. Also, the link to #SoupWebsocketConnection from the SoupSession websockets calls eventually leads to a link to more documentation about websockets, but there's no corresponding link to more documentation about reverse HTTP here... >+ * Since: 2.50 2.52 (both functions) >+ reverse-http-test \ If you don't end up making this a standalone test, then move it to examples/ instead.
The inter-diff, to make reviewing easier: http://paste.fedoraproject.org/235347/97293714/
Created attachment 305819 [details] [review] WIP: Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer). The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
Comment on attachment 305819 [details] [review] WIP: Add reverse-HTTP client support >+/** >+ * SOUP_REVERSE_HTTP_ERROR: move this stuff down to just before the reverse-http functions >+ * Since: 2.52 Sigh. 2.54 everywhere. Sorry. >- * Since: 2.50 >+ * Since: 2.52 > */ > SoupWebsocketConnection * > soup_session_websocket_connect_finish (SoupSession *session, This change seems to be spurious? >+ item = g_task_get_task_data (task); >+ if (item && item->protocol_switching_handler_id > 0) { >+ g_signal_handler_disconnect (msg, item->protocol_switching_handler_id); >+ item->protocol_switching_handler_id = 0; >+ } You don't need to do all that. Just do: g_signal_handlers_disconnect_matched (msg, G_SIGNAL_MATCH_DATA, 0, 0, NULL, NULL, task); (which is what the websocket code does now). Don't forget to remove protocol_switching_handler_id from soup-message-queue.h. >diff --git a/tests/reverse-http-test.c b/tests/reverse-http-test.c as mentioned before, please move this to examples, because it's not an automated test. >+#define APPLETV_IP "192.168.0.42" >+#define APPLETV_PORT "7000" >+#define SESSION_ID "1bd6ceeb-fffd-456c-a09c-996053a7a080" >+ >+#define TEST_URL "http://192.168.0.33:12345/foo.mp4" >+#define CONTENT_LOCATION_PARAM "Content-Location: " TEST_URL "\nStart-Position: 0.0\n" some of that should really be command-line arguments...
(In reply to Dan Winship from comment #18) > Comment on attachment 305819 [details] [review] [review] > WIP: Add reverse-HTTP client support > > >+/** > >+ * SOUP_REVERSE_HTTP_ERROR: > > move this stuff down to just before the reverse-http functions Done. > >+ * Since: 2.52 > > Sigh. 2.54 everywhere. Sorry. Done. > >- * Since: 2.50 > >+ * Since: 2.52 > > */ > > SoupWebsocketConnection * > > soup_session_websocket_connect_finish (SoupSession *session, > > This change seems to be spurious? Yes, reverted it, thanks. > >+ item = g_task_get_task_data (task); > >+ if (item && item->protocol_switching_handler_id > 0) { > >+ g_signal_handler_disconnect (msg, item->protocol_switching_handler_id); > >+ item->protocol_switching_handler_id = 0; > >+ } > > You don't need to do all that. Just do: > > g_signal_handlers_disconnect_matched (msg, G_SIGNAL_MATCH_DATA, > 0, 0, NULL, NULL, task); Done. > (which is what the websocket code does now). Don't forget to remove > protocol_switching_handler_id from soup-message-queue.h. Done too. > >diff --git a/tests/reverse-http-test.c b/tests/reverse-http-test.c > > as mentioned before, please move this to examples, because it's not an > automated test. > > >+#define APPLETV_IP "192.168.0.42" > >+#define APPLETV_PORT "7000" > >+#define SESSION_ID "1bd6ceeb-fffd-456c-a09c-996053a7a080" > >+ > >+#define TEST_URL "http://192.168.0.33:12345/foo.mp4" > >+#define CONTENT_LOCATION_PARAM "Content-Location: " TEST_URL "\nStart-Position: 0.0\n" > > some of that should really be command-line arguments... Moved. I'll try to make this a bit easier to use.
(In reply to Bastien Nocera from comment #19) > Done. you didn't attach the updated patch...
(In reply to Dan Winship from comment #20) > (In reply to Bastien Nocera from comment #19) > > Done. > > you didn't attach the updated patch... Because I didn't have time to update the test application :( Do you want the rest of the patch without the test app, and I swear I'll get to making the test app in a mergeable state?
At this point I'll say "I want the test app", but maybe I'll get less picky as we get closer to 3.20 code freeze :)
Created attachment 314931 [details] [review] Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer). The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
Created attachment 314932 [details] [review] WIP: reverse HTTP example
(In reply to Dan Winship from comment #22) > At this point I'll say "I want the test app", but maybe I'll get less picky > as we get closer to 3.20 code freeze :) Bearing in mind that the test app only works with an Apple TV, or something compatible, so not great. Ideally, you'd know how to solve where I'm stuck in the server support, I could finish implementing that, and I'd write a 2 test apps that can talk to each other. Or there's another implementation of the server side support I could target in the client test app.
Created attachment 314933 [details] [review] Add reverse-HTTP client support This allows an HTTP client (SoupSession) to become an HTTP server (SoupServer).
Created attachment 314934 [details] [review] WIP: reverse HTTP example The test application tries to connect to an AirPlay-compatible set-top box: http://nto.github.io/AirPlay.html#servicediscovery-airplayservice
Fixed the patches' commit messages, sorry about that.
Dan, do you have any pointers regarding implementing the reverse-HTTP server support?
(In reply to Bastien Nocera from comment #29) > Dan, do you have any pointers regarding implementing the reverse-HTTP server > support? Do you have any hints as to how I could implement this? I'd really like to land this at some point. Otherwise I'll make an open-coded server, and use this in the tests, so the client support can land.
Claudio, if you have any pointers, so I can finish this...
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/libsoup/issues/35.