After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 627738 - Add WebSocket support
Add WebSocket support
Status: RESOLVED FIXED
Product: libsoup
Classification: Core
Component: API
unspecified
Other Linux
: Normal normal
: ---
Assigned To: libsoup-maint@gnome.bugs
libsoup-maint@gnome.bugs
Depends on:
Blocks:
 
 
Reported: 2010-08-23 15:55 UTC by Eduardo Lima Mitev
Modified: 2015-03-01 15:39 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Eduardo Lima Mitev 2010-08-23 15:55:04 UTC
The WebSocket protocol enables two-way, low-latency communication between a user agent and a remote host. Combined with the HTML5's WebSocket API (http://dev.w3.org/html5/websockets/), it provides an alternative to HTTP polling for bidirectional communication from a Web page to a remote server. 
This technique can be used for a variety of Web applications: games, stock tickers, multiuser applications with simultaneous editing, user interfaces exposing server-side services in real time, etc.

The protocol spec is still a draft. Current latest version is 76 (http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76). There is a Work Group at the Internet Engineering Task Force (IETF) steering the development of the protocol (http://datatracker.ietf.org/wg/hybi/charter/).

Upcoming new version of the spec is expected to differ greatly from current, so perhaps it makes little sense to start implementing the current version. However, discussions can be started now towards defining the high-level API (which should be fairly independent of protocol details), and how to integrate it within libsoup.
Comment 1 Eduardo Lima Mitev 2010-08-23 16:23:45 UTC
Here some questions/thoughts in order to start debate:

* WebSocket streams are based on messages, which (in next versions of the spec) will support fragmentation into frames. 
   - What would be the best API to deliver received messages to user?  GInputStream model don't seem to fit well here.
   - Do we want to provide access to individual frames of a partially received message as it is arriving, or instead buffer all frames until message is loaded completely, then deliver? I think the former will be desired for applications needing to interchange long messages in form of streams, but I would say that for simplicity, by now we can focus on delivering only complete messages.

 * Could a SoupWsServer run standalone, or always associated to a SoupServer? Due to Same Origin Policy (http://en.wikipedia.org/wiki/Same_origin_policy), which applies to WebSocket connections, it is difficult to decouple both, as they have to share the same locally bound address. I presume some sort of service selector will have to be inserted to discriminate the types of stream (WS vs. HTTP/HTTPS/FTP/etc).

One possible solution would be to embed the selector within the SoupServer, associating it a SoupWsServer to redirect the requests identified as WS streams. This way we can reuse the SoupServer's header parsing routine. Something like:

void soup_server_attach_websocket_server (SoupServer *server, SoupWsServer *ws_server);

Upon attachment, all streams identified as WS will be passed to the SoupWsServer, which from there on takes ownership of the connection and complete the request. For simplicity, only one SoupWsServer could be attached to a SoupServer.
Comment 2 Dan Winship 2010-08-23 16:56:02 UTC
(In reply to comment #0)
> The protocol spec is still a draft. Current latest version is 76
> (http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76).

Actually, that's now been superceded by
http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-00
although I'm not sure there are any changes other than the filename/
ownership.

(In reply to comment #1)
> * WebSocket streams are based on messages, which (in next versions of the spec)
> will support fragmentation into frames. 
>    - What would be the best API to deliver received messages to user? 
> GInputStream model don't seem to fit well here.

I/O-wise (ie, ignoring the security issues, etc), WebSockets are a lot like SCTP streams, which we do plan to support via GIOStream like other socket types.

One thing to compare with is accepting a connection on a listening socket, which is also more "event-like" than "I/O-like". In GIO, this is handled in two layers: GSocketListener provides a gio-like async API (where you "proactively" call g_socket_listener_accept_async(), which will eventually trigger its callback when a client connects), and then GSocketService provides a signal-based API on top of that (where it just emits a signal when a connection comes in). So we could do something like that; if you're actively trying to get data off the socket, you can use g_input_stream_read{,_async}, but if you're just waiting for data that you don't know when it's coming, you can wait for a signal instead.

>    - Do we want to provide access to individual frames of a partially received
> message as it is arriving

My understanding was that higher-level APIs were expected to only provide complete messages, and if you need to stream a lot of data, it's up to the apps and servers to break that stream into a series of smaller messages. (But I haven't read the spec in a while, maybe I'm misremembering?)

>  * Could a SoupWsServer run standalone, or always associated to a SoupServer?

Probably both. Or at least, you want to be able to run a WS server that isn't also an HTTP server, since that may be used in some cases.

SoupServer is in need of some API improvements anyway (eg, bug 621138, bug 561547, bug 522519 among others). I am thinking I may do an ABI break for 3.0 if we can get the gio-based client-side stuff in, and so we could break SoupServer ABI at that point too.
Comment 3 Eduardo Lima Mitev 2010-08-23 21:05:26 UTC
(In reply to comment #2)
> (In reply to comment #0)
> > The protocol spec is still a draft. Current latest version is 76
> > (http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-76).
> 
> Actually, that's now been superceded by
> http://tools.ietf.org/html/draft-ietf-hybi-thewebsocketprotocol-00
> although I'm not sure there are any changes other than the filename/
> ownership.
> 

That's right, although it looks like that version is obsolete too. Up-to-date version is at: http://www.whatwg.org/specs/web-socket-protocol/.

> (In reply to comment #1)
> > * WebSocket streams are based on messages, which (in next versions of the spec)
> > will support fragmentation into frames. 
> >    - What would be the best API to deliver received messages to user? 
> > GInputStream model don't seem to fit well here.
> 
> I/O-wise (ie, ignoring the security issues, etc), WebSockets are a lot like
> SCTP streams, which we do plan to support via GIOStream like other socket
> types.
> 

Yes, but I guess in practice it will be a TCP connection what the other end will be expecting, so I don't see how an SCTP connection could fit.

> So we could do something like that; if you're actively trying to get
> data off the socket, you can use g_input_stream_read{,_async}, but if you're
> just waiting for data that you don't know when it's coming, you can wait for a
> signal instead.
> 

The problem with g_input_stream_read{,_async} is that user can provide a buffer and a size shorter than the message buffered for delivery. In that case, common solution is making the operation fail but providing the real size of the message so that user can realloc buffer and call read again. This solution, however, is ugly because g_input_stream_read{,_async} would have to set the GError, yet returning a positive return value as the real size, which is a misuse of the API.

For message reading, I prefer a solution based on a "message-received" signal, together with a "receive_message" method. Something like:

gssize soup_ws_client/server_receive_message (SoupWsClient/Server *self,
                                              void *msg,
                                              GError **error);

which should only be called during the signal handler. Problem with this solution is that user is forced to always process (or cache) the message, so it cannot proactively decide when to receive messages (assuming disconnecting the signal is not an option).

> My understanding was that higher-level APIs were expected to only provide
> complete messages, and if you need to stream a lot of data, it's up to the apps
> and servers to break that stream into a series of smaller messages. (But I
> haven't read the spec in a while, maybe I'm misremembering?)
> 

Some people had commented in spec's mailing list that it could be useful to send very large frames (in the order of Gygabytes) over WebSocket using sendfile(), to avoid the stream to visit the CPU. If we only provide complete message delivery, we will have to be prepared to allocate really long buffers (at risk of exhausting RAM memory). I can't think of a practical solution to this, without limiting the message maximum length to something reasonable, or having a per-fragment delivery API. Of course, for browser specific implementations there is no choice as the JS API only deal with full message delivery, as you stated.

> >  * Could a SoupWsServer run standalone, or always associated to a SoupServer?
> 
> Probably both. Or at least, you want to be able to run a WS server that isn't
> also an HTTP server, since that may be used in some cases.
> 

Agree. Then I guess some common code in SoupServer will have to be refactored outside to be reused by SoupWsServer. I'm not very familiar (yet) with libsoup internals so I leave this to experts.
Comment 4 Dan Winship 2010-08-23 21:40:31 UTC
(In reply to comment #3)
> That's right, although it looks like that version is obsolete too. Up-to-date
> version is at: http://www.whatwg.org/specs/web-socket-protocol/.

although, as I understand it, the plan going forward is for the IETF version to be the master copy

> > I/O-wise (ie, ignoring the security issues, etc), WebSockets are a lot like
> > SCTP streams, which we do plan to support via GIOStream like other socket
> > types.
> 
> Yes, but I guess in practice it will be a TCP connection what the other end
> will be expecting, so I don't see how an SCTP connection could fit.

No, my point was just that gio will eventually want to be able to deal with SCTP GSocketConnections, and that will require solving some of the same issues as we need to solve for WebSockets. Eg:

> The problem with g_input_stream_read{,_async} is that user can provide a buffer
> and a size shorter than the message buffered for delivery.

This is also an issue for SCTP, so we'll need a solution there as well. It's possible that we want it to work one way when using GSocketConnection, but a different way when using GSocket directly...

> For message reading, I prefer a solution based on a "message-received" signal,
> together with a "receive_message" method.

Having it be signal-based is a bit like how streaming in SoupMessage works now (it emits read-chunk as it receives each chunk of the response body), but this has turned out to be problematic for lots of use cases and lots of people would rather have a more GInputStream-like solution.

But this sort of depends on how it gets used... the original intent (still reflected in the current version of the JS API) was that this was for sending back "events" from the server, and so a signal-based API would make sense. But people are now talking about more streaming-like use cases as well...

> Some people had commented in spec's mailing list that it could be useful to
> send very large frames (in the order of Gygabytes) ...
> Of course, for browser specific
> implementations there is no choice as the JS API only deal with full message
> delivery, as you stated.

Well, if the JS API stays the way it is now, then people aren't going to be sending gigabyte-length messages. Or alternatively, if people really need to be able to send gigabyte-length messages, then the JS API won't stay the way it is now.
Comment 5 Eduardo Lima Mitev 2010-09-17 10:29:58 UTC
(In reply to comment #4)
> > The problem with g_input_stream_read{,_async} is that user can provide a buffer
> > and a size shorter than the message buffered for delivery.
> 
> This is also an issue for SCTP, so we'll need a solution there as well. It's
> possible that we want it to work one way when using GSocketConnection, but a
> different way when using GSocket directly...

I have been thinking on a solution using GIO input/output streaming API, before giving up for a signal-based approach. My proposal is to create GFramedInputStream and GFramedOutputStream, inheriting GFilterInputStream and GFilterOutputStream respectively. They would work as follows:

GFramedOutputStream
-------------------
This would have a method 'g_framed_output_stream_set_frame_len (stream, len)' that is called by the user to specify the length of the message that is being written.

When writing to the stream, a header is added to every data block, specifying whether this is the last fragment of the message or not (1 bit), and the length of the following fragment, encoded as follows (and inspired on WebSocket's new frame header):

    0 1 2 3 4 5 6 7 8
    +-+-------------+
    |M| Short Length|
    |O| 0-125=actual|
    |R| 126=+2 bytes|
    |E| 127=+8 bytes|

The MORE bit tells if more fragments of the message follow (MORE=1), or if it is the last fragment (MORE=0). The remaining 7 bits represent the fragment length: if <=125, then the 7 bits are the length of the fragment, if ==126, then length is the next 2 bytes, and if ==127, then length is the subsequent 8 bytes.

The user is free to call 'g_framed_output_stream_set_frame_len ()' at any moment, even after data has already been written into the stream. If that happens, the block headers previously written had the MORE bit on, as message length was considered to be unknown at that moment. Then, when the user specifies the frame length, the stream will properly mark the block where that length is reached. This would allow for sending messages whose length is unknown in advance, as for live generated data.

GFramedInputStream
------------------
This would have two methods:

    * 'g_framed_input_stream_get_frame_len (stream)': Returns the length of the message if it is already known by the stream, or 0 otherwise.

    * 'g_framed_input_stream_more_fragments (stream)':  This method returns TRUE if more fragments of the same message are expected, or FALSE otherwise. The user is expected to call this method after a read operation, in order to know if it is safe to deliver the already read data as a message, or if he/she should keep reading.

This stream, internally, will remove and interpret the header added by the output stream, passing the data up as a normal GInputStreamFilter, yet giving message information to the user using the above API.

-------------

I believe this approach could work for both WebSocket and SCTP streams. The implementation for the length encoding is commented to be straight-forward in the hybi list.

Opinions?
Comment 6 Dan Winship 2010-09-21 12:18:43 UTC
I don't think that particular framing is generically useful; for WebSockets, we want to use websocket framing (whatever that turns out to be), and for SCTP, the framing is part of the lower-level packet structure
Comment 7 Dan Winship 2011-04-12 14:32:35 UTC
It appears that the spec is starting to stabilize and it might make sense to start thinking about this again.
Comment 8 Eduardo Lima Mitev 2011-04-12 15:37:36 UTC
Yes, version 06 of the spec seems pretty much ratified now, we can start implementing that one. While version 00 (previously 76) is the only one currently implemented by browsers, it is likely to be dropped soon due to a security issues found.

About the high-level API, I now agree it makes more sense to implement whatever we specifically need to provide the websockets API as defined by W3C, and don't force using GIO streams. So I'm now in favor of something like a simple signal providing the packet received, its size and its type (binary vs. text). Also, a close signal providing a wasClean, reason and code arguments.

About my initial concern related to running a websocket server standalone and the same-origin policy, that issue is pretty much fixed with CORS, so I wouldn't worry much about this right now.
Comment 9 Eduardo Lima Mitev 2011-04-12 15:45:29 UTC
BTW, I have version 00 of the spec implemented at https://gitorious.org/eventdance/eventdance/blobs/websocket/evd/evd-websocket-server.c, also using GIO/GLib API. I will be implementing version 06 there as well as in libsoup. There should be many common bits, so I leave the link for reference.
Comment 10 Eduardo Lima Mitev 2011-08-31 08:27:07 UTC
I have been unable to allocate time to work on this lately, and likely I'm not going to have a chance before October. I'm sorry.

If somebody is urgent about this feature, please feel free to start working on it. I will try to help as much as possible and retake in less busy times.
Comment 11 Thomas Bechtold 2012-11-14 15:31:17 UTC
Just for the record. Found https://github.com/djdeath/libsoup.git which started to implement websockets.
Comment 12 Lionel Landwerlin 2014-01-17 17:43:50 UTC
I just pushed an import of the WebSocket implementation available from the Cockpit project : https://github.com/cockpit-project/cockpit/tree/master/src/websocket

It's available here : https://github.com/djdeath/libsoup/tree/websocket-cockpit

Hopes it helps to make this happen :)
Comment 13 Dan Winship 2014-01-18 21:26:16 UTC
wow. this is awesome.


The initial import commit should make it clearer exactly what changes you made from the cockpit code. (Did you just s/cockpit/soup/ or are there any other changes?)

_soup_websocket_util_header_contains() can be replaced with soup_header_contains(), and I guess maybe we should have soup_header_equals() and soup_header_empty() too?


In terms of public API, I think we want to use SoupSession and SoupServer rather than exposing SoupWebsockClient and SoupWebsockServer. Something like:

  GIOStream *soup_session_connect_websocket (SoupSession   *session,
                                             const char    *uri,
                                             const char    *origin,
                                             const char   **protocols,
                                             GCancellable  *cancellable,
                                             GError       **error);
  void       soup_session_connect_websocket_async  (...);
  GIOStream *soup_session_connect_websocket_finish (...);

initially, connect_websocket() would just do something like:

    return soup_websocket_client_new (uri, origin, protocols);

But later on, we can start getting rid of the HTTP and IO code in soup-websocket-connection.c, and use SoupSession's code instead (which will then get us authentication, cookies, proxies, SSL, etc). (Bug 721343 makes a start at letting us use SoupSession's internal machinery to create connections for non-HTTP purposes.)

(I'm not sure if the API should return GIOStream or SoupWebsocketConnection, or if it should take a "const char *uri" or a SoupURI...)


On the SoupServer side, we'd want something like

  void soup_server_add_websocket_handler (SoupServer             *server,
                                          const char             *path,
                                          const char             *origin,
                                          const char            **protocols,
                                          SoupWebsocketCallback   callback,
                                          gpointer                user_data,
                                          GDestroyNotify          dnotify);

  typedef void (*SoupWebsocketCallback) (SoupServer              *server,
                                         const char              *path,
                                         SoupWebsocketConnection *connection,
                                         SoupClientContext       *client,
                                         gpointer                 user_data);

I think? I need to read the websockets specs now to get a better idea about this...
Comment 14 Lionel Landwerlin 2014-01-18 22:16:40 UTC
> The initial import commit should make it clearer exactly what changes you made
> from the cockpit code. (Did you just s/cockpit/soup/ or are there any other
> changes?)

The initial import is just a s/cockpit/soup/ indeed.
I didn't find a way to get the tests and everything altogether because they rely on internal functions.

> 
> _soup_websocket_util_header_contains() can be replaced with
> soup_header_contains(), and I guess maybe we should have soup_header_equals()
> and soup_header_empty() too?

Ok, I'll do that.

> 
> 
> In terms of public API, I think we want to use SoupSession and SoupServer
> rather than exposing SoupWebsockClient and SoupWebsockServer. Something like:
> 
>   GIOStream *soup_session_connect_websocket (SoupSession   *session,
>                                              const char    *uri,
>                                              const char    *origin,
>                                              const char   **protocols,
>                                              GCancellable  *cancellable,
>                                              GError       **error);
>   void       soup_session_connect_websocket_async  (...);
>   GIOStream *soup_session_connect_websocket_finish (...);
> 
> initially, connect_websocket() would just do something like:
> 
>     return soup_websocket_client_new (uri, origin, protocols);
> 
> But later on, we can start getting rid of the HTTP and IO code in
> soup-websocket-connection.c, and use SoupSession's code instead (which will
> then get us authentication, cookies, proxies, SSL, etc). (Bug 721343 makes a
> start at letting us use SoupSession's internal machinery to create connections
> for non-HTTP purposes.)
> 
> (I'm not sure if the API should return GIOStream or SoupWebsocketConnection, or
> if it should take a "const char *uri" or a SoupURI...)

Sounds cool, I can look into that too.

> 
> 
> On the SoupServer side, we'd want something like
> 
>   void soup_server_add_websocket_handler (SoupServer             *server,
>                                           const char             *path,
>                                           const char             *origin,
>                                           const char            **protocols,
>                                           SoupWebsocketCallback   callback,
>                                           gpointer                user_data,
>                                           GDestroyNotify          dnotify);
> 
>   typedef void (*SoupWebsocketCallback) (SoupServer              *server,
>                                          const char              *path,
>                                          SoupWebsocketConnection *connection,
>                                          SoupClientContext       *client,
>                                          gpointer                 user_data);
> 
> I think? I need to read the websockets specs now to get a better idea about
> this...

I'm adding a signal on the SoupWebsocketServer atm to let accept or reject a client based on criteria specific to the application. Is that what this API is meant to do?
Comment 15 Dan Winship 2014-01-19 00:32:38 UTC
The way that SoupServer works is that you register handlers on paths, and when a request comes in matching one of those paths, it calls the corresponding handler. You can also add SoupAuthDomains to the server to force some or all paths to require HTTP authentication.

There's currently not any other way of filtering things before the handler gets run. Instead, you just do whatever other checking you need from the handler. (In the HTTP case, you can just return whatever HTTP error you want on the SoupMessage... I guess in the websocket API I proposed above, I was assuming that it would always accept the websocket connection, and if you wanted to return an error, you'd have to return it at the websocket level. Maybe that's not the best way to do it though... maybe you should register both a SoupServerCallback and a SoupWebsocketCallback on the path, and the SoupServerCallback would be called first, and if it sets the status on its SoupMessage, then SoupServer would just return that message to the caller. Otherwise SoupServer would handle the websocket handshake, returning "101 Switching Protocols" to the client, and then eventually invoke the SoupWebsocketCallback.
Comment 16 Lionel Landwerlin 2014-01-22 17:31:55 UTC
Ok, I just pushed my latest update with the API you described :

https://github.com/djdeath/libsoup/commits/websocket-cockpit

I suppose one more thing I should be doing is only return WebsocketConnection once the WebSocket is ready to be used. Right now you still have to way for the WebSocket handshake to be completed.

Dan, let me know what you think.
Comment 17 Dan Winship 2014-03-18 22:01:55 UTC
Sorry for not getting back to you sooner... busy with work stuff.

The branch wip/server-steal has a bunch of SoupServer-related stuff that I'm planning to push as soon as we are branched for 2.46. In particular, it has a method soup_client_context_steal_connection() that cleanly steals the GIOStream of a particular SoupServer connection. You should be able to use that from soup-server.c:call_websocket_handler(). (It's possible the API should be changed in some way to better deal with sending an HTTP response before yielding the connection?)

There's also wip/proxy-connect (which is actually built on top of wip/server-steal), which has the start of an API that would be used as the base of soup_session_websocket_connect*(). The functions there make use of the existing SoupSession queue infrastructure rather than creating SoupConnections directly, which is primarily to deal with the possibility of having to connect through an HTTP proxy, which is something you presumably want for websockets as well. Again, the API may not be 100% right (and also, the code there probably doesn't currently work; see bug 721343 comment 12).


Additional comments on the current state of the branch:

AVAILABLE_IN annotations and "Since" headers should be 2.48 now.


> soup-headers: straighten parsing with header lines missing ':'

Unfortunately, the current behavior is necessary to deal with certain stupid servers. (Generally situations where PHP scripts accidentally write garbage into the headers.)


> tests: add websocket tests from the Cockpit project

commit message seems wrong here, since this is actually just porting the tests for the SoupMessageHeaders change


> message-headers: add contains() method

>+#define _GNU_SOURCE /* For strcasestr() */

You can't use libc's case-insensitive/case-converting functions for protocol data, because they're locale-sensitive and so run into the stupid dotless-i vs dotted-i distinction in Turkish. There's no g_ascii_strcasestr(), but you can just have soup_message_headers_contains() call soup_header_contains(), which does what you want anyway.


> uri: add support for WebSocket schemes

I think you need to update soup_scheme_default_port() too?


> websocket-connection: expose async-context property

explicitly-specified GMainContexts are legacy API. SoupWebsocketConnection should just always use the thread-default main context. Feel free to add a

  g_return_if_fail (priv->use_thread_context == TRUE, NULL);

to the start of soup_session_websocket_connect(), etc.


> server: add WebSocket handlers support

You can have the SoupServerWebsocketCallback gtk-docs just say "See the documentation for SoupServerCallback for more details on how paths are handled" rather than copying all the gory details over.
Comment 18 Lionel Landwerlin 2014-05-17 15:57:13 UTC
Just finished updating my branch following your comments :

https://github.com/djdeath/libsoup/tree/websocket-cockpit

I included the commits from :

https://git.gnome.org/browse/libsoup/log/?h=wip/proxy-connect
https://git.gnome.org/browse/libsoup/log/?h=wip/server-steal

and made use of these new APIs. Though it required some slight tweaks for them to be used in websocket scenario.
Comment 19 Dan Winship 2014-06-07 15:36:54 UTC
Hoping to get a chance to look at this this weekend. One thing I note after quickly skimming is that some of your fixes to existing libsoup code look like they might end up breaking other things... You should probably install apache and php, etc, so you can run the full "make check" suite, because that will catch a lot of places where an innocuous-looking fix actually breaks functionality.
Comment 20 Dan Winship 2014-06-07 16:05:30 UTC
(pushed a fixup to wip/proxy-connect to deal with the loss of the SoupConnection:ssl property)
Comment 21 Nicolas Dufresne (ndufresne) 2014-09-24 15:45:59 UTC
Hello, I just poked at Lionel branch, rebased it on top of master. Outside a race with idle callback between Client and Connection (I'll try to wrap a proper patch) this seems to work mostly. Let me know if I can help in anyways. I'll be using this branch to build some sort of minimal WebRTC server, that should help me comment on the API later.
Comment 22 Dan Winship 2014-12-14 14:42:06 UTC
OK... hacked on this at the Web Engines Hackfest last week. I think it is now in suitable shape to commit, though it would be good to get feedback from the people actually using it first... It's in the "websocket" branch in git.


In the course of integrating the code into SoupSession and SoupServer, I ended up throwing out about half of the original cockpit code, and reorganizing a bunch more, to the extent that it no longer really made sense to start with a clean import and then make changes to it. So it just gets dumped in in a single commit now.

The biggest change from Lionel's original version is that there is no longer any public API for handshaking; it's assumed you'll use SoupSession to do the client-side handshake (thereby getting proxies and authentication and stuff like that for free), and SoupServer to do the server side. Is there anyone who was using this code, but needed to do all the handshaking themselves?
Comment 23 Nicolas Dufresne (ndufresne) 2014-12-14 14:53:58 UTC
On my side, client side is using soup_websocket_client_new(uri, origin, cancellable), does that still exist ?

I'll try and port, while having a look at the new API this week. Thanks a lot for working on this.
Comment 24 Nicolas Dufresne (ndufresne) 2014-12-14 15:33:05 UTC
So after a quick look, things have shuffled around indeed. So if I understood well, connecting through websocket would look like:

1. s = soup_session_new ()
2. m = soup_message_new ("GET", "ws://...")
3. soup_session_websocket_connect_async (s, m, orig, protos, cancel, cb, data)
4. On cb, soup_session_websocket_connect_finish()

That seems like a nicer blend with soup, specially that now you can do HTTP request and create websocket connection from the same session. Another difference is that the "open" callback is now gone, since the connect_async() callback handle this role. Let's try and use it now, but it all look very good changes to me.

I don't think it's really needed to expose soup-websocket-client.h. Unless there is someone would like to implement a GProxy on top of it ?
Comment 25 Nicolas Dufresne (ndufresne) 2014-12-14 16:21:06 UTC
I did a quick port, the biggest issue I have now is that while libsoup client to libsoup server works, chrome client to libsoup server won't anymore. I get error code 400, no message.
Comment 26 Lionel Landwerlin 2014-12-14 16:26:01 UTC
Hey there,

Just tried the branch, and got my stuff (noflo-gnome) to work with it after a few patches : https://github.com/djdeath/libsoup/tree/websocket

Nicolas, I believe https://github.com/djdeath/libsoup/commit/20e7403fd284e8d98e93406920b7498095e34917 should fix your problem.

Dan, thank a lot for spending time on this! :)
Comment 27 Nicolas Dufresne (ndufresne) 2014-12-14 16:35:35 UTC
Wow that was fast, works for me ! Final concern, do we really want to set a timeout by default ? (at least I get timeout after a little while).
Comment 28 Dan Winship 2014-12-14 18:40:40 UTC
(In reply to comment #26)
> few patches : https://github.com/djdeath/libsoup/tree/websocket

>  * soup_websocket_connection_send:
>  * @self: the WebSocket
>  * @type: the data type of message
>- * @data: the message contents
>+ * @data: (type utf8): the message contents
>  * @length: the length of @data, or -1 if @data is a NUL-terminated string

That won't work; the function is supposed to support both binary and string data. But maybe we should just split it into send_text() and send_binary()?

> websocket: server: ignore extensions

oops. not sure where that check came from...

(In reply to comment #27)
> Final concern, do we really want to set a
> timeout by default ? (at least I get timeout after a little while).

Doh... the "timeout" property on the SoupSession gets pushed all the way down to the GSocket, so it would still be around even after stealing the connection. That's confusing and probably not what the caller wants though...
Comment 29 Lionel Landwerlin 2014-12-15 02:29:56 UTC
(In reply to comment #28)
> (In reply to comment #26)
> > few patches : https://github.com/djdeath/libsoup/tree/websocket
> 
> >  * soup_websocket_connection_send:
> >  * @self: the WebSocket
> >  * @type: the data type of message
> >- * @data: the message contents
> >+ * @data: (type utf8): the message contents
> >  * @length: the length of @data, or -1 if @data is a NUL-terminated string
> 
> That won't work; the function is supposed to support both binary and string
> data. But maybe we should just split it into send_text() and send_binary()?

What about this :

https://github.com/djdeath/libsoup/commit/c4172212975e6f7349cd3ee72c809999f2d88baa#diff-5c7a8cf4ec3d800c4e0bc624a26addafR1412

Stolen from g_file_set_contents :)
Comment 30 Dan Winship 2015-02-10 11:57:30 UTC
[mass-moving all "UNCONFIRMED" libsoup bugs to "NEW" after disabling the "UNCONFIRMED" status for this product now that bugzilla.gnome.org allows that. bugspam-libsoup-20150210]
Comment 31 Dan Winship 2015-02-27 23:05:32 UTC
OK, repushed websocket branch; I plan to merge this to master this weekend and put it in a release next week. (Oh wait, note to self: fill in appropriate bug numbers in the commit messages.)

Changes from the previous version:

  - split soup_websocket_connection_send() into send_text() and
    send_binary(), to deal with the fact that bindings would need
    different annotations for the two cases

  - merged the rest of Lionel's fixes

  - exposed some lower-level non-SoupSession/SoupServer-bound APIs, in case
    people want that...

  - absorbed SoupWebsocketClient and SoupWebsocketServer into
    SoupWebsocketConnection since the setup/validate APIs are all now
    separate from the I/O APIs, and so there was nothing distinct
    left in the client and server subclasses

Oh, I forgot to deal with the timeout bug. OK, so, add bug numbers to the commit messages, and fix the timeout bug. But other than that, I think this is what's going in.
Comment 32 Lionel Landwerlin 2015-02-28 09:05:35 UTC
Just tested, works fine :)
Comment 33 Dan Winship 2015-03-01 15:39:58 UTC
pushed to master!

(I also added some more documentation, and also found an alternative libsoup-only fix for bug 722723, so the websocket code no longer requires glib >= 2.42)