GNOME Bugzilla – Bug 785730
New elements: libSRT src/sink
Last modified: 2017-11-07 19:35:46 UTC
SRT is another type of UDP-based protocol that optimizes streaming across unpredictable networks. The SRT library is developing in github(https://github.com/Haivision/srt). I created an initial version of srtsrc and srtsink. Although the protocol is based on UDP, the APIs look like connection oriented protocol so in my initial version, srtsrc is a client, srtsink is a server. I tested streaming by the below pipeline. For srtsink, I couldn't find a proper TS file so I just transcoded sintel trailer. gst-launch-1.0 -v \ filesrc location=~/trailer.mp4 \ ! decodebin name=d d. ! vaapih264enc ! queue ! mpegtsmux name=mux ! rtpmp2tpay ! srtsink \ d. ! avenc_ac3 ! mux. gst-launch-1.0 -v \ srtsrc uri="srt://127.0.0.1:7001" \ caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)MP2T" \ ! rtpmp2tdepay ! decodebin name=d d. ! autovideosink sync=false d. ! autoaudiosink sync=false
Created attachment 356778 [details] [review] [1/2] srtsrc
Created attachment 356779 [details] [review] [2/2] srtsink
Should it be srtclientsrc and srtserversink then?
Created attachment 356814 [details] [review] [1/2] srtclientsrc
Created attachment 356815 [details] [review] [2/2] srtserversink
Created attachment 356818 [details] [review] [2/2] srtserversink
Created attachment 356831 [details] [review] [2/2] srtserversink changed to use 'struct' instead of just socket descriptor in a signal.
> changed to use 'struct' instead of just socket descriptor in a signal. How will this work in an application? G_TYPE_POINTER is not usable in bindings, and the client struct will not be exposed in any header files for the application.
(In reply to Tim-Philipp Müller from comment #8) > > changed to use 'struct' instead of just socket descriptor in a signal. > > How will this work in an application? G_TYPE_POINTER is not usable in > bindings, and the client struct will not be exposed in any header files for > the application. I rush to implement so I missed some points. I'll submit the patch again. Thanks for advice.
Created attachment 356906 [details] [review] [2/2] srtserversink
Also, in SRT the client/server is not linked to sender/receiver. So I think we could have a client-sender to a server-receiver too.
Created attachment 357196 [details] [review] [1/2] srtclientsrc correct copyright
Created attachment 357197 [details] [review] [2/2] srtserversink correct copyright
Created attachment 357448 [details] [review] RFC: srt elements I assumed no one started reviewing the proposed patches. So I squashed and re-organized codes. In addition, as Olivier's suggestion, srtserversrc and srtclientsink are introduced too.
Created attachment 358647 [details] [review] RFC: srt elements Added more enhancements. Thanks, Olivier!
Wouldn't it be easier if the server/client selection was done via the URI instead of creating new elements? libsrt has a tool called stransmit, which works like that. And it seems there is also the simultaneous open mode, which is not covered by these elements. stransmit behaves like this: # client mode - assuming that 192.168.1.1 is a remote IP srt://192.168.1.1:7000 srt://192.168.1.1:7000?mode=client srt://192.168.1.1:7000?mode=caller # server mode - assuming that 192.168.1.1 is a local interface IP srt://192.168.1.1:7000?mode=server srt://192.168.1.1:7000?mode=listener srt://:7000?adapter=192.168.1.1 srt://:7000 (binds to 0.0.0.0) # SO mode - assuming that 192.168.1.1 is a local interface IP and 192.168.1.2 is a remote one srt://192.168.1.2:7000?mode=rendezvous (connect to 192.168.1.2 and bind to 0.0.0.0) srt://192.168.1.2:7000?mode=rendezvous&adapter=192.168.1.1 (connect to 192.168.1.2 and bind to 192.168.1.1)
Would it be possible to set the rank of srtserversrc to secondary? As is, playbin will sometimes select srtserversrc instead of srtclientsrc. Also, from my testing it appears that srtserversink will accept and handle multiple client connections. Is this intended behaviour? I don't believe the stransmit app works in this way, but supporting multiple clients would be a fantastic feature :)
Great work! -) Btw,it seems that SRT did not provide compilation of Android platform instructions, no way to quickly test demo on Android.
(In reply to George Kiagiadakis from comment #16) > Wouldn't it be easier if the server/client selection was done via the URI > instead of creating new elements? I thought socket type per element would be easier than URI parameters. From the beginning, I considered a feature to switching socket type by setting property (or parsing URL) in an element, but I felt I needed to use more if-else to set socket options. How about introducing a kind of wrapper bin to deploy a proper socket type element by parsing URI parameters? > it seems there is also the simultaneous open mode, > which is not covered by these elements. Yes, I wasn't sure that mode and suffered from lack of testing for server and client connections. (In reply to Brendan Lockhart from comment #17) > Would it be possible to set the rank of srtserversrc to secondary? I think it could be. This idea could be related to the above (introducing a wrapper bin). My simple idea is lowering the ranks of source elements to secondary or marginal, then the wrapper bin has primary rank. What do you guys think about this idea? > Also, from my testing it appears that srtserversink will accept and handle > multiple client connections. Is this intended behaviour? I don't believe the > stransmit app works in this way, but supporting multiple clients would be a > fantastic feature :) Yes, it's an intended implementation because I thought we can get a live stream via srt so any client can be connected, even though stransmit doesn't provide/support multiple clients. (In reply to luckychou from comment #18) > Btw,it seems that SRT did not provide compilation of Android platform > instructions, no way to quickly test demo on Android. Are there any tutorial or instruction of using 'libsrt' on Android? I can't find that kind of information while quick searching. It would be a starting point anyway.
Created attachment 360715 [details] [review] srt elements Added little more tunning of socket options.
Is it possible that the line: "static GParamSpec *properties[PROP_LAST + 1];" in gstsrtclientsrc.c has an extra "+ 1"? I think that line may have been missed in your last patch. That script may also need the param spec for PROP_LATENCY. If the srtserversink supports multiple clients, should we add a mutex surrounding changes to the clients GList?
(In reply to Brendan Lockhart from comment #21) > Is it possible that the line: > "static GParamSpec *properties[PROP_LAST + 1];" in gstsrtclientsrc.c has an > extra "+ 1"? I think that line may have been missed in your last patch. That > script may also need the param spec for PROP_LATENCY. Oh, I missed some patches in my local while squashing. I should verify it. Thanks for reviewing. > If the srtserversink supports multiple clients, should we add a mutex > surrounding changes to the clients GList? Yes. Maybe.
Created attachment 360750 [details] [review] srt elements Add mutex locks to serversink to protect clients list
Created attachment 362912 [details] [review] Updated patch with a bunch of things I learned while doing interop testing. In particular, added a latency property to all elements. Also added the rendez-vous mode to the *client* elements. Factored out some of the common code between the client elements
Created attachment 362914 [details] [review] Same as previous patch, but I had forgotten to merge in the mutex protecting the client list. I also took the liberty to replace it with the object lock instead of introducing anotehr mutex.
Merged Author: Justin Kim <justin.kim@collabora.com> Date: Mon Jul 31 14:38:34 2017 +0900 srt: Introduce SRT source and sink SRT[0] is an open source transport technology[1] that optimizes streaming performance across unpredictable networks. Although SRT is based on UDP, it works like connection-oriented protocol. However, it doesn't mean that the SRT server or client is necessarily to link to a receiver or a sender so, here, the pairs of source and sink elements are introduced. - srtserversink: SRT server to feed SRT stream - srtclientsrc: SRT client to get SRT stream from srtserversink - srtclientsink: SRT client to send SRT stream - srtserversrc: SRT server to listen from srtclientsink [0] https://github.com/Haivision/srt [1] http://www.srtalliance.org/ https://bugzilla.gnome.org/show_bug.cgi?id=785730