GNOME Bugzilla – Bug 634419
Selecting a subset of streams available in rtspsrc
Last modified: 2013-08-21 18:31:33 UTC
We are currently developing a GStreamer-based video/audio streaming application that can receive RTSP/RTP streams from security cameras. We currently need to enhance the rtspsrc element and want to solicit feedback from the GStreamer developer community. Some cameras we need to receive streams from have both audio and video streams available and will describe both a video and an audio stream in their response to a DESCRIBE request. The GStreamer rtspsrc element will send a SETUP request for each of the media streams listed in the SDP of the DESCRIBE response and the camera will transmit both streams to our application when using the rtspsrc element. In certain circumstances, we would like to change the behavior of rtspsrc in our application. Sometimes, when a camera (herein referred to as a “device”) indicates that it has multiple streams available in its DESCRIBE response, we would like to send a SETUP request for only one of the streams. Stated more generally, if a DESCRIBE response indicates that it has M number of media streams available, we might want to send SETUP requests for some subset of the available streams less than M. This would tell the device to only transmit those streams we’re interested in when we send a PLAY request. To accomplish this, we were thinking of changing rtspsrc so that RTP ghost pads would be created for each stream that the device enumerates in its DESCRIBE response. However, we would only send a SETUP request (and hook up all the internal rtspsrc bin plumbing) when the appropriate ghost pad is linked to another element’s sink pad. This way, the device would only play those streams we’re interested in when we send the RTSP PLAY request. This change is obviously a significant departure from how rtspsrc currently operates where all streams listed in the DESCRIBE response’s SDP are set up by rtspsrc in the state transition from READY to PAUSED. As such, we wanted to pick your brain briefly before moving too far on this change. A few questions: 1. Is this enhancement something that would merit inclusion in the GStreamer project? 2. If so, would it make sense for it to be a completely different rtspsrc-like element (maybe something like “rtspsrc-selective”)? Or would it be preferable to have both “modes” reside in a single rtspsrc element and we would change the operating mode of rtspsrc based on an element property? 3. Are there any issues that jump out as being something to watch for as we think about implementing this scheme?
Before submitting this enhancement request to bugzilla, I contacted Wim Taymans and here was his response: " My first thought is that your approach of only sending the SETUP for linked pads, is not how I would like to see this done in GStreamer. Pad linking and stream selection should be two separate issues IMO. I would like to suggest a different approach where the rtspsrc emits a signal that the app can connect to, to select the streams it would like to select. You would first have a signal when the SDP becomes available. The app can then inspect the complete SDP and see what streams are interesting. The selection of the streams could be implemented with an action signal. There could also be an action signal to retrieve the currently selected streams. We don't really have a good API for doing stream selection yet but if we were to make an interface for this, we would probably end up with methods that are similar to the approach above. Hope that helps, Wim "
Ideally, we don't want our app to parse SDP. I think all the information we need to determine which streams are interesting would be in the caps of the src pads created by rtspsrc. The app could figure out whether a stream is of interest when the app gets the "pad-added" signal. Maybe if we're not interested in a stream, we can implement an action signal that would "unselect" the stream associated with an rtspsrc pad and remove the pad.
(In reply to comment #2) > Ideally, we don't want our app to parse SDP. I think all the information we > need to determine which streams are interesting would be in the caps of the src > pads created by rtspsrc. The app could figure out whether a stream is of > interest when the app gets the "pad-added" signal. The problem is that the setup request is already sent and the server is already streaming for a while when the pad-added signal is emitted. I would prefer a way to select streams before the SETUP is performed. > Maybe if we're not > interested in a stream, we can implement an action signal that would "unselect" > the stream associated with an rtspsrc pad and remove the pad. That should certainly be possible and you can still use this to unselect streams in the pad-added signal.
how about adding a set of flags to enable/disable all audio/video streams? That would be easy to add and use.
With this you could currently remove streams from the SDP you are not interested in. commit 1db7e620607350db94528d40aa3d932ce9fd215c Author: Wim Taymans <wim.taymans@collabora.co.uk> Date: Mon Jul 1 17:28:55 2013 +0200 rtspsrc: add signal to notify of the SDP This way, the app can look and modify the SDP.
commit 7c950ef3f2a15af174eab15492da662d02a9cf4b Author: Wim Taymans <wim.taymans@collabora.co.uk> Date: Tue Jul 2 10:37:50 2013 +0200 rtspsrc: add select-stream signal Add a signal to let the app select what streams will be selected. See https://bugzilla.gnome.org/show_bug.cgi?id=634419
This seems to be fixed now?