GNOME Bugzilla – Bug 711257
Streaming does not always start with a key-frame
Last modified: 2014-02-25 22:28:32 UTC
As pipelines driven by live sources are set to state PLAYING after DESCRIBE, streaming is not guaranteed to start with a key-frame. The stream will basically start with an arbitrary frame when the transports are configured in PLAY. Any ideas how to fix that problem in a generic way would be greatly appreciated :) How about tweaking gst_rtsp_media_set_state() to support GST_STATE_READY and set media/pipeline to state READY from handle_describe() after generating the SDP: handle_describe() { ... media = find_media(); /* creates and prepares the media, pipeline is set to PLAYING if source is live */ sdp = create_sdp(); rtsp_media_set_state (media, GST_STATE_READY); ... } This will require corresponding changes to handle_play() in order to be able to create the RTP-Info header. gst_rtsp_session_media_set_state (sessmedia, GST_STATE_PLAYING) will be called right before quering payloaders for sequence numbers and rtp times. For that we would also need to wait until all payloaders produce one buffer and possibly block pads so that PLAY response is sent before any RTP data. Please advise if this approach makes sense?
Another approach is to send am upstream force key unit event, then drop all buffers before a downstream force key unit event comes back. This would also work if more than one client listen to the same encoded stream.
I think a combination of the following would work: - in DESCRIBE, prepare and unprepare the media to probe the SDP, cache the SDP on the factory - in SETUP, generate streams from the cached SDP without setting the pipeline to PLAYING. This assumes the media streams and their types don't change since caching the SDP - in PLAY, set things to playing and link the dynamic pads to the streams - for new clients, send force key-unit to quickly generate a keyframe Alternatively, a cache could be placed before the sinks, when a new client is connected, we quickly send the cached media from a last keyframe to fill up the client buffers and make it start from a keyframe.
Just as a side note. I currently solve this issue on the client side, just waiting for a keyframe. Anyone working on any of the proposals?
I talked to Wim on IRC a while ago and we agreed on an intermediate solution for now where the media is reset(if an option is explicitly enabled) after doing the SDP. And reset would mean that the pipeline is set to state NULL but all the streams kept so that they cane be used in SETUP for configuring the transports. I have basically implemented that and will soon upload the patch.
In addition to my previous comment, that solution will only work for medias without dynamic payloaders. Later on it will be fixed for the dynamic case as well and complemented with the force key unit event for medias which are shared.
Created attachment 261343 [details] [review] client: reset media after doing the SDP Patch 5/5: To avoid streams starting with delta frames and CPU load before the actual RTSP PLAY command we now reset the media after doing the SDP. Reset means that the pipeline state will be set to NULL but all streams will be kept and used for configuring the transports in SETUP. The pipeline will be played in PLAY. This approach currently only works for medias without dynamic payloaders. Can be enabled on a per factory/media basis.
Created attachment 261344 [details] [review] stream: add API for blocking streams 1/5
Created attachment 261345 [details] [review] media: add API for blocking streams 2/5
Created attachment 261346 [details] [review] media: add support for resetting media 3/5 Reset means that the pipeline state will be set to NULL but all streams will be kept. Media must be prepared first. Reset currently only works for medias without dynamic payloaders.
Created attachment 261347 [details] [review] media-factory: enable media reset on a per factory level 4/5
Created attachment 261351 [details] [review] media: add support for resetting media 3/5 (fixing small bug in the previous patch, gst_rtsp_media_undo_reset calls now start_preroll instead of start_prepare) Reset means that the pipeline state will be set to NULL but all streams will be kept. Media must be prepared first. Reset currently only works for medias without dynamic payloaders.
Created attachment 262823 [details] [review] client: reset media after doing the SDP 5/5 (fixed patch - check function return values) To avoid streams starting with delta frames and CPU load before the actual RTSP PLAY command we now reset the media after doing the SDP. Reset means that the pipeline state will be set to NULL but all streams will be kept and used for configuring the transports in SETUP. The pipeline will be played in PLAY. This approach currently only works for medias without dynamic payloaders. Can be enabled on a per factory/media basis.
It all looks a bit too much with all the new blocking API and such... I can't help but think that there is a more simple solution for this. What we want is that after the SDP has been made, we 'stop' the pipeline. For non-live, this is already the case (we are in PAUSED), for live pipelines this actually does something. Then there should be an option to set the pipeline state to PAUSED or NULL when we stop after the SDP. And it should automatically do the right thing when we resume (to PLAYING) This would also make it work when you PAUSE the live pipeline from the client. For live pipelines, we never want to let data go through when we _prepare() so we need to install blocking probes. We need to remove those probes when we go to PLAYING.
I commited something similar but with the following changes: - simplify the blocking of the stream by using a message. I think this should still be done by leaving the sinks in PAUSED and using the regular ASYNC_DONE message to wait for preroll - preroll all live streams in blocking mode so that no data leaves the sinks when we set them to PLAYING. Make streams unblock when the media is set to PLAYING. - introduce the concept of 'suspend'. There are different ways to suspend the pipeline after generating the SDP and in PAUSED. Current suspend modes are PAUSED, which sets the pipeline to paused (useful for on-demand rtsp sources), NONE, which does nothing and is the old/new default behaviour and RESET, which sets the pipeline to the NULL state, which is what this bug is about. Please check if this solves your problem, I would still like to simplify the blocking later.
Seems to work fine, the only problem i have noticed so far is that media is not unprepared after media_set_state (NULL). May be something like this will solve it: http://pastebin.com/yBKKhEGj
commit 3b4894c4f1ea28cc317f0c53b110cf678d27bf10 Author: Ognyan Tonchev <ognyan@axis.com> Date: Fri Nov 29 15:50:23 2013 +0100 media: also do state change in suspended state