GNOME Bugzilla – Bug 537382
playbin: stream switching issue between different (sparse) video streams
Last modified: 2018-11-03 11:14:21 UTC
go to http://www.river-valley.tv/conferences/lgm2008/#0203-Michael_Dominic_Kostrzewa and download the qt-mov file (http://media.river-valley.tv/conferences/lgm2008/quicktime/0203-Michael_Dominic_Kostrzewa.php) play it with the seek tets application using playbin2. Swithcing the videotrack does not work.
some more info, when I switch the video track in seek tool, it prints setting current video track 2 but I still have to seek a bit manualy to get frames from the new stream.
Without looking at the code, a problem could be that playbin2 should send FLUSH_START/FLUSH_STOP events on the audio/video/text pads before switching the stream, to flush all queues, etc.
Created attachment 148551 [details] [review] initial patch Makes input-selector block waiting for the buffer endtime before dropping a buffer. Improves the switch from a 'normal' video stream to a sparse one (in this movie, from track 3 to 1)
Looks good but for your locking questions I need to take a closer look. Does it also improve switching from one normal to another normal audio/video track?
Still haven't tested, but shouldn't make much difference because frames have much smaller duration in this case. The other fix should be getting a way to drop downstream buffers (including the one that is currently at the sink being rendered) before pushing the new stream buffer, making the switch 'instantaneous'. Fixing this, the switch from sparse to 'normal' streams should work and we won't have to fix multiqueue. Any ideas on how to do it?
part-events has a new not-implemented 'drain' event that seems like what we are trying to do, except that we don't want buffers to be played :)
Ok, what should we do about this? I implemented something similar for input-selector last month and was going to push it soon. My approach was to synchronize based on the running time instead of waiting for the clock
Comment on attachment 148551 [details] [review] initial patch As discussed on IRC using the running time is better than the clock. This is committed now.
commit 216258fbc275c5acdec6d91028cf4f274bb5f9f4 Author: Sebastian Dröge <sebastian.droege@collabora.co.uk> Date: Thu May 26 11:41:50 2011 +0200 playbin2: Let the input-selectors sync all streams to the running time This is especially needed when switching between a non-sparse and sparse video stream, see bug #537382. It also lowers the time needed for switching between streams a bit. Only thing that does not work quite good now is switching from the sparse to the non-sparse stream. The switch will only happen after the current, long-duration frame is expired.
*** Bug 633742 has been marked as a duplicate of this bug. ***
Created attachment 273552 [details] [review] inputselector: add flush on switch property Defaults to false for backward compatibility. It is used to send a flush event downstream when switching pads to immeditately start displaying the new active stream. Very useful when switching from a sparse to a non-sparse stream, avoiding major delays for the switch to happen. The flushes are sent from the set_property call because waiting to do it from the streaming thread would take too long for sparse streams and this is what we want to avoid.
Created attachment 273553 [details] [review] playbin: activate flush-on-switch and sync on the clock on inputselector To be able to make stream switches happen faster
These 2 patches are a partial solution to the problem. With those the switch from sparse to non-sparse works immediately. The problem is when switching back to the sparse one. The sparse stream is already much ahead (3 minutes) and when the non-sparse stream gets deactivated it will block waiting for a buffer from the sparse stream but that won't happen as the queues will fill up before the next sparse buffer is found on the demuxer. Not sure how to handle this, the first buffers that would be pushed are already at inputselector in the cached_buffers queue but it is not a good idea to push those from outside the correct streaming thread. Any ideas here?
Also, the playbin patch makes inputselector use the clock rather than running time to sync streams, this is mostly to avoid it dropping the cached buffers while they could still be needed downstream. Using the running time for the sync and the clock for dropping buffers from the cache would also work.
If you sync on the clock in input-selector, you basically make your element a live element... and also all buffers are going to arrive too late in the sink :) I don't think that's a viable solution for this problem. The flushing makes sense though.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/14.