GNOME Bugzilla – Bug 703846
Streams added/removed ... without creating a new group
Last modified: 2016-06-02 07:43:12 UTC
Currently the logic for streams being added from a demuxer (or any other element with dynamic source pads for that matter) will be: * Add new source pads * emit no-more-pads * All pads that were added before "no-more-pads" are considered as part of a "group" If the configuration changes (say from 1 video + 1 audio stream to the same but with an additional audio streams), the demuxer/element needs to: * Remove all pads * Add all pads again * emit no-more-pads * All new pads are considered as part of a new "group" (since there was a previous "no-more-pads"). The same logic would happen for the opposite (An audio stream going away). Note that I am not talking about "chained" setups (like chained oggs or switching mpeg-ts/dvb programs), but changes/updates in the "same" underlying stream (i.e. elementary streams being added/removed as an update). For the chained use-cases the logic is fine (though could benefit from the following discussion). There are a few problems with this: * From a logical point of view it is not a "new" group but an update. * Due to the tear-down/drain system, we end up with lost data and potentially corrupted end result. ** You might be adding/removing pads in the middle of "another" stream not being at a fully-recoverable point. ** The example that comes to mind is you're viewing a H264+AC3 stream, A new additional audio stream is added, you tear-down/re-setup "H264 + 2 audio streams" ... and start pushing h264 data from the middle of nowhere, no SPS/PSS, the video decoder will end up pushing out incomplete/corrupted video until it sees the next SPS/PSS. * Overhead of temporarily having several decoders (while the previous group drains out and the new group prerolls) working at the same time. * Not being able to re-use elements for the same stream (And avoid re-fetching a bufferpool, initialize devices, ....) Examples: * mpeg-ts streams When watching one program, the configuration could change when the underlying content changes (ex: Movie shown in 5.1 AC3 with 4 subtitle channels and 3 different audio... and switches to the news which is 2.0 AC3 with only 1 subtitle channel and no extra audio channel). And the other way round. * CC (CEA 608/708) embedded in mpeg/h264 elementary streams A parser could add a source pad when it detects Closed Caption is contained in the elementary stream private data. * HLS alternative streams. You might not want to expose the alternate streams until they are actually chosen, and when you chose one hlsdemux would have to create a new source pad ... which is an additional stream (and not a new group). * ... Proposals (?) * A new signal to notify pad addition is complementary and not from a new group ? "pad-updated" before adding the pad ? * The opposite: unless a certain signal is emitted, pads being added are considered to be updates of the existing group ? "new-group-start" ? The first solution might be better since it would be backwards compatible.
Or maybe just consider newly added pads that don't happen after all pads are removed as updates to the current group? That would be backwards compatible behaviour too and currently decodebin is explicitly ignoring such pads. I don't see a problem with that, shouldn't break any existing code... just (as all your other proposals) are probably not easy to implement
That last solution might just work indeed ... After all if you remove all streams, it means what's coming is completely unrelated to the previous program/stream/... Will try to figure out something based on that logic.
I'm back working on #704881 and the solution there probably needs to be based on the design you have in mind here; do you have any further thoughts on this and/or more breadcrumbs that will lead to work in this area that would be acceptable?
(In reply to Edward Hervey from comment #2) > That last solution might just work indeed ... After all if you remove all > streams, it means what's coming is completely unrelated to the previous > program/stream/... > > Will try to figure out something based on that logic. In HLS it seems this is not entirely true - when switching bitrates, it removes all pads but keeps using the same group so that stream-synchroniser won't create glitches at the transition. At least, I think that's why it's doing it.
This is solved by streams/decodebin3. *** This bug has been marked as a duplicate of bug 758960 ***