GNOME Bugzilla – Bug 772230
decodebin3-parse: Make it generate pending input streams
Last modified: 2018-11-03 11:50:19 UTC
Created attachment 336615 [details] gstdecodebin3-parse: Make it to generate pending input streams Deal All. When multiple source elements, in such all streams are fed separately, parsebin is generated by number of streams. And it is managed by main and other inputs(eg. DecodebinInput) to configure and connect MQ's input slots. Actually, I have a scenario video and audio stream ES data is feeding separately and 2 of parsebins are generated as well. But, streams are changed in such audio codec is changed in running time, pipeline(Playbiin3) is stopped or pending in sometimes. I reviewed the log with decodebin3 and decodebin3-parsebin. Sometimes, other input(DecodebinInput for video) removed the input-stream(DecodebinInputStream for old audio) and disconnected to MQ's input slot. And then, new input stream(DecodebinInputStream for new audio) is not created and connected to MQ's input slot. At that time, parsebin_buffer_probe for video pad is called early than new audio.
==================================== 0:00:01.991314917 4866 0x6f8026c0 FIXME decodebin3 gstdecodebin3-parse.c:396:parsebin_buffer_probe:<decodebin3-0> Need a lock ! 0:00:01.991353501 4866 0x6f8026c0 LOG decodebin3 gstdecodebin3-parse.c:397:parsebin_buffer_probe:<decodebin3-0> parse input locking from thread 0x6f8026c0 0:00:01.991385126 4866 0x6f8026c0 LOG decodebin3 gstdecodebin3-parse.c:397:parsebin_buffer_probe:<decodebin3-0> parse input locked from thread 0x6f8026c0 0:00:01.991431376 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:398:parsebin_buffer_probe:<parsebin1:src_0> Got a buffer ! UNBLOCK ! 0:00:01.991460542 4866 0x6f8026c0 FIXME decodebin3 gstdecodebin3-parse.c:404:parsebin_buffer_probe:<decodebin3-0> Re-use existing input streams if/when possible 0:00:01.991527292 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:407:parsebin_buffer_probe:<decodebin3-0> Removing unused streams 0:00:01.991595709 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:413:parsebin_buffer_probe:<decodebin3-0> Checking input stream 0x6f93afa8 0:00:01.991703626 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:417:parsebin_buffer_probe:<decodebin3-0> Removing pad block on input 0x6f93afa8 pad <parsebin1:src_0> 0:00:01.991726042 4866 0x6f802000 DEBUG decodebin3 gstdecodebin3-parse.c:225:parse_chain_output_probe:<parsebin0:src_1> Got event eos 0:00:01.991754209 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:413:parsebin_buffer_probe:<decodebin3-0> Checking input stream 0x6f92d870 0:00:01.991772542 4866 0x6f802000 DEBUG decodebin3 gstdecodebin3-parse.c:269:parse_chain_output_probe:<parsebin0:src_1> real input pad, marking as EOS 0:00:01.991809917 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:417:parsebin_buffer_probe:<decodebin3-0> Removing pad block on input 0x6f92d870 pad <parsebin0:src_1> 0:00:01.991855251 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:363:remove_input_stream:<decodebin3-0> Removing input stream 0x6f92d870 (8b32a917d381c3871e1e3b65910f28d1/001) 0:00:01.991847917 4866 0x6f802000 DEBUG decodebin3 gstdecodebin3.c:1510:gst_decodebin3_handle_message:<decodebin3-0> Got Message structure-change 0:00:01.991900751 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3.c:1510:gst_decodebin3_handle_message:<decodebin3-0> Got Message structure-change 0:00:01.991926251 4866 0x6f802000 DEBUG decodebin3 gstdecodebin3.c:1510:gst_decodebin3_handle_message:<decodebin3-0> Got Message structure-change 0:00:01.991943376 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3.c:1510:gst_decodebin3_handle_message:<decodebin3-0> Got Message structure-change 0:00:01.991975876 4866 0x6f802000 DEBUG decodebin3 gstdecodebin3-parse.c:558:parsebin_pad_removed_cb:<parsebin0:src_1> removed 0:00:01.991991417 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3.c:1971:get_slot_for_input:<decodebin3-0> input 0x6f92d870 (stream 0x6f908cf0 8b32a917d381c3871e1e3b65910f28d1/001) 0:00:01.992057584 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3.c:1983:get_slot_for_input:<decodebin3-0> Returning already specified slot 0 0:00:01.992001834 4866 0x6f802000 DEBUG decodebin3 gstdecodebin3-parse.c:568:parsebin_pad_removed_cb:<parsebin0:src_1> stream 0x6f92d870 0:00:01.992102584 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:379:remove_input_stream:<decodebin3-0> slot 0x6f93b4f0 cleared 0:00:01.992136626 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:413:parsebin_buffer_probe:<decodebin3-0> Checking input stream 0x6f93afa8 0:00:01.992176334 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:430:parsebin_buffer_probe:<decodebin3-0> Creating new streams (if needed) 0:00:01.992205042 4866 0x6f8026c0 LOG decodebin3 gstdecodebin3-parse.c:463:parsebin_buffer_probe:<decodebin3-0> parse input unlocking from thread 0x6f8026c0 0:00:01.992264501 4866 0x6f8026c0 LOG decodebin3 gstdecodebin3-parse.c:469:parsebin_buffer_probe:<decodebin3-0> Slot 0 input:(nil) drain_eos:0 0:00:01.992294334 4866 0x6f8026c0 DEBUG decodebin3 gstdecodebin3-parse.c:471:parsebin_buffer_probe:<multiqueue0:sink_0> Sending EOS to unused slot 0:00:01.992439917 4866 0x6f9098c0 DEBUG decodebin3 gstdecodebin3.c:1754:multiqueue_src_probe:<multiqueue0:src_0> Got event 0x6f97cc60 eos 0:00:01.992473917 4866 0x6f9098c0 FIXME decodebin3 gstdecodebin3.c:1854:multiqueue_src_probe:<multiqueue0:src_0> EOS on multiqueue source pad. input:(nil) 0:00:01.992680459 4866 0x6f9098c0 DEBUG decodebin3 gstdecodebin3.c:1858:multiqueue_src_probe:<multiqueue0:src_0> last EOS for input, forwarding and removing slot 0:00:01.992725792 4866 0x6f8026c0 LOG decodebin3 gstdecodebin3-parse.c:469:parsebin_buffer_probe:<decodebin3-0> Slot 1 input:0x6f93afa8 drain_eos:0 ====================================
Created attachment 336616 [details] [review] gstdecodebin3-parse: Make it to generate pending input streams Deal All. Please check the log and review the patch. Thanks
Dear All. I am thinking that latest upstream code is not considered the multiple inputs(eg. multiple parsebins) with changing stream or upstream change(eg. codec change) in running time. I think we would have to make it generate pending input streams even if parsebin_buffer_probe for other side is called early. Please review my patch and share your opinions. Thanks.
Created attachment 336855 [details] [review] gstdecodebin3-parse: Make it generate pending input streams Dear All I updated my patch that a parsebin does not need to be involved other's pending pads and make their input streams and connect MQ's slots when stream is changed in running time. Please check and review my patch. Thanks.
Dear All Let me explain and summarize the issued problem. 1) First Stream (1-audio and 1-video) ------------------------------------------------ * audio Stream '3413b58d3355261af6f2acc9fea088dc/001' caps : audio/x-eac3, framed=(boolean)true, rate=(int)48000, channels=(int)2 It is configured in parsebin0(qtdemux1) and parsebin0:src_0 is exposed. * video Stream '5ce6e6326043ee59a7efaa509b3da693/002' caps : video/x-h264, stream-format=(string)avc, ... It is configured in parsebin1(qtdemux0) and parsebin1:src_1 is exposed. ------------------------------------------------ 2) Second Stream (1-audio and 1-video) : Just audio stream(codec) is changed from eac3 to mpeg. Video is same. ------------------------------------------------ * audio Stream '8b32a917d381c3871e1e3b65910f28d1/001' caps : audio/mpeg, mpegversion=(int)4, ... It is configured in parsebin0(qtdemux1) and parsebin0:src_1 is exposed. ------------------------------------------------ : At that time, old Input Stream is removed and new Input Stream is created. And new Input Stream is connected MQ's slot as well. Because, buffer is probed in DecodebinInput's pending pads(parsebin0:src_1) for audio. 2-1) <parsebin0:src_0> receive EOS event. 2-2) <parsebin0:src_1> receive a buffer. 2-3) <parsebin0:src_0> pad removed. 2-4) Input Stream(3413b58d3355261af6f2acc9fea088dc/001)removed. 2-5) New input Stream(8b32a917d381c3871e1e3b65910f28d1/001) created. 2-6) New input Stream is linked to MQ's slot. 3) Third Stream (1-audio and 1-video) : Just audio stream is changed. Video is same. ------------------------------------------------ * audio Stream '48d9544b186a7530ac2380e02a4bf759/001' caps : audio/mpeg, mpegversion=(int)4, ... It is configured in parsebin0(qtdemux1) and parsebin0:src_2 is exposed. ------------------------------------------------ : At that time, old Input Stream is removed and MQ's Output Stream is removed abnormally by EOS event. Because, buffer is probed in DecodebinInputStream->srcpad(parsebin1:src_0) for video. Then, it removed other's DecodebinInputStream and MQ's Output Stream. 3-1) <parsebin1:src_0> receive a buffer. 3-2) <parsebin0:src_1> receive EOS event. 3-3) Input Stream(8b32a917d381c3871e1e3b65910f28d1/001) removed. 3-4) <parsebin0:src_1> pad removed. 3-5) New input Stream(48d9544b186a7530ac2380e02a4bf759/001) it not created. 3-6) Sends EOS event to MQ's slot and MQ's Output Stream is removed. I think that a DecodebinInput does not involve other's DecodebinInput. Please review my problem and let me know if you have any of questions or opinions. And check my patch is acceptable or not. Thanks.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/296.