GNOME Bugzilla – Bug 549254
[playbin/decodebin] Doesn't handle pads that are added much later than the other(s) correctly
Last modified: 2010-02-09 10:11:38 UTC
Please describe the problem: If there is a delay of >= 3secs in the audio stream of an flv file playbin hangs, while USE_DECODBIN=1 hangs when audio stream delay >= 5secs. It seems there is a problem due to some kind of underrun not been handled properly, probably in decodebin's. This only seems to be a problem if there is mp3 involve as mad consumes lots of data before actual decoding ? Steps to reproduce: 1. gst-launch-0.10 videotestsrc ! ffenc_flv ! queue ! ffmux_flv name=m ! filesink location=test.flv audiotestsrc timestamp-offset=3000000000 ! audioconvert ! lame ! mp3parse ! queue ! m. 2. gst-launch-0.10 playbin uri=file:///home/kaps/test_streams/test.flv 3. USE_DECODEBIN2=1 gst-launch-0.10 playbin uri=file:///home/kaps/test_streams/test.flv Actual results: 1. Generates the test file. 2. Hanges after sometime 3. Doesnt preroll at all. Expected results: should play audio after the specified delay. Does this happen every time? yes Other information:
FYI, it doesn't hang with playbin2
Created attachment 117347 [details] flv with 5 sec delay audio
Created attachment 117348 [details] with 4.9 sec audio delay
Created attachment 117350 [details] Gst Debug Log for 5 sec delay audio stream
There is a race with playbin2, it works sometimes and sometimes not. I would like to put the bug in another way. If you try to run the attached test_5sec_audio_delay.flv stream, using gst-launch-0.10 -v filesrc location=test_5sec_audio_delay.flv ! decodebin2 ! fakesink, you will see that the pipeline will hang since its not able to preroll. It seems that there is one empty queue and one full queue (plz refer the attached logs). Since the empty queue condition is not being checked, the full queue waits for more space to happen, but neither its size is increased nor the data is consumed since other queue is still empty. This I cross verified by changing the max-size-time property of multiqueue from 5 sec to 10 secs in decodebin2. Also you can check test_4.9sec_audio_delay.flv which works properly with above pipeline. A thought: In case of mp3 the audio queue can easily get empty by consuming all data without generating any sample, and hence not prerolling.
This is a race we've seen in other MPEG TS content. If the video queue is full and has been increased multiple times to make audio preroll and suddenly the audio queue gets empty, multiqueue just does nothing and blocks. There's a FIXME in the underrun callback of the internal queues which explains that it will be tricky to increase the queue sizes from the src thread. I guess the problem comes exactly from there and the underrun handling needs to be implemented.
After going over this on IRC, the real issue seems to be with flvdemux not outputting proper 'filler' newsegments before the first buffer. For the 5s example file, flvdemux should output the following when it encouters the first audio buffer (timestampped 5s) and creates the audio src pad: * gst_event_new_new_segment(False, 1.0, GST_FORMAT_TIME, 0, GST_CLOCK_TIME_NONE, 0); # this is the initial new_segment, similar to what it's outputting right now * gst_event_new_new_segment(True, 1.0, GST_FORMAT_TIME, 0, 5 * GST_SECOND, 0); * gst_event_new_new_segment(False, 1.0, GST_FORMAT_TIME, 5 * GST_SECOND, GST_CLOCK_TIME_NONE, 5 * GST_SECOND); # These two will close the previous segment, informing downstream multiqueue that 5 seconds have passed.
There is actually a playbin or decodebin blocking this fix. If the second pad is added much later than the first the pipeline will stall in any case, no matter what newsegment events are sent (Wim knows details ;) ). Correct for the above example would probably be (only) gst_event_new_new_segment(False, 1.0, GST_FORMAT_TIME, 0, GST_CLOCK_TIME_NONE, 5 * GST_SECOND); If all pads would be added almost at the same time (+- ~2 seconds) the above would work. Also for gaps one has to send filler newsegment events every second. As this can't be done here the first thing to fix is decodebin/playbin to handle late-added pads properly. I have a patch locally to fix the segment handling of flvdemux that would work after the decodebin/playbin fix ;)
This works with latest GIT now, no idea when exactly it was fixed.