After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 549254 - [playbin/decodebin] Doesn't handle pads that are added much later than the other(s) correctly
[playbin/decodebin] Doesn't handle pads that are added much later than the ot...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other All
: Normal major
: 0.10.26
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks: 558252
 
 
Reported: 2008-08-25 06:02 UTC by kapil
Modified: 2010-02-09 10:11 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
flv with 5 sec delay audio (850.01 KB, application/x-flash-video)
2008-08-25 15:35 UTC, kapil
Details
with 4.9 sec audio delay (834.30 KB, application/x-flash-video)
2008-08-25 15:37 UTC, kapil
Details
Gst Debug Log for 5 sec delay audio stream (9.85 KB, text/plain)
2008-08-25 15:38 UTC, kapil
Details

Description kapil 2008-08-25 06:02:43 UTC
Please describe the problem:
If there is a delay of >= 3secs in the audio stream of an flv file playbin hangs, while USE_DECODBIN=1 hangs when audio stream delay >= 5secs. It seems there is a problem due to some kind of underrun not been handled properly, probably in decodebin's. This only seems to be a problem if there is mp3 involve as mad consumes lots of data before actual decoding ?

Steps to reproduce:
1. gst-launch-0.10 videotestsrc ! ffenc_flv ! queue ! ffmux_flv name=m ! filesink location=test.flv audiotestsrc timestamp-offset=3000000000 ! audioconvert ! lame ! mp3parse ! queue ! m.
2. gst-launch-0.10 playbin uri=file:///home/kaps/test_streams/test.flv
3. USE_DECODEBIN2=1 gst-launch-0.10 playbin uri=file:///home/kaps/test_streams/test.flv 




Actual results:
1. Generates the test file.
2. Hanges after sometime
3. Doesnt preroll at all.

Expected results:
should play audio after the specified delay.

Does this happen every time?
yes

Other information:
Comment 1 Edward Hervey 2008-08-25 11:31:34 UTC
FYI, it doesn't hang with playbin2
Comment 2 kapil 2008-08-25 15:35:58 UTC
Created attachment 117347 [details]
flv with 5 sec delay audio
Comment 3 kapil 2008-08-25 15:37:25 UTC
Created attachment 117348 [details]
with 4.9 sec audio delay
Comment 4 kapil 2008-08-25 15:38:22 UTC
Created attachment 117350 [details]
Gst Debug Log for 5 sec delay audio stream
Comment 5 kapil 2008-08-25 15:39:16 UTC
There is a race with playbin2, it works sometimes and sometimes not.

I would like to put the bug in another way. 
If you try to run the attached test_5sec_audio_delay.flv stream, using 
gst-launch-0.10 -v filesrc location=test_5sec_audio_delay.flv ! decodebin2 ! fakesink, 
you will see that the pipeline will hang since its not able to preroll. It seems that there is one empty queue and one full queue (plz refer the attached logs). Since the empty queue condition is not being checked, the full queue waits for more space to happen, but neither its size is increased nor the data is consumed since other queue is still empty. This I cross verified by changing the  max-size-time property of multiqueue from 5 sec to 10 secs in decodebin2.
 
Also you can check test_4.9sec_audio_delay.flv which works properly with above pipeline.


A thought: In case of mp3 the audio queue can easily get empty by consuming all data without generating any sample, and hence not prerolling.
 
Comment 6 Julien MOUTTE 2008-08-25 15:44:51 UTC
This is a race we've seen in other MPEG TS content.

If the video queue is full and has been increased multiple times to make audio preroll and suddenly the audio queue gets empty, multiqueue just does nothing and blocks.

There's a FIXME in the underrun callback of the internal queues which explains that it will be tricky to increase the queue sizes from the src thread. I guess the problem comes exactly from there and the underrun handling needs to be implemented.
Comment 7 Edward Hervey 2008-08-26 14:14:52 UTC
After going over this on IRC, the real issue seems to be with flvdemux not outputting proper 'filler' newsegments before the first buffer.

For the 5s example file, flvdemux should output the following when it encouters the first audio buffer (timestampped 5s) and creates the audio src pad:

* gst_event_new_new_segment(False, 1.0, GST_FORMAT_TIME, 0, GST_CLOCK_TIME_NONE, 0);
 # this is the initial new_segment, similar to what it's outputting right now

* gst_event_new_new_segment(True, 1.0, GST_FORMAT_TIME, 0, 5 * GST_SECOND, 0);
* gst_event_new_new_segment(False, 1.0, GST_FORMAT_TIME, 5 * GST_SECOND, GST_CLOCK_TIME_NONE, 5 * GST_SECOND);
 # These two will close the previous segment, informing downstream multiqueue that 5 seconds have passed.
Comment 8 Sebastian Dröge (slomo) 2008-10-28 15:13:25 UTC
There is actually a playbin or decodebin blocking this fix. If the second pad is added much later than the first the pipeline will stall in any case, no matter what newsegment events are sent (Wim knows details ;) ).

Correct for the above example would probably be (only)
gst_event_new_new_segment(False, 1.0, GST_FORMAT_TIME, 0,
GST_CLOCK_TIME_NONE, 5 * GST_SECOND);


If all pads would be added almost at the same time (+- ~2 seconds) the above would work. Also for gaps one has to send filler newsegment events every second.

As this can't be done here the first thing to fix is decodebin/playbin to handle late-added pads properly. I have a patch locally to fix the segment handling of flvdemux that would work after the decodebin/playbin fix ;)
Comment 9 Sebastian Dröge (slomo) 2009-09-23 15:49:35 UTC
This works with latest GIT now, no idea when exactly it was fixed.