GNOME Bugzilla – Bug 693993
[subtitleoverlay] deadlock with mp4 file during preroll
Last modified: 2013-08-03 15:22:49 UTC
The following pipeline is stuck in preroll when using subtitleoverlay, the video pad never unblocks: gst-launch-1.0 souphttpsrc location=http://absolut.zogzog.org/share/samples/mp4/Escargot.mp4 ! qtdemux name=dmx \ multiqueue name=mq \ dmx.video_0 ! mq. mq. ! avdec_h264 ! videoconvert ! overlay.video_sink \ dmx.subtitle_0 ! mq. mq. ! overlay.subtitle_sink \ subtitleoverlay name=overlay wait-text=false ! videoconvert ! xvimagesink The same pipeline works when using textoverlay directly: gst-launch-1.0 souphttpsrc location=http://absolut.zogzog.org/share/samples/mp4/Escargot.mp4 ! qtdemux name=dmx \ multiqueue name=mq \ dmx.video_0 ! mq. mq. ! avdec_h264 ! videoconvert ! overlay.video_sink \ dmx.subtitle_0 ! mq. mq. ! overlay.text_sink \ textoverlay name=overlay wait-text=false ! videoconvert ! xvimagesink
Created attachment 237218 [details] [review] Skip disabled tracks in qtdemux This patch fixes this issue by just ignoring the text track which is marked as disabled in the trak atom. It should be anyway since this track is actually not text but chapters.
Pipeline to see how packets are being pushed out: gst-launch-1.0 souphttpsrc location=http://192.168.0.10/Escargot.mp4 ! qtdemux name=d \ d.subtitle_0 ! fakesink silent=false async=false name=textsink \ d.audio_0 ! fakesink silent=false async=false name=audiosink \ d.video_0 ! fakesink silent=false async=false name=videosink Shows this, in push mode: videosink:sink dts: 0:00:24.920000000, duration: 0:00:00.040000000 textsink:sink dts: 0:00:00.000000000, duration: 0:00:25.080000000 audiosink:sink dts: 0:00:24.917333333, duration: 0:00:00.021333333 videosink:sink dts: 0:05:31.960000000, duration: 0:00:00.040000000 textsink:sink dts: 0:00:25.080000000, duration: 0:05:07.040000000 videosink:sink dts: 0:05:32.000000000, duration: 0:00:00.040000000 but this in pull mode (with queue2 ring-buffer-max-size=500000 before qtdemux): textsink:sink dts: 0:00:00.000000000, duration: 0:00:25.080000000 audiosink:sink dts: 0:00:00.021333333, duration: 0:00:00.021333333 videosink:sink dts: 0:00:00.120000000, duration: 0:00:00.040000000 videosink:sink dts: 0:00:25.080000000, duration: 0:00:00.040000000 textsink:sink dts: 0:00:25.080000000, duration: 0:05:07.040000000 audiosink:sink dts: 0:00:25.002666666, duration: 0:00:00.021333334 This might be due to the way the two text samples are packed: chunk offset (stco) total_entries 2 offset 0 6994308 (6ab984) offset 1 31545768 (1e159a8) So it seems that in push mode the first packet of the text stream is only pushed out after the other streams have seen already ~25 seconds worth of the stream. That probably doesn't play too well with $something, though I would've expected multiqueue to just ignore this stream then once the other streams are filled up enough. Which seems to be what happens: gst-launch-1.0 -v souphttpsrc location=http://192.168.0.10/Escargot.mp4 ! qtdemux name=d \ multiqueue name=mq \ d.subtitle_0 ! mq.sink_0 mq.src_0 ! fakesink silent=false async=false name=textsink \ d.audio_0 ! mq.sink_1 mq.src_1 ! fakesink silent=false name=audiosink \ d.video_0 ! mq.sink_2 mq.src_2 ! fakesink silent=false name=videosink so the issue must be somewhere else, maybe in subtitleoverlay as you say.
I looked at the file and it seems that subtitle overlay waits for the next buffer in order to render the first one. This behavior does make sense if subtitle buffers can overlap. Can they though? If subtitle buffers could overlap, the renderer would need to wait for the next buffer to be sure that there are no other subtitles that need to be rendered with current buffer. If subtitle buffers can't overlap, the renderer can just render current buffer for the entire duration and does not need to wait for the next one. Not being able to overlap subtitle buffers can be limiting though. If subtitles can overlap, then perhaps the demuxer should be nice and send a tiny gap event after each subtitle sample so that the rest of the pipeline will know it doesn't need to wait for the duration of the sample.
subparse does not support overlapping subtitles. I have a patch for assrender to support this here: https://bugzilla.gnome.org/show_bug.cgi?id=625113, which lets libass handle queuing the subtitles and render them. I don't think it would be as straightforward to do that with subparse though.
Support for this probably also needs to be added in the pango elements
And we should add support for overlapping subtitles more widely, assrender for example already handles that.
(In reply to comment #6) > And we should add support for overlapping subtitles more widely, assrender for > example already handles that. No it does not. I've posted a patch and test files in bug #625113 for that.
(In reply to comment #7) > (In reply to comment #6) > > And we should add support for overlapping subtitles more widely, assrender for > > example already handles that. > > No it does not. I've posted a patch and test files in bug #625113 for that. Oh right, that's not done yet :) But libass can do it
This is fixed in 1.1.3 now that qtdemux sends gap events in push mode