GNOME Bugzilla – Bug 614612
[API] overlay: need generic overlay base class for textoverlay, assrender, tiger, xsub, dvdspu etc.
Last modified: 2018-11-03 11:16:35 UTC
When encoding videos from MKV to MP4 for my Blackberry using gst-launch, I ended up with trouble regarding subtitle rendering: - textoverlay can word-wrap, but looks ugly (no anti aliasing) - cairotextoverlay looks good, but cannot word wrap, and, which is the focus of this report, stalls the pipeline whatever I try. An example video (released online by the Chaos Computer Club Cologne, subtitles by me) can be found at http://rm.endoftheinternet.org/~nexuiz/gst-cairotextoverlay-hang/ The includes shell script, if called with the argument "fail", performs: gst-launch \ filesrc location="17.mkv" ! matroskademux name=demuxer \ demuxer.subtitle_00 ! queue ! sub. \ demuxer.video_00 ! queue ! decodebin ! queue ! cairotextoverlay name=sub ! queue ! xvimagesink sync=false The pipeline will hang (i.e. playback wills top) at the first line of subtitles (in what I ACTUALLY want to encode, I get that hang only after about 20 seconds of input). If cairotextoverlay is replaced by textoverlay, the pipeline does NOT stall and playback continues up to the end. If the filesrc is doubled, it seems to not stall - the video plays indeed back to the end - but then gst-launch does not exit. The "last words" with added -v -m are: Got message #4883 from element "pipeline0" (eos): no message details Got EOS from element "pipeline0". Execution ended after 72374852699 ns. Setting pipeline to PAUSED ... Setting pipeline to READY ... /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstQueue:queue3.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstQueue:queue3.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstCairoTextOverlay:sub.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstCairoTextOverlay:sub.GstPad:text_sink: caps = NULL /GstPipeline:pipeline0/GstCairoTextOverlay:sub.GstPad:video_sink: caps = NULL /GstPipeline:pipeline0/GstQueue:queue0.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstQueue:queue0.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstQueue:queue2.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstQueue:queue2.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstMatroskaDemux:demuxer2.GstPad:subtitle_00: caps = NULL /GstPipeline:pipeline0/GstMatroskaDemux:demuxer2.GstPad:audio_00: caps = NULL /GstPipeline:pipeline0/GstMatroskaDemux:demuxer2.GstPad:video_00: caps = NULL /GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:src0: caps = NULL /GstPipeline:pipeline0/GstDecodeBin:decodebin0/ffdec_h264:ffdec_h2640.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstDecodeBin:decodebin0/ffdec_h264:ffdec_h2640.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstDecodeBin:decodebin0.GstGhostPad:sink: caps = NULL /GstPipeline:pipeline0/GstQueue:queue1.GstPad:src: caps = NULL /GstPipeline:pipeline0/GstQueue:queue1.GstPad:sink: caps = NULL /GstPipeline:pipeline0/GstMatroskaDemux:demuxer.GstPad:subtitle_00: caps = NULL /GstPipeline:pipeline0/GstMatroskaDemux:demuxer.GstPad:audio_00: caps = NULL /GstPipeline:pipeline0/GstMatroskaDemux:demuxer.GstPad:video_00: caps = NULL Setting pipeline to NULL ... Freeing pipeline ... gst-launch \ filesrc location="17.mkv" ! matroskademux name=demuxer \ filesrc location="17.mkv" ! matroskademux name=demuxer2 \ demuxer2.subtitle_00 ! queue ! sub. \ demuxer.video_00 ! queue ! decodebin ! queue ! cairotextoverlay name=sub ! queue ! xvimagesink sync=false Any idea whether this is a) a bug in cairotextoverlay, or b) in my command line, and in case of b), how I can fix it?
Created attachment 177305 [details] [review] collectpads: add the ability to mark pads as sparse This will allow those pads to not wait for data to become available before calling the callback, and thus not block the pipeline if none is received for a while.
Created attachment 177306 [details] [review] cairotextoverlay: mark text pad as sparse This is so we don't stall when there's no text input for a while.
cairotextoverlay uses GstCollectPads, which waits for data on all its inputs before generating input. After the first subtitle, there's a large pause with no text input, and textoverlay ends up stalling when the buffering fills up. So I've added a way to mark sparse pads so GstCollectPads will not try to wait for input on those, and marked the text pad in cairotextoverlay.
Note there is already long since a GstCollectPads2 in https://bugzilla.gnome.org/show_bug.cgi?id=415754 that deals (a.o.) with these sparse issues (a.o. intended for subtitle muxing) [that may recently have seen some more work in mixer context]. There is something to be said for getting that one finally sorted out ...
All changes I did to collectpads2 are in gst-plugins-good/gst/videomixer for videomixer2. IIRC I didn't change much
For the record, my patch is wrong: it doesn't preserve ABI compatibility, there is no spare space in the structures to store the necessary data to track which streams are sparse.
Comment on attachment 177305 [details] [review] collectpads: add the ability to mark pads as sparse no space available to keep ABI compatibility.
Comment on attachment 177306 [details] [review] cairotextoverlay: mark text pad as sparse no space available to keep ABI compatibility.
The best solution would be to create a base class from the pango plugin elements and use it for pango and cairo.
Well, we have the base class now, but it's internal to the pango plugin. I'm not sure if we need to port cairotextoverlay at all seeing that the pango one uses pangocairo now - do we? Can we close this bug?
Sebastian, Vincent, can we close this bug ?
No, it would still be very useful. See all the code duplication between textoverlay, assrender, tiger, xsub, dvdspu and others.
Would help with bug 531166
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/32.