GNOME Bugzilla – Bug 741306
qtmux: handle overlapping subtitle buffers properly
Last modified: 2018-11-03 14:56:29 UTC
Overlapping subtitle buffers (buffers that start before previous buffer ends) are legal in many containers (i.e. MKV), but can not be represented in MP4. Trying to remux such file currently results in invalid file, because timestamp delta between buffer end time and next buffer start time is negative In order to handle this well, the muxer should split and combine buffers in a way so that no information is lost. I.e. Two overlapping input buffers should be muxed as three separate buffers, with second buffer combining content from both input buffers. [=======Buffer In 1========] [=======Buffer In 2========] [Buffer Out 1][Buffer Out 2][Buffer Out 3] Sample MKV file with overlapped subtitles https://s3.amazonaws.com/MatejK/Samples/OverlappedSubtitles.mkv Remuxed file without the patch https://s3.amazonaws.com/MatejK/Samples/OverlappedSubtitlesWithoutPatch.mp4 Remuxed file with the patch https://s3.amazonaws.com/MatejK/Samples/OverlappedSubtitlesWithPatch.mp4
Created attachment 292402 [details] [review] Patch to handle overlapping buffers properly
Review of attachment 292402 [details] [review]: Out of curiosity, what is generating those broken subtitle streams? Anyway, I think it would be better to solve this outside of qtmux because this isn't specific to qt/mp4 container and we would need to replicate it on other muxers and also could be used during rendering. So, why not put this on an element by itself?
Uh. What do you mean by broken subtitle streams? Overlapping subtitles are a perfectly valid and common thing. I.e. web vtt specification: http://www.webmproject.org/docs/container/ > The timestamps for WebVTT cues can overlap in time. It's only MP4 (and maybe FLV?) that can't represent this, because there are no timestamps, just durations. I think this restriction is muxer-specific, so I'm not sure if separate element is really the answer here.
flvmux doesn't currently support subtitles though. I'm not aware of any other muxer with such restriction right now. But if there are any I agree separate plugin could make sense.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/146.