GNOME Bugzilla – Bug 638288
qtmux: fails to handle out-of-order buffers without duration
Last modified: 2011-04-04 10:08:51 UTC
Created attachment 177199 [details] [review] diff describing my changes to solve the issue After upgrading gst + plugins remuxing flv -> mp4 is now broken and I have traced the issue to the added CTS offset code in flvdemux. The current code correctly calculates PTS using embedded CTS offsets, which is great for playback but not for remuxing. When remuxing I want DTS set on the buffers, not PTS. If buffers are stamped with PTS, they arrive out of order to the downstream muxer if video is AVC/H264 with B-frames. That's not really good when using for example the gst qtmux element. I solved this by adding a new property on flvdemux and I wanted to share my solution. What I want to do: gst-launch filesrc location=in.flv ! flvdemux ! queue ! video/x-h264,width=640,height=360 ! qtmux ! filesink location=out.mp4 What I get: ERROR: from element /GstPipeline:pipeline0/GstQTMux:qtmux0: Could not multiplex stream. Additional debug info: gstqtmux.c(2186): gst_qt_mux_add_buffer (): /GstPipeline:pipeline0/GstQTMux:qtmux0: Received buffer without timestamp/duration. Problem is illustrated by (fakesink output reformatted for clarity): gst-launch -v filesrc location=in.flv ! flvdemux ! queue ! video/x-h264,width=640,height=360 ! fakesink num-buffers=5 chain timestamp: 0:00:00.040000000, duration: none chain timestamp: 0:00:00.120000000, duration: none chain timestamp: 0:00:00.080000000, duration: none chain timestamp: 0:00:00.200000000, duration: none chain timestamp: 0:00:00.160000000, duration: none Note the out of order timestamps due to B-frames. With my changes: gst-launch -v filesrc location=in.flv ! flvdemux avc-cts=false ! queue ! video/x-h264,width=640,height=360 ! fakesink num-buffers=10 chain timestamp: 0:00:00.000000000, duration: none chain timestamp: 0:00:00.040000000, duration: none chain timestamp: 0:00:00.080000000, duration: none chain timestamp: 0:00:00.120000000, duration: none chain timestamp: 0:00:00.160000000, duration: none Timestamps are now monotonically increasing and I can go on remuxing my file: gst-launch filesrc location=in.flv ! flvdemux avc-cts=false ! queue ! video/x-h264,width=640,height=360 ! qtmux ! filesink location=out.mp4 out.mp4 plays back fine on all players I have tested, including gst. avc-cts defaults to TRUE so this change will not break playback of AVC videos but enables easy remuxing if set to false.
Created attachment 177200 [details] Example of flv with AVC and B-frames
The usual DTS/PTS problem again, though I am not quite sure where this would currently pose a concrete problem other than in qtmux. That is, this feels like trying to fix something in qtmux by hacking a bit in flvdemux. FWIW, I was shortly planning/needing to modify/fix some timestamp heuristics in qtmux (again) and might as well keep this in mind also.
Since it is supposed to be normal (i.e. according to spec) for buffers to pass around being decorated with PTS (though possibly inconvenient), moving this to qtmux as the core/source of the problem (though hard to really blame it).
And following commit should handle this case when using the "reorder" dts-method (assuming qtmux is actually/really the only place where this is a problem, feel free to note otherwise). commit 38c93429784d10ddd4da0dbe390616b7e9196729 Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> Date: Mon Jan 3 17:24:23 2011 +0100 qtmux: adjust nasty case timestamp tracking That is, all sorts of problems arise with re-ordered input timestamps that tend to defy automagic handling for every case, so allow for a few variations that can be tried depending on circumstances. Also try to document accordingly. Also fixes #638288.
I can confirm that this solves my problem in a much more elegant way than my solution. There is no point in arguing with the gst masters on what problems should be solved in which component. You guys know the code way better than me. I would however like to +1 on the inline comment 0.11 Phew, can we (pretty) please please sort out DTS/PTS on buffers ... Thanks again!
*** Bug 646012 has been marked as a duplicate of this bug. ***