GNOME Bugzilla – Bug 624212
mp4mux produces incorrect frame rates when h264 input uses bframes
Last modified: 2010-08-21 19:15:56 UTC
Environment: gstreamer-plugins-bad-0.10.19-1.fc12.x86_64 gstreamer-plugins-ugly-0.10.15-1.fc12.x86_64 gstreamer-plugins-base-0.10.29-1.fc12.x86_64 x264-libs-0.0.0-0.27.20091109git3daa02e.fc12.x86_64 (newest git versions of -ugly and x264 show the same behavior) Steps to reproduce: Baseline with no bug: 1. gst-launch-0.10 -e -vm audiotestsrc num-buffers=100 is-live=true ! faac ! mp4mux name=mux videotestsrc num-buffers=100 do-timestamp=true is-live=true ! "video/x-raw-yuv, framerate=(fraction)30000/1001" ! ffmpegcolorspace ! x264enc bframes=0 ! mux. mux. ! filesink location=foo2.mp4 2. $ mp4info foo2.mp4 mp4info version 1.5.0.1 foo2.mp4: Track Type Info 1 audio MPEG-4 AAC LC, 2.321 secs, 0 kbps, 44100 Hz 2 video H264 Main@2.1, 3.336 secs, 1121 kbps, 320x240 @ 29.976019 fps Now change bframes=0 to bframes=1. 3. gst-launch-0.10 -e -vm audiotestsrc num-buffers=100 is-live=true ! faac ! mp4mux name=mux videotestsrc num-buffers=100 do-timestamp=true is-live=true ! "video/x-raw-yuv, framerate=(fraction)30000/1001" ! ffmpegcolorspace ! x264enc bframes=1 ! mux. mux. ! filesink location=foo2.mp4 4. $ mp4info foo2.mp4 mp4info version 1.5.0.1 foo2.mp4: Track Type Info 1 audio MPEG-4 AAC LC, 2.321 secs, 0 kbps, 44100 Hz 2 video H264 Main@2.1, 6.573 secs, 563 kbps, 320x240 @ 15.213753 fps Expected Result: I expect the duration and frame rate of the resulting stream to be the same whether bframes are used or not. [Come to think of it, maybe the bug is in x264enc?]
Still an issue with git latest pre-release. > bframes=0: video H264 Main@2.1, 3.336 secs, 1121 kbps, 320x240 @ 29.976019 fps > bframes=1: video H264 Main@2.1, 6.573 secs, 563 kbps, 320x240 @ 15.213753 fps $ gst-launch-0.10 -v videotestsrc num-buffers=5 ! "video/x-raw-yuv,framerate=(fraction)30000/1001" ! x264enc bframes=1 ! fakesink | grep chain timestamp: 0:00:00.000000000, duration: 0:00:00.033366666 timestamp: 0:00:00.066733333, duration: 0:00:00.033366667 timestamp: 0:00:00.033366666, duration: 0:00:00.033366667 timestamp: 0:00:00.133466666, duration: 0:00:00.033366666 timestamp: 0:00:00.100100000, duration: 0:00:00.033366667 $ gst-launch-0.10 -v videotestsrc num-buffers=5 ! "video/x-raw-yuv,framerate=(fraction)30000/1001" ! x264enc bframes=0 ! fakesink | grep chain timestamp: 0:00:00.000000000, duration: 0:00:00.033366666 timestamp: 0:00:00.033366666, duration: 0:00:00.033366667 timestamp: 0:00:00.066733333, duration: 0:00:00.033366667 timestamp: 0:00:00.100100000, duration: 0:00:00.033366666 timestamp: 0:00:00.133466666, duration: 0:00:00.033366667 > I expect the duration and frame rate of the resulting stream to be the same > whether bframes are used or not. Agreed. My guess would be that mp4mux tries to estimate the framerate or durations or something from the timestamps of the first two buffers, that could explain the factor 2. Worse, it's not just some value in the header that's wrong, but the timestamps on demuxing are all off too: gst-launch-0.10 filesrc location=foo-N.mp4 ! qtdemux ! progressreport ! fakesink yields last timestamp of ca. 6 secs vs. 3 secs.
mp4mux does not guess any framerate, nor does it have to, since the qt/iso format has no framerate to record as metadata. It is up to the reader/demuxer to either not care or come up with some estimate based on timestamps (which is typically based on the first few ts, and can as such be off for e.g. a recording with some jitter on first timestamps). That said, indeed looks like qtmux ts handling with pts input ts is not ok. However, note that some heuristic is involved as ts input may be dts or pts in practice (even though in theory it should be pts).
Created attachment 167953 [details] [review] qtmux: autodetect out-of-order timestamps qtmux: autodetect out-of-order input timestamps and determine DTS accordingly Favour using input buffer timestamps for DTS, but fallback to using buffer duration (accumulation) if input ts detected out-of-order.
FWIW, it seems advisable to include this patch into upcoming release, otherwise a stock -ugly and -bad pipeline x264enc ! qtmux will have crappy result, so unless there are any objections ...
commit 0ee6ce901ddec2538783c4d271a02ec38e71ee5f Author: Mark Nauwelaerts <mark.nauwelaerts@collabora.co.uk> Date: Mon Aug 16 12:36:24 2010 +0200 qtmux: autodetect out-of-order input timestamps and determine DTS accordingl Favour using input buffer timestamps for DTS, but fallback to using buffer duration (accumulation) if input ts detected out-of-order. Fixes #624212.