GNOME Bugzilla – Bug 797084
"gaps in prefill mode" appear with splitmuxsink 29.97 FPS
Last modified: 2018-09-19 12:32:08 UTC
Created attachment 373550 [details] qtmux:6,splitmuxsink:6 log Pipeline: gst-launch-1.0 -v videotestsrc ! video/x-raw,width=1280,height=720,framerate=30000/1001 ! queue ! videoconvert ! avenc_prores ! queue ! splitmuxsink name=m muxer="qtmux reserved-prefill=true reserved-max-duration=4000000000" max-size-bytes=0 max-size-time=4000000000 location=fragment%05d.mov audiotestsrc ! queue ! audio/x-raw,rate=48000 ! audiobuffersplit output-buffer-duration=1001/30000 strict-buffer-size=true ! queue ! m.audio_%u When finalising the second fragment, it complains that "can't support gaps in prefill mode". Looking at the qtmux logs, the audio pad is a late stream by 0:00:00.033366667 (the duration of one audio buffer). Looks like the ends of the video and the audio streams are not aligned properly in splitmuxsink, and the first fragment ends up containing one extra audio buffer. Attaching qtmux:6,splitmuxsink:6 log.
The timestamps are slightly confusing, because avenc_prores adds 0.166666666 to the outgoing DTS/PTS, so values between the video and audio stream don't match up directly. Timestamp logs: /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105628 bytes, dts: 0:00:03.970466666, pts: 0:00:03.970466666, duration: 0:00:00.033366666, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d54026790 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105520 bytes, dts: 0:00:04.003833332, pts: 0:00:04.003833332, duration: 0:00:00.033366667, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d54013910 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105662 bytes, dts: 0:00:04.037199999, pts: 0:00:04.037199999, duration: 0:00:00.033366667, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d54013800 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105530 bytes, dts: 0:00:04.070566666, pts: 0:00:04.070566666, duration: 0:00:00.033366666, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7fa34c016c30 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105620 bytes, dts: 0:00:04.103933332, pts: 0:00:04.103933332, duration: 0:00:00.033366667, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7fa34800d080 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105592 bytes, dts: 0:00:04.137299999, pts: 0:00:04.137299999, duration: 0:00:00.033366667, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7fa34802d280 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105543 bytes, dts: 0:00:04.170666666, pts: 0:00:04.170666666, duration: 0:00:00.033366666, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7fa34801c2e0 /GstPipeline:pipeline0/GstFakeSink:video: last-message = chain ******* (video:sink) (105586 bytes, dts: 0:00:04.204033332, pts: 0:00:04.204033332, duration: 0:00:00.033366667, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7fa34802e3c0 /GstPipeline:pipeline0/GstFakeSink:audio: last-message = chain ******* (audio:sink) (3202 bytes, dts: none, pts: 0:00:03.903895833, duration: 0:00:00.033354167, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f1560014c10 /GstPipeline:pipeline0/GstFakeSink:audio: last-message = chain ******* (audio:sink) (3204 bytes, dts: none, pts: 0:00:03.937250000, duration: 0:00:00.033375000, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f1560010e20 /GstPipeline:pipeline0/GstFakeSink:audio: last-message = chain ******* (audio:sink) (3204 bytes, dts: none, pts: 0:00:03.970625000, duration: 0:00:00.033375000, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d54009ce0 /GstPipeline:pipeline0/GstFakeSink:audio: last-message = chain ******* (audio:sink) (3202 bytes, dts: none, pts: 0:00:04.004000000, duration: 0:00:00.033354166, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d540098a0 /GstPipeline:pipeline0/GstFakeSink:audio: last-message = chain ******* (audio:sink) (3204 bytes, dts: none, pts: 0:00:04.037354166, duration: 0:00:00.033375000, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d5400d040 /GstPipeline:pipeline0/GstFakeSink:audio: last-message = chain ******* (audio:sink) (3202 bytes, dts: none, pts: 0:00:04.070729166, duration: 0:00:00.033354167, offset: -1, offset_end: -1, flags: 00000000 , meta: none) 0x7f7d540131a0 In the audio stream, the first buffer in the new fragment should be 04.004000000, which corresponds with PTS 4.170666666 on the video (4.170666666 - 0.166666666 = 4.004), and at first glance it seems that's what splitmuxsink is trying to do. I'm not sure where the error happens.
It looks like the problem is that the video stream in 2nd fragment starts with a buffer that should be in the previous fragment: 0:00:04.292081505 19301 0x55f0369734a0 LOG qtmux gstqtmux.c:4798:gst_qt_pad_adjust_buffer_dts:<qtmux0> selected pad video_0 with PTS 0:00:03.970633333 and DTS +0:00:03.970633333
Created attachment 373604 [details] [review] splitmuxsink: Pass the TS for starting a new file explicitly. When sending a start new fragment command, include the output TS at which the new file should start, which ensures the last sample is sent to the previous file, and not the next when GOPs are 1 buffer. Add a test case for splitmux handling single-GOP-per-buffer input streams correctly.
(In reply to Vivia Nikolaidou from comment #0) > Pipeline: > gst-launch-1.0 -v videotestsrc ! > video/x-raw,width=1280,height=720,framerate=30000/1001 ! queue ! > videoconvert ! avenc_prores ! queue ! splitmuxsink name=m muxer="qtmux > reserved-prefill=true reserved-max-duration=4000000000" max-size-bytes=0 > max-size-time=4000000000 location=fragment%05d.mov audiotestsrc ! queue ! > audio/x-raw,rate=48000 ! audiobuffersplit output-buffer-duration=1001/30000 > strict-buffer-size=true ! queue ! m.audio_%u This still fails after the patch attached :(
The failure was because of https://bugzilla.gnome.org/show_bug.cgi?id=797133 .
Upon reflection, I'm not sure this patch does anything. I spent a few hours on it, but seem to have gotten confused somewhere in my testing. I think I should not push it.
Then let's close this one :)