GNOME Bugzilla – Bug 793457
flvmux: broken output timestamps/segment
Last modified: 2018-03-02 16:11:29 UTC
Probably a consequence of the port to GstAggregator: $ gst-launch-1.0 videotestsrc ! x264enc ! flvmux ! fakesink silent=false -v | grep -e Segment -e chain | head -n 5 GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x7f0088005d20 13 bytes, dts: none, pts: none, duration: none 271 bytes, dts: none, pts: none, duration: none 66 bytes, dts: none, pts: none, duration: none 7177 bytes, dts: 999:59:59.933333334, pts: 1000:00:00.000000000, duration: 0:00:00.033333333
What I'd like from flvmux is: - Outgoing buffer timestamps match incoming running time, so we can insert a flvmux in a running pipeline without having to offset the outgoing segment. - Outgoing bitstream timestamps match outgoing buffer timestamps, rebased to zero to match FLV spec. Potential problems found: - Some of the timestamp fields on flvmux and its pads seem to mix up signed and unsigned handling. Should probably be signed throughout to handle negative times correctly. - flvmux clips an incoming negative running time (DTS problem) instead of passing it through. - Do we want PTS on the outgoing buffers at all? I think flvmux should only concern itself with DTS timestamps (falling back to PTS if missing). - GstAggregator seems to only read PTS for the start-time handling, not DTS. Does this need fixing?
> - Do we want PTS on the outgoing buffers at all? I think flvmux should only > concern itself with DTS timestamps (falling back to PTS if missing). To be clear, I think the only thing we do need incoming PTS for is to write the composition time offset for AVC video frames. As far as the buffer timestamps go I think we should only look at GST_BUFFER_DTS_OR_PTS and only write GST_BUFFER_DTS.
Olivier, are you going to work on this or someone else? Otherwise we should revert the port to aggregator for 1.14 for the time being.
I'm planning to fix this in the short term.
Leaving it for now for 1.13.2, but will revert next week if we don't have a solution by then.
I fixed the most pressing concern with the patch below. It now produces no DTS at all and a PTS that is always increasing but starting from 0. This matches the behaviour of mpegtsmux. The only other muxer we have which can do h.264 and be streamable is mastroska and it's behaviour is strange, it seems to pass through the DTS vs PTS information, but the result is a mix of buffers with both pts & dts set and some with dts=none but pts set.. I'm not convinced it's better. It also means the current code drops all information about negative DTSes. commit c0bf793c05cf793aa18a8548cda702625e388115 Author: Olivier Crête <olivier.crete@collabora.com> Date: Thu Mar 1 17:15:02 2018 -0500 flvmux: Set PTS based on running time https://bugzilla.gnome.org/show_bug.cgi?id=793457
Also fixed the unit tests, the only thing missing is not killing negative DTS, this was like that before so it's not really a blocker. commit 96261ce220bebb12267dc8dd0aef53b2c6013077 Author: Olivier Crête <olivier.crete@collabora.com> Date: Thu Mar 1 18:24:33 2018 -0500 flvmux: Duration & unit tests The muxed buffers will not carry the duration of the incoming buffers. https://bugzilla.gnome.org/show_bug.cgi?id=793457
Cool, let's close this then. The lack of a timestamp on the first two non-header buffers is owing to the negative DTS then? The 1.12 flvmux outputs this on the first two data buffers: 7177 bytes, dts: none, pts: 0:00:00.000000000, duration: 0:00:00.033333333 5368 bytes, dts: none, pts: 0:00:00.133333333, duration: 0:00:00.033333333 That's what you meant on irc needs further discussing? Let's open a new bug for that then I guess so we can discuss what to do.