GNOME Bugzilla – Bug 746032
audiomixer: doesn't handle buffers with repeating pts properly (broken discont detection)
Last modified: 2015-03-12 17:13:00 UTC
Created attachment 299097 [details] Comparison of the output audio stream with and without audiomixer If one sends a stream of buffers with duplicate pts to audiomixer, it happily takes those as sequential buffers and increments the pts on the output buffers. This is not how audio sinks handle buffers with duplicate pts, and causes audio-video to go out of sync. Tim is investigating this; pasting his findings below: ---- gst-launch-1.0 filesrc location=output-gdp-audio.bin ! gdpdepay ! rtpjitterbuffer ! rtpmp4adepay ! decodebin ! audiomixer ! audiorate ! wavenc ! filesink location=with.wav gst-launch-1.0 filesrc location=output-gdp-audio.bin ! gdpdepay ! rtpjitterbuffer ! rtpmp4adepay ! decodebin ! audiorate ! wavenc ! filesink location=without.wav (see attachment for how the two look like in audacity) Basically the code that checks for discontinuities is an echo chamber, it doesn't track divergence from actual position, but just compares each buffer with the next. And then after mixing a buffer it does pad->next_offset = buffer_end_offset; so if you send as input: 0-1 1-2 1-2 1-2 2-3 then it will happily increase the audiomixer output offset every time, and next_offset will be reset on the second/third buffer (so here we lost sync to the output offset and are now systematically behind), and we'll never exceed any discont thresholds.
Created attachment 299098 [details] Sample rtsp audio stream which yields duplicate timestamps with rtpjitterbuffer Usage: gst-launch-1.0 filesrc location=output-gdp-audio.bin ! gdpdepay ! rtpjitterbuffer ! rtpmp4adepay ! decodebin ! ...
(In reply to Nirbheek Chauhan from comment #0) > This is not how audio sinks handle buffers with duplicate > pts, and causes audio-video to go out of sync. Actually the code in audiomixer should be the same as in the audio sinks.
commit 472069a699e48b50bd7e5ecb710f4b496ade6205 Author: Sebastian Dröge <sebastian@centricular.com> Date: Thu Mar 12 17:11:31 2015 +0000 audiomixer: Fix discont detection and buffer alignment code Actually accumulate the sample counter to check the accumulated error between actual timestamps and expected ones instead of just resetting the error back to 0 with every new buffer. Also don't reset discont_time whenever we don't resync. The whole point of discont_time is to remember when we first detected a discont until we actually act on it a bit later if the discont stayed around for discont_wait time. https://bugzilla.gnome.org/show_bug.cgi?id=746032