GNOME Bugzilla – Bug 736397
flvdemux: Per-stream resyncing behavior causes A/V-desyncs
Last modified: 2014-11-25 14:49:20 UTC
We have a RTMP setup here where there is usually a large gap (about 4 to 8 seconds) in the video tags soon after connection, while audio tags continue seamlessly. flvdemux decides to collapse the gap in the video stream, causing audio and video to desynchronize. As a workaround I've already patched flvdemux and raised the resync threshold to 30 seconds. A proper solution would be preferred, though.
You mean that the stream has audio from the beginning, but the first video tags come after 4-8 seconds?
No; there's a burst of both video and audio at the beginning, then the video has a gap.
Here's a dump that shows the problem: http://pkgbuild.com/~heftig/input.flv . The FLV looks like this: 0 to 3 seconds: Sequence headers and normal playback 3 to ~4 seconds: Only audio tags At ~4 seconds: Video sequence header and a single frame ~4 to ~5 seconds: Only audio tags At ~5 seconds: Video sequence header ~5 to ~5.6 seconds: Only audio tags ~5.6 to ~7.2 seconds: Nothing ~7.2 seconds: Sequence headers and normal playback The gap in the video tags between ~5 and ~7.2 seconds is collapsed. The gap in the audio tags between ~5.6 and ~7.2 seconds isn't quite large enough to cause a resync. Anyway, the result is an A/V desync of about 2.2 seconds. If the audio gap was collapsed, the result would still be a desync of about 0.7 seconds. (That dump contains material from CC-licensed videos, for details see http://new.make.tv/imprint.56.en.html).
Maybe the code could be changed to not resync on forward skips? As I understand, the issue in bug 631430 was actually about a regression back to 0, not a gap.
I'm not fan of the resync code I've seen. It's design is flawed, so anything that would narrow it's trigger is probably the right way forward.
Created attachment 291007 [details] [review] patch
Review of attachment 291007 [details] [review]: To me this is the way forward, but I'd like Vince to comment on this, as he seem to be the one who had a use case for that RESYNC code.
I think the reasoning for not resyncing does make sense, but it would also make sense if the discont was negative. I'm also not convinced the original file that this fixed was made for was not buggy in the first place. I guess it boils down to "does the flv format allow discontinuities in timestamps". If yes, then resyncing makes more sense, and if not, then keeping timestamps as they are makes more sense.In the latter case, if we still find a discont (so we have a buggy file), then resyncing anyway seems like not a bad option. Rereading the original bug, it looks like the file was originally buggy on the server, then replaced with one with correct timestamps, so it makes sense to apply this new patch.
Review of attachment 291007 [details] [review]: Le's go for it
I've merged it in master, please test, if that's all good, I'd be happy to see this backported to 1.4. commit bf73d834b2c1fed9894fa587b149d4162873b126 Author: Jan Alexander Steffens (heftig) <jsteffens@make.tv> Date: Wed Nov 19 16:12:38 2014 +0100 flvdemux: Restrict resyncing to TS regressions The behavior of resyncing video and audio indepen- dently can cause A/V desyncs. Lets restrict resyncs to jumps backward for now. https://bugzilla.gnome.org/show_bug.cgi?id=736397
Created attachment 291340 [details] [review] backport to 1.4 This is the backport to 1.4 I've been running with. It solves my problem.
Thanks you. commit 39e2e47b5dc794cbbd2b5584895019745b700a00 Author: Jan Alexander Steffens (heftig) <jsteffens@make.tv> Date: Wed Nov 19 16:12:38 2014 +0100 flvdemux: Restrict resyncing to TS regressions The behavior of resyncing video and audio indepen- dently can cause A/V desyncs. Lets restrict resyncs to jumps backward for now. https://bugzilla.gnome.org/show_bug.cgi?id=736397 Conflicts: gst/flv/gstflvdemux.c