GNOME Bugzilla – Bug 640023
Jitterbuffer: does not put the same gst timestamp on packets with the same RTP timestamp
Last modified: 2013-02-06 16:20:12 UTC
Currently, the jitterbuffer does the skew computation idependantly for every incoming packet. If a large video frame is split into separate packets, they all have the same RTP timestamp, so they should all have the same GStreamer timestamp. If the decoder uses the GStreamer timestamp to get the PTS after decoding, it could get confused (and ffdec_h264 does with the patch attached to bug #640012).
Created attachment 178806 [details] [review] rtpjitterbuffer: Only compute skew once if multiple consecutive packets have the same ts
Actually, this bug actually causes serious confusing to the interpolation code in gstffmpegdec even without the patch from bug #640012
Unfortunately, this makes it calculate the clock skew only on the first packet of a fragmented frame. Maybe it's better to feed all timestamps in the skew correction but only use a more recent skew for the timestamps when the rtp time changes. you have to be careful about reordered packets too, so it should probably be done on the output side of things. Maybe the easiest is to store the current skew in the buffer offset or something and then on the output size apply it to the buffer timestamp. The real problem is again in ffmpeg, it uses the timestamp difference between consecutive packets to estimate the duration. It's difficult to do this right.. Also setting access-unit to TRUE on the depayloader (or using caps) should fix this because then each buffer is a complete frame and ffmpeg will do the right thing.
I actually think that feeding subsequent packets to the skew algorithm will give the wrong result. Since the skew algorithm assumes that the RTP timestamp on the packet is related to the time when it was sent. If you have multiple consecutive packets with the same timestamp X, I think its safe to assume that the first one was sent around time X and the subsequent one were sent afterwards. So doing the skew computation from subsequent packets just adds noise..
yes, you are right. I'll have a look at this.
Did you ever see the current algorithm fail? if so, how did it happen?
It's 100% reproducible with the fs2-gui.py example in farsight2. But only if I have two sessions (audio and video), not with a single one. And for some reason it works fine if I do it with gst-launch.
commit c3077012c0914a8d3a103a858d820f4a3ec3c900 Author: Wim Taymans <wim.taymans@collabora.co.uk> Date: Wed Feb 6 17:15:11 2013 +0100 jitterbuffer: do skew estimation only for new timestamps Only run the skew estimation code when we have a new RTP timestamp. If we have the same RTP timestamp, we simply use the previous estimation. This works because the new observation with the same RTP timestamp has to have a bigger receiver time and is thus not going to influence the estimation except for causing more jitter. Fixes https://bugzilla.gnome.org/show_bug.cgi?id=640023