GNOME Bugzilla – Bug 549774
audio / video synchronization glitch during rtsp playback
Last modified: 2008-09-25 11:10:34 UTC
Please describe the problem: I have a 2 mbit/sec video clip being streamed as rtsp over localhost from Darwin Streaming Server. My command line is: gst-launch -v playbin uri=rtsp://localhost/filename.mp4 A few seconds into the clip, the audio drops out for just under a second, and gst-launch prints this: WARNING: from element /playbin0/abin/audiosink/audiosink-actual-sink-alsa: Compensating for audio synchronisation problems Additional debug info: gstbaseaudiosink.c(1190): gst_base_audio_sink_render (): /playbin0/abin/audiosink/audiosink-actual-sink-alsa: Unexpected discontinuity in audio timestamps of more than half a second (0:00:00.811836734), resyncing From that point on, audio and video remain out of sync by just under a second. VLC and mplayer handle this stream with no such problem. Gstreamer plays the video clip directly from the filesystem with no such problem. The glitch only shows up when using gstreamer to stream it as rtsp://localhost. I'm guessing gst-plugins-base is causing this because I think it contains audiosink-actual-sink-alsa, which seems to be emitting the warning message. The video clip in question is encoded using h.264 video and aac audio. I'm running xubuntu hardy, which includes gstreamer-plugins-base 0.10.18. Steps to reproduce: Actual results: Expected results: Does this happen every time? Other information:
This happens almost every time. A similar audio dropout followed by loss of a/v sync happens with another (similar) video clip, but in its case, the dropout and sync shift are less than half a second, and no warning message is printed. Perhaps audiosink-actual-sink-alsa is merely reacting to the problem, not causing it? Unfortunately, I don't have rights to redistribute the video clips.
Quicktime player 7.3.1 on Windows XP plays the stream (over a LAN) just fine as well.
It could be because of the RTCP lip-sync code. Maybe the server is sending something weird and other players ignore it? Can you attach a gzipped log like: GST_DEBUG=*rtp*:5 gst-launch -v playbin uri=rtsp://... >debug.log 2>&1 so that this can be verified?
Created attachment 117594 [details] GST_DEBUG=*rtp*:5 Attaching the output from this command: GST_DEBUG_NO_COLOR=1 GST_DEBUG=*rtp*:5 gst-launch -v playbin uri=rtsp://localhost/filename.mp4 >debug.log 2>&1 The clip is served by Darwin Streaming Server 5.5.5, which as far as I know doesn't have any RTCP timestamp bugs that would cause this.
The SR packets from the server tell the client to apply a 800ms offset. What versions are you using of gstreamer, gst-plugins-base. gst-plugins-good, gst-plugins-bad?
Created attachment 117975 [details] relevant packets captured before and after the problem showed up (In reply to comment #5) > The SR packets from the server tell the client to apply a 800ms offset. No, I don't believe that's true. I just did a packet capture from a session that exhibited the problem, and calculated the NTP wallclock time of an audio and video RTP packet that arrived within one millisecond of each other. I used the most recently received RTCP packets to perform this calculation. The wallclock times of both packets came out almost exactly the same. I am attaching the relevant packets from my packet capture. I stripped out all RTP packets except the very first (before the sync problem) and nearly last (after the sync problem), due to file size and content redistribution concerns. > What versions are you using of gstreamer, gst-plugins-base. gst-plugins-good, > gst-plugins-bad? dpkg says: gstreamer-tools 0.10.18-4ubuntu1 gstreamer0.10-plugins-bad 0.10.6-5 gstreamer0.10-plugins-base 0.10.18-3 gstreamer0.10-plugins-good 0.10.7-3ubuntu0.1 gstreamer0.10-tools 0.10.18-4ubuntu1 gstreamer0.10-x 0.10.18-3
I see the problem now, the first packet is received very quickly, the second packet after +-800ms. The second packet causes a resync in the jitterbuffer because the gap is too big, this probably does not get propagated correctly to the sync-code, I'll have to investigate a little more.
Created attachment 118037 [details] packets captured during the initial second (In reply to comment #7) > I see the problem now, the first packet is received very quickly, the second > packet after +-800ms. Yes, my packet capture does look like that on the audio track, and similar on the video track. > The second packet causes a resync in the jitterbuffer > because the gap is too big, this probably does not get propagated correctly to > the sync-code, I'll have to investigate a little more. I see. So because of the jitter buffer and/or the sync code, gstreamer gets fooled by the gap between packets, and stops following the SR synchronization information? I'm attaching another packet capture excerpt, this time covering the initial second of playback.
(In reply to comment #8) > Yes, my packet capture does look like that on the audio track, and similar on > the video track. Oops: upon looking again, I think I misspoke about the video track. Only the audio track seems to have an unusual gap between the first and second RTP packet arrival times.
yes, that's the issue. I'm now rewriting the sync code a bit, hopefully I can commit something later today that should fix the issue (and others). I have the darwin streaming server installed. Can you make the file that you are trying to stream available somewhere so that I can reproduce this particular issue?
I reworked the code a bit to better handle this case, had to move some things around resulting in this commit: If you can test cvs, please report, if you can provide the problematic clip, I'll test for you. * gst/rtpmanager/gstrtpbin.c: (on_sender_timeout), (create_session), (gst_rtp_bin_associate), (gst_rtp_bin_sync_chain), (gst_rtp_bin_class_init), (gst_rtp_bin_request_new_pad): * gst/rtpmanager/gstrtpbin.h: Add signal to notify listeners when a sender becomes a receiver. Tweak lip-sync code, don't store our own copy of the ts-offset of the jitterbuffer, don't adjust sync if the change is less than 4msec. Get the RTP timestamp <-> GStreamer timestamp relation directly from the jitterbuffer instead of our inaccurate version from the source. * gst/rtpmanager/gstrtpjitterbuffer.c: (gst_rtp_jitter_buffer_chain), (gst_rtp_jitter_buffer_loop), (gst_rtp_jitter_buffer_get_sync): * gst/rtpmanager/gstrtpjitterbuffer.h: Add G_LIKELY macros, use global defines for max packet reorder and dropouts. Reset the jitterbuffer clock skew detection when packets seqnums are changed unexpectedly. * gst/rtpmanager/gstrtpsession.c: (on_sender_timeout), (gst_rtp_session_class_init), (gst_rtp_session_init): * gst/rtpmanager/gstrtpsession.h: Add sender timeout signal. * gst/rtpmanager/rtpjitterbuffer.c: (rtp_jitter_buffer_reset_skew), (calculate_skew), (rtp_jitter_buffer_insert), (rtp_jitter_buffer_get_sync): * gst/rtpmanager/rtpjitterbuffer.h: Add some G_LIKELY macros. Keep track of the extended RTP timestamp so that we can report the RTP timestamp <-> GStreamer timestamp relation for lip-sync. Remove server timestamp gap detection code, the server can sometimes make a huge gap in timestamps (talk spurts,...) see #549774. Detect timetamp weirdness instead by observing the sender/receiver timestamp relation and resync if it changes more than 1 second. Add method to report about the current rtp <-> gst timestamp relation which is needed for lip-sync. * gst/rtpmanager/rtpsession.c: (rtp_session_class_init), (on_sender_timeout), (check_collision), (rtp_session_process_sr), (session_cleanup): * gst/rtpmanager/rtpsession.h: Add sender timeout signal. Remove inaccurate rtp <-> gst timestamp relation code, the jitterbuffer can now do an accurate reporting about this. * gst/rtpmanager/rtpsource.c: (rtp_source_init), (rtp_source_update_caps), (calculate_jitter), (rtp_source_process_rtp): * gst/rtpmanager/rtpsource.h: Remove inaccurate rtp <-> gst timestamp relation code. * gst/rtpmanager/rtpstats.h: Define global max-reorder and max-dropout constants for use in various subsystems.
I just finished testing a fresh checkout from cvs, and it looks like you fixed this a/v sync problem. Thanks! While testing with gst-launch, I noticed a pop in the audio which consistently shows up about 12.5 seconds from the start of the clip. The pop is not audible when I play the same stream in vlc or mplayer. Do you think it's related to this bug?
vlc and mplayer have different ways of maintaining sync, in gstreamer we drop 20ms or add 20ms of silence when clocks drift too much. This can cause an annoying click but we have nothing better for now.