GNOME Bugzilla – Bug 747881
rtpmp4gdepay does not calculate timestamp for RTP packets with multiple Access Units
Last modified: 2015-08-16 13:37:29 UTC
While remuxing from an RTSP stream, matroskamux generates output with choppy audio. It can be reproduced with: $ gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="video/x-h264; audio/mpeg,mpegversion=(int)4" name=d ! h264parse ! matroskamux name=m ! filesink location=oute.mkv d. ! aacparse ! m. CTRL+C after a while $ gst-play-1.0 oute.mkv The output file will play back normally under ffplay and mpv, but not under gst-play or vlc. Remuxing the RTSP stream with ffmpeg results in a file that is playable by ffplay, mpv, gst-play, and vlc. $ ffmpeg -i rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov -vcodec copy -acodec copy outf.mkv
Hello, How about add queue element behind the demux? as below, $ gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="video/x-h264; audio/mpeg,mpegversion=(int)4" name=d ! queue ! h264parse ! matroskamux name=m ! filesink location=oute.mkv d. ! queue ! aacparse ! m.
The same thing happens with or without the queue. You can record only the audio stream as well and the error will still be present: $ gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="audio/mpeg,mpegversion=(int)4" ! aacparse ! matroskamux ! filesink location=oute.mkv On playback we get some discontinuity warning messages: $ GST_DEBUG=audiobasesink:4 gst-play-1.0 -q oute.mkv 0:00:01.469927406 4232 0x7f01e4003de0 WARN audiobasesink gstaudiobasesink.c:1608:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.256083333, resyncing 0:00:03.105018865 4232 0x7f01e4003de0 WARN audiobasesink gstaudiobasesink.c:1608:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.428666666, resyncing 0:00:04.818736537 4232 0x7f01e4003de0 WARN audiobasesink gstaudiobasesink.c:1608:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.173000000, resyncing If we change the discont-wait property in pulsesink, we get more or less time of uninterrupted audio playback, but then there noticeable audio/video desync.
It seems matroskamux is dropping buffers: $ GST_DEBUG=*matroska*:4 gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="audio/mpeg,mpegversion=(int)4" ! aacparse ! matroskamux ! filesink location=oute.mkv [...] 0:00:03.573547607 18922 0x7fa1a80031e0 WARN matroskamux matroska-mux.c:3409:gst_matroska_mux_write_data:<matroskamux0:audio_0> Invalid buffer timestamp; dropping buffer 0:00:03.573800677 18922 0x7fa1a80031e0 WARN matroskamux matroska-mux.c:3409:gst_matroska_mux_write_data:<matroskamux0:audio_0> Invalid buffer timestamp; dropping buffer 0:00:03.574030080 18922 0x7fa1a80031e0 WARN matroskamux matroska-mux.c:3409:gst_matroska_mux_write_data:<matroskamux0:audio_0> Invalid buffer timestamp; dropping buffer [...]
Created attachment 301684 [details] [review] rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs
The issue was rtpmp4gdepay not calculating timestamps for RTP packets with multiple Access Units. With proposed patch, this gst-launch line properly records the stream. $ gst-launch-1.0 -e rtspsrc protocols=tcp location=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov name=src ! queue ! rtph264depay ! h264parse ! matroskamux name=mux ! filesink location=output.mkv src. ! rtpmp4gdepay ! aacparse ! mux.
Comment on attachment 301684 [details] [review] rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs >- /* make sure we don't use the timestamp again for other AUs in this >- * RTP packet. */ >- timestamp = -1; >+ if (rtpmp4gdepay->constantDuration != 0 && i + 1 < num_AU_headers) { >+ /* if we have constantDuration, calculate timestamp for next AU >+ * in this RTP packet. */ >+ timestamp += (rtpmp4gdepay->constantDuration * GST_SECOND) / >+ depayload->clock_rate; >+ } else { >+ /* otherwise, make sure we don't use the timestamp again for other >+ * AUs. */ >+ timestamp = GST_CLOCK_TIME_NONE; >+ } Not sure this && i+1<num check is actually needed? Aren't we going to quit the loop right after that anyway? Otherwise it looks fine to me, and seems to work. Wim?
I agree, the i+1<num check seems useless. I also tested without it and it makes no difference, as expected. Otherwise, the patch looks good to me. Shall we merge it?
commit 23b5a346751df9a4bc4f8e3a4c8196f51ea052db Author: Ramiro Polla <ramiro.polla@collabora.co.uk> Date: Wed Apr 15 22:51:51 2015 +0200 rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs Use constantDuration to calculate the timestamp of non-first AU in the RTP packet. If constantDuration is not present in the MIME parameters, its value must be calculated based on the timing information from two consecutive RTP packets with AU-Index equal to 0. https://bugzilla.gnome.org/show_bug.cgi?id=747881