After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 747881 - rtpmp4gdepay does not calculate timestamp for RTP packets with multiple Access Units
rtpmp4gdepay does not calculate timestamp for RTP packets with multiple Acces...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal normal
: 1.5.90
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2015-04-15 02:57 UTC by Ramiro Polla
Modified: 2015-08-16 13:37 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs (2.58 KB, patch)
2015-04-15 21:08 UTC, Ramiro Polla
committed Details | Review

Description Ramiro Polla 2015-04-15 02:57:13 UTC
While remuxing from an RTSP stream, matroskamux generates output with choppy audio. It can be reproduced with:

$ gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="video/x-h264; audio/mpeg,mpegversion=(int)4" name=d ! h264parse ! matroskamux name=m ! filesink location=oute.mkv d. ! aacparse ! m.

CTRL+C after a while

$ gst-play-1.0 oute.mkv

The output file will play back normally under ffplay and mpv, but not under gst-play or vlc.

Remuxing the RTSP stream with ffmpeg results in a file that is playable by ffplay, mpv, gst-play, and vlc.
$ ffmpeg -i rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov -vcodec copy -acodec copy outf.mkv
Comment 1 Wonchul Lee 2015-04-15 07:18:01 UTC
Hello,

How about add queue element behind the demux?

as below,

$ gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="video/x-h264; audio/mpeg,mpegversion=(int)4" name=d ! queue ! h264parse ! matroskamux name=m ! filesink location=oute.mkv d. ! queue ! aacparse ! m.
Comment 2 Ramiro Polla 2015-04-15 10:02:31 UTC
The same thing happens with or without the queue. You can record only the audio stream as well and the error will still be present:
$ gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="audio/mpeg,mpegversion=(int)4" ! aacparse ! matroskamux ! filesink location=oute.mkv

On playback we get some discontinuity warning messages:
$ GST_DEBUG=audiobasesink:4 gst-play-1.0 -q oute.mkv 
0:00:01.469927406  4232 0x7f01e4003de0 WARN           audiobasesink gstaudiobasesink.c:1608:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.256083333, resyncing
0:00:03.105018865  4232 0x7f01e4003de0 WARN           audiobasesink gstaudiobasesink.c:1608:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.428666666, resyncing
0:00:04.818736537  4232 0x7f01e4003de0 WARN           audiobasesink gstaudiobasesink.c:1608:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.173000000, resyncing

If we change the discont-wait property in pulsesink, we get more or less time of uninterrupted audio playback, but then there noticeable audio/video desync.
Comment 3 Ramiro Polla 2015-04-15 12:28:10 UTC
It seems matroskamux is dropping buffers:
$ GST_DEBUG=*matroska*:4 gst-launch-1.0 -e uridecodebin uri=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov caps="audio/mpeg,mpegversion=(int)4" ! aacparse ! matroskamux ! filesink location=oute.mkv
[...]
0:00:03.573547607 18922 0x7fa1a80031e0 WARN             matroskamux matroska-mux.c:3409:gst_matroska_mux_write_data:<matroskamux0:audio_0> Invalid buffer timestamp; dropping buffer
0:00:03.573800677 18922 0x7fa1a80031e0 WARN             matroskamux matroska-mux.c:3409:gst_matroska_mux_write_data:<matroskamux0:audio_0> Invalid buffer timestamp; dropping buffer
0:00:03.574030080 18922 0x7fa1a80031e0 WARN             matroskamux matroska-mux.c:3409:gst_matroska_mux_write_data:<matroskamux0:audio_0> Invalid buffer timestamp; dropping buffer
[...]
Comment 4 Ramiro Polla 2015-04-15 21:08:10 UTC
Created attachment 301684 [details] [review]
rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs
Comment 5 Ramiro Polla 2015-04-15 21:10:29 UTC
The issue was rtpmp4gdepay not calculating timestamps for RTP packets with multiple Access Units.

With proposed patch, this gst-launch line properly records the stream.
$ gst-launch-1.0 -e rtspsrc protocols=tcp location=rtsp://wowzaec2demo.streamlock.net/vod/mp4:BigBuckBunny_115k.mov name=src ! queue ! rtph264depay ! h264parse ! matroskamux name=mux ! filesink location=output.mkv src. ! rtpmp4gdepay ! aacparse ! mux.
Comment 6 Tim-Philipp Müller 2015-05-26 20:32:04 UTC
Comment on attachment 301684 [details] [review]
rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs

>-          /* make sure we don't use the timestamp again for other AUs in this
>-           * RTP packet. */
>-          timestamp = -1;
>+          if (rtpmp4gdepay->constantDuration != 0 && i + 1 < num_AU_headers) {
>+            /* if we have constantDuration, calculate timestamp for next AU
>+             * in this RTP packet. */
>+            timestamp += (rtpmp4gdepay->constantDuration * GST_SECOND) /
>+                depayload->clock_rate;
>+          } else {
>+            /* otherwise, make sure we don't use the timestamp again for other
>+             * AUs. */
>+            timestamp = GST_CLOCK_TIME_NONE;
>+          }

Not sure this && i+1<num check is actually needed? Aren't we going to quit the loop right after that anyway?

Otherwise it looks fine to me, and seems to work.

Wim?
Comment 7 George Kiagiadakis 2015-08-14 10:21:41 UTC
I agree, the i+1<num check seems useless. I also tested without it and it makes no difference, as expected.

Otherwise, the patch looks good to me. Shall we merge it?
Comment 8 George Kiagiadakis 2015-08-14 10:40:19 UTC
commit 23b5a346751df9a4bc4f8e3a4c8196f51ea052db
Author: Ramiro Polla <ramiro.polla@collabora.co.uk>
Date:   Wed Apr 15 22:51:51 2015 +0200

    rtpmp4gdepay: fix timestamps for RTP packets with multiple AUs

    Use constantDuration to calculate the timestamp of non-first AU in the
    RTP packet.

    If constantDuration is not present in the MIME parameters, its value
    must be calculated based on the timing information from two consecutive
    RTP packets with AU-Index equal to 0.

    https://bugzilla.gnome.org/show_bug.cgi?id=747881