After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 766267 - rtpmp4adepay: Should reject caps without codec configuration
rtpmp4adepay: Should reject caps without codec configuration
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2016-05-11 11:24 UTC by Peter Maersk-Moller
Modified: 2018-11-03 15:09 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Peter Maersk-Moller 2016-05-11 11:24:40 UTC
Trying to RTP stream AAC audio fails. The modules aacparse, avdec_aac or decodebin all fails to output packets. Probable cause is something with rtpmp4apay/rtpmp4adepay.

Scripts documenting the problem:

Sender (one of the following):
$ AUDIO='audio/x-raw,rate=48000,channels=2'
$ gst-launch-1.0 -v audiotestsrc is-live=1 ! $AUDIO ! faac ! queue ! rtpmp4apay ! udpsink host=127.0.0.1 port=14002
$ gst-launch-1.0 -v audiotestsrc is-live=1 ! $AUDIO ! avenc_aac  ! queue ! rtpmp4apay ! udpsink host=127.0.0.1 port=14002
$ gst-launch-1.0 -v audiotestsrc is-live=1 ! $AUDIO ! avenc_aac compliance=-2 ! queue ! rtpmp4apay ! udpsink host=127.0.0.1 port=14002

Receiver:
$ gst-launch-1.0 -v udpsrc port=14002 caps="application/x-rtp,media=audio,payload=96,clock-rate=48000,encoding-name=MP4A-LATM" do-timestamp=1 ! rtpmp4adepay ! aacparse ! identity silent=0 ! fakesink

At the receiver end you will see no packets coming from aacparse. You can replace aacparse with avdec_aac or decodebin and get same result.

Regards
Peter
Comment 1 Sebastian Dröge (slomo) 2016-05-11 11:35:00 UTC
Not a blocker as it's not a regression, or is it?
Comment 2 Peter Maersk-Moller 2016-05-11 11:45:35 UTC
(In reply to Sebastian Dröge (slomo) from comment #1)
> Not a blocker as it's not a regression, or is it?

Well, it has worked before. Obviously MPEG-4 audio over RTP has worked before, so yes, it is a regression as it is not working now.

That said, not being able to either stream MPEG-4 audio over RTP or play same or both, can hardly have a severity of "normal".

P
Comment 3 Tim-Philipp Müller 2016-05-11 11:53:38 UTC
RTP streaming of AAC works just fine with gst-rtsp-server and gst-play for what it's worth (using mp4a pay/depay), so it must be something about the combination of elements.

Time spent arguing about the bug severity would probably better be spent investigating the issue.

If the exact same pipeline/script is known to work with a certain older version of GStreamer, that's useful information, then we can compare/bisect.
Comment 4 Sebastian Dröge (slomo) 2016-05-11 12:14:55 UTC
(In reply to Peter Maersk-Moller from comment #2)
> (In reply to Sebastian Dröge (slomo) from comment #1)
> > Not a blocker as it's not a regression, or is it?
> 
> Well, it has worked before. Obviously MPEG-4 audio over RTP has worked
> before, so yes, it is a regression as it is not working now.

My question was mostly: did it work before and with which versions :) If it worked before and is a regression, then it is indeed a blocker and we should get it fixed before the next release.

If OTOH it never worked it would be less important.
Comment 5 Peter Maersk-Moller 2016-05-11 21:06:23 UTC
(In reply to Sebastian Dröge (slomo) from comment #4)
> My question was mostly: did it work before and with which versions :) If it
> worked before and is a regression, then it is indeed a blocker and we should
> get it fixed before the next release.
> 
> If OTOH it never worked it would be less important.

Understood. Seems not to work in 1.6.3, 1.4.5 and 1.3.0, unfortunately :-(
Comment 6 Sebastian Dröge (slomo) 2016-05-11 21:25:16 UTC
Thanks for confirming :) That should be easy to bisect then
Comment 7 Tim-Philipp Müller 2016-05-11 21:44:48 UTC
*not* work... ;)
Comment 8 Sebastian Dröge (slomo) 2016-05-12 05:38:18 UTC
Meh, I can't read :) Sorry
Comment 9 Sebastian Dröge (slomo) 2016-05-12 06:38:42 UTC
Yes, it's a bug in those pipelines. The receiver RTP caps are not complete, they are missing cpresent=(string)0 and config=(string)40002320.
Comment 10 Peter Maersk-Moller 2016-05-12 10:11:43 UTC
Confirming setting caps for the udpsrc for audio to

audio_caps='application/x-rtp,media=audio,payload=96,clock-rate=44100,encoding-name=MP4A-LATM,cpresent=(string)0,config=(string)40002320'

works for streaming AAC using rtp as well as rtpbin for version 1.8.1.

I actually tried this before the initial report, but back then I omitted the '(string)' qualifier for the config parameter. And it that case it failed. First time ever I came across omitting a string qualifier for a parameter made a difference, but I guess that is pure chance.

That said, when using the following receiver script and sender script, the audio pauses for few hundred milliseconds every 2 seconds. Can anybody else confirm this and should I open another bug report?

Thanks.

Receiver pipeline script
  VDEPAY=rtph264depay
  audio_caps='application/x-rtp,media=audio,payload=96,clock-rate=44100,encoding-name=MP4A-LATM,cpresent=(string)0,config=(string)40002320'
  gst-launch-1.0 -v rtpbin name=rtpbin buffer-mode=slave                   \
        udpsrc do-timestamp=1 caps=$video_caps port=$port_video_rtp     !\
          rtpbin.recv_rtp_sink_0                                         \
        udpsrc port=$port_video_rtcp                                    !\
          rtpbin.recv_rtcp_sink_0                                        \
        udpsrc caps=$audio_caps port=$port_audio_rtp                    !\
          rtpbin.recv_rtp_sink_1                                         \
        udpsrc port=$port_audio_rtcp                                    !\
          rtpbin.recv_rtcp_sink_1                                        \
        rtpbin.                                                         !\
          $VDEPAY ! decodebin name=vdecoder                             !\
          videoconvert ! autovideosink                                   \
        rtpbin.                                                         !\
          decodebin name=adecoder ! audioconvert                        !\
          autoaudiosink

Sender pipeline script:
  port_base=14000
  port_video_rtp=$port_base
  port_video_rtcp=$(($port_base+1))
  port_audio_rtp=$(($port_base+2))
  port_audio_rtcp=$(($port_base+3))
  host=127.0.0.1
  AUDIOFORMATOUT="audio/mpeg,mpegversion=4,stream-format=raw"
  AUDIOFORMAT="audio/x-raw,format=S16LE,layout=interleaved,rate=44100,channels=2"
  VIDSRC="videotestsrc is-live=1"
  VIDEOFORMAT="video/x-raw,format=I420,pixel-aspect-ratio=1/1,interlace-mode=progressive,width=640,height=480,framerate=25/1"
  VIDEOFORMATOUT="video/x-h264,alignment=au,stream-format=byte-stream,profile=main"
  gst-launch-1.0 -v rtpbin name=rtpbin                     \
        audiotestsrc is-live=1                          !\
        queue ! $AUDIOFORMAT                            !\
        audioconvert ! faac bitrate=128000 ! aacparse   !\
        $AUDIOFORMATOUT                                 !\
        rtpmp4apay                                      !\
        rtpbin.send_rtp_sink_1                           \
          rtpbin.send_rtp_src_1                         !\
          udpsink host=$host port=$port_audio_rtp        \
          rtpbin.send_rtcp_src_1                        !\
          udpsink host=$host port=$port_audio_rtcp sync=false async=false \
        videotestsrc is-live=1                          !\
        $VIDEOFORMAT                                    !\
        queue ! videoconvert                            !\
        x264enc bitrate=1500 tune=zerolatency speed-preset=2 key-int-max=60 bframes=0 !\
        $VIDEOFORMATOUT                                 !\
        h264parse ! rtph264pay                          !\
        rtpbin.send_rtp_sink_0                           \
          rtpbin.send_rtp_src_0                         !\
          udpsink host=$host port=$port_video_rtp        \
          rtpbin.send_rtcp_src_0                        !\
          udpsink host=$host port=$port_video_rtcp sync=false async=false
Comment 11 Peter Maersk-Moller 2016-05-12 10:17:40 UTC
(In reply to Sebastian Dröge (slomo) from comment #9)
> Yes, it's a bug in those pipelines. The receiver RTP caps are not complete,
> they are missing cpresent=(string)0 and config=(string)40002320.

Hi Sebastian.

So apparently some caps were not complete, however sometimes when caps are not complete, the information about it is in some cases either hidden or difficult to detect. What is the official GStreamer "SHOULD/MUST" policy for modules/plugins about printing informative error messages about missing caps in such cases?

Best regards and thanks for the effort and help.
Comment 12 Sebastian Dröge (slomo) 2016-05-12 10:25:00 UTC
(In reply to Peter Maersk-Moller from comment #10)

> I actually tried this before the initial report, but back then I omitted the
> '(string)' qualifier for the config parameter. And it that case it failed.
> First time ever I came across omitting a string qualifier for a parameter
> made a difference, but I guess that is pure chance.

There are some defaulting rules. If you don't provide a type and it looks like a number, it's going to be an integer. However the (non-common) RTP fields in the caps are all strings as they are supposed to be stored in SDP in a generic way.

> That said, when using the following receiver script and sender script, the
> audio pauses for few hundred milliseconds every 2 seconds. Can anybody else
> confirm this and should I open another bug report?

New bug please :)

(In reply to Peter Maersk-Moller from comment #11)

> So apparently some caps were not complete, however sometimes when caps are
> not complete, the information about it is in some cases either hidden or
> difficult to detect. What is the official GStreamer "SHOULD/MUST" policy for
> modules/plugins about printing informative error messages about missing caps
> in such cases?

Yes that would make sense. Want to prepare a patch for this?
Comment 13 Sebastian Dröge (slomo) 2016-05-18 10:30:38 UTC
Peter?
Comment 14 Peter Maersk-Moller 2016-05-18 21:47:12 UTC
(In reply to Sebastian Dröge (slomo) from comment #13)
> Peter?

Sorry. Been away for a funeral.

> New bug please :)

Ok. Need to ask a few questions in the thread https://lists.freedesktop.org/archives/gstreamer-devel/2016-May/058511.html before filing the bug report. Let me get up to speed and come back on this.

> Yes that would make sense. Want to prepare a patch for this?

I'm afraid I'm not that much into the code yet and being involved in a few other projects, I keep trying to write code there and ONLY (so far yet) limit it to thoroughly documents bugs in GStreamer so they hopefully are easier to fix.

Regards
Peter
Comment 15 Tim-Philipp Müller 2016-05-20 08:26:44 UTC
Is it not possible here to send the config inline as well?
Comment 16 Peter Maersk-Moller 2016-05-23 09:00:14 UTC
Hi.

I wrote earlier this:
> That said, when using the following receiver script and sender script, the
> audio pauses for few hundred milliseconds every 2 seconds. Can anybody else
> confirm this and should I open another bug report?

This is not an issue. I was mistaken and mixed a 48000 Hz config with a 44100 Hz stream. My mistake. I apologize for the inconvenience.

Regards
Peter MM
Comment 17 Peter Maersk-Moller 2016-05-23 12:09:14 UTC
Would it make sense to suggest the rtpbin or the rtpmp4adepay module checks, when checking that all necessary caps are set, that the config parameter in the caps agrees with the clock-rate parameter (AAC specific only) in the same caps? 

application/x-rtp,media=(string)audio,clock-rate=(int)44100,encoding-name=(string)MP4A-LATM,cpresent=(string)0,config=(string)40002420

Here the 44100 for clock-rate should match a 24 in the config. 48000 should match a 23, 64000 match a 22 etc.

Regards
Peter
Comment 18 Sebastian Dröge (slomo) 2016-05-23 12:22:41 UTC
rtpmp4apay/depay could do that, yes
Comment 19 Edward Hervey 2018-05-05 11:31:45 UTC
Still valid. Someone should provide a patch
Comment 20 GStreamer system administrator 2018-11-03 15:09:15 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/273.