GNOME Bugzilla – Bug 766267
rtpmp4adepay: Should reject caps without codec configuration
Last modified: 2018-11-03 15:09:15 UTC
Trying to RTP stream AAC audio fails. The modules aacparse, avdec_aac or decodebin all fails to output packets. Probable cause is something with rtpmp4apay/rtpmp4adepay. Scripts documenting the problem: Sender (one of the following): $ AUDIO='audio/x-raw,rate=48000,channels=2' $ gst-launch-1.0 -v audiotestsrc is-live=1 ! $AUDIO ! faac ! queue ! rtpmp4apay ! udpsink host=127.0.0.1 port=14002 $ gst-launch-1.0 -v audiotestsrc is-live=1 ! $AUDIO ! avenc_aac ! queue ! rtpmp4apay ! udpsink host=127.0.0.1 port=14002 $ gst-launch-1.0 -v audiotestsrc is-live=1 ! $AUDIO ! avenc_aac compliance=-2 ! queue ! rtpmp4apay ! udpsink host=127.0.0.1 port=14002 Receiver: $ gst-launch-1.0 -v udpsrc port=14002 caps="application/x-rtp,media=audio,payload=96,clock-rate=48000,encoding-name=MP4A-LATM" do-timestamp=1 ! rtpmp4adepay ! aacparse ! identity silent=0 ! fakesink At the receiver end you will see no packets coming from aacparse. You can replace aacparse with avdec_aac or decodebin and get same result. Regards Peter
Not a blocker as it's not a regression, or is it?
(In reply to Sebastian Dröge (slomo) from comment #1) > Not a blocker as it's not a regression, or is it? Well, it has worked before. Obviously MPEG-4 audio over RTP has worked before, so yes, it is a regression as it is not working now. That said, not being able to either stream MPEG-4 audio over RTP or play same or both, can hardly have a severity of "normal". P
RTP streaming of AAC works just fine with gst-rtsp-server and gst-play for what it's worth (using mp4a pay/depay), so it must be something about the combination of elements. Time spent arguing about the bug severity would probably better be spent investigating the issue. If the exact same pipeline/script is known to work with a certain older version of GStreamer, that's useful information, then we can compare/bisect.
(In reply to Peter Maersk-Moller from comment #2) > (In reply to Sebastian Dröge (slomo) from comment #1) > > Not a blocker as it's not a regression, or is it? > > Well, it has worked before. Obviously MPEG-4 audio over RTP has worked > before, so yes, it is a regression as it is not working now. My question was mostly: did it work before and with which versions :) If it worked before and is a regression, then it is indeed a blocker and we should get it fixed before the next release. If OTOH it never worked it would be less important.
(In reply to Sebastian Dröge (slomo) from comment #4) > My question was mostly: did it work before and with which versions :) If it > worked before and is a regression, then it is indeed a blocker and we should > get it fixed before the next release. > > If OTOH it never worked it would be less important. Understood. Seems not to work in 1.6.3, 1.4.5 and 1.3.0, unfortunately :-(
Thanks for confirming :) That should be easy to bisect then
*not* work... ;)
Meh, I can't read :) Sorry
Yes, it's a bug in those pipelines. The receiver RTP caps are not complete, they are missing cpresent=(string)0 and config=(string)40002320.
Confirming setting caps for the udpsrc for audio to audio_caps='application/x-rtp,media=audio,payload=96,clock-rate=44100,encoding-name=MP4A-LATM,cpresent=(string)0,config=(string)40002320' works for streaming AAC using rtp as well as rtpbin for version 1.8.1. I actually tried this before the initial report, but back then I omitted the '(string)' qualifier for the config parameter. And it that case it failed. First time ever I came across omitting a string qualifier for a parameter made a difference, but I guess that is pure chance. That said, when using the following receiver script and sender script, the audio pauses for few hundred milliseconds every 2 seconds. Can anybody else confirm this and should I open another bug report? Thanks. Receiver pipeline script VDEPAY=rtph264depay audio_caps='application/x-rtp,media=audio,payload=96,clock-rate=44100,encoding-name=MP4A-LATM,cpresent=(string)0,config=(string)40002320' gst-launch-1.0 -v rtpbin name=rtpbin buffer-mode=slave \ udpsrc do-timestamp=1 caps=$video_caps port=$port_video_rtp !\ rtpbin.recv_rtp_sink_0 \ udpsrc port=$port_video_rtcp !\ rtpbin.recv_rtcp_sink_0 \ udpsrc caps=$audio_caps port=$port_audio_rtp !\ rtpbin.recv_rtp_sink_1 \ udpsrc port=$port_audio_rtcp !\ rtpbin.recv_rtcp_sink_1 \ rtpbin. !\ $VDEPAY ! decodebin name=vdecoder !\ videoconvert ! autovideosink \ rtpbin. !\ decodebin name=adecoder ! audioconvert !\ autoaudiosink Sender pipeline script: port_base=14000 port_video_rtp=$port_base port_video_rtcp=$(($port_base+1)) port_audio_rtp=$(($port_base+2)) port_audio_rtcp=$(($port_base+3)) host=127.0.0.1 AUDIOFORMATOUT="audio/mpeg,mpegversion=4,stream-format=raw" AUDIOFORMAT="audio/x-raw,format=S16LE,layout=interleaved,rate=44100,channels=2" VIDSRC="videotestsrc is-live=1" VIDEOFORMAT="video/x-raw,format=I420,pixel-aspect-ratio=1/1,interlace-mode=progressive,width=640,height=480,framerate=25/1" VIDEOFORMATOUT="video/x-h264,alignment=au,stream-format=byte-stream,profile=main" gst-launch-1.0 -v rtpbin name=rtpbin \ audiotestsrc is-live=1 !\ queue ! $AUDIOFORMAT !\ audioconvert ! faac bitrate=128000 ! aacparse !\ $AUDIOFORMATOUT !\ rtpmp4apay !\ rtpbin.send_rtp_sink_1 \ rtpbin.send_rtp_src_1 !\ udpsink host=$host port=$port_audio_rtp \ rtpbin.send_rtcp_src_1 !\ udpsink host=$host port=$port_audio_rtcp sync=false async=false \ videotestsrc is-live=1 !\ $VIDEOFORMAT !\ queue ! videoconvert !\ x264enc bitrate=1500 tune=zerolatency speed-preset=2 key-int-max=60 bframes=0 !\ $VIDEOFORMATOUT !\ h264parse ! rtph264pay !\ rtpbin.send_rtp_sink_0 \ rtpbin.send_rtp_src_0 !\ udpsink host=$host port=$port_video_rtp \ rtpbin.send_rtcp_src_0 !\ udpsink host=$host port=$port_video_rtcp sync=false async=false
(In reply to Sebastian Dröge (slomo) from comment #9) > Yes, it's a bug in those pipelines. The receiver RTP caps are not complete, > they are missing cpresent=(string)0 and config=(string)40002320. Hi Sebastian. So apparently some caps were not complete, however sometimes when caps are not complete, the information about it is in some cases either hidden or difficult to detect. What is the official GStreamer "SHOULD/MUST" policy for modules/plugins about printing informative error messages about missing caps in such cases? Best regards and thanks for the effort and help.
(In reply to Peter Maersk-Moller from comment #10) > I actually tried this before the initial report, but back then I omitted the > '(string)' qualifier for the config parameter. And it that case it failed. > First time ever I came across omitting a string qualifier for a parameter > made a difference, but I guess that is pure chance. There are some defaulting rules. If you don't provide a type and it looks like a number, it's going to be an integer. However the (non-common) RTP fields in the caps are all strings as they are supposed to be stored in SDP in a generic way. > That said, when using the following receiver script and sender script, the > audio pauses for few hundred milliseconds every 2 seconds. Can anybody else > confirm this and should I open another bug report? New bug please :) (In reply to Peter Maersk-Moller from comment #11) > So apparently some caps were not complete, however sometimes when caps are > not complete, the information about it is in some cases either hidden or > difficult to detect. What is the official GStreamer "SHOULD/MUST" policy for > modules/plugins about printing informative error messages about missing caps > in such cases? Yes that would make sense. Want to prepare a patch for this?
Peter?
(In reply to Sebastian Dröge (slomo) from comment #13) > Peter? Sorry. Been away for a funeral. > New bug please :) Ok. Need to ask a few questions in the thread https://lists.freedesktop.org/archives/gstreamer-devel/2016-May/058511.html before filing the bug report. Let me get up to speed and come back on this. > Yes that would make sense. Want to prepare a patch for this? I'm afraid I'm not that much into the code yet and being involved in a few other projects, I keep trying to write code there and ONLY (so far yet) limit it to thoroughly documents bugs in GStreamer so they hopefully are easier to fix. Regards Peter
Is it not possible here to send the config inline as well?
Hi. I wrote earlier this: > That said, when using the following receiver script and sender script, the > audio pauses for few hundred milliseconds every 2 seconds. Can anybody else > confirm this and should I open another bug report? This is not an issue. I was mistaken and mixed a 48000 Hz config with a 44100 Hz stream. My mistake. I apologize for the inconvenience. Regards Peter MM
Would it make sense to suggest the rtpbin or the rtpmp4adepay module checks, when checking that all necessary caps are set, that the config parameter in the caps agrees with the clock-rate parameter (AAC specific only) in the same caps? application/x-rtp,media=(string)audio,clock-rate=(int)44100,encoding-name=(string)MP4A-LATM,cpresent=(string)0,config=(string)40002420 Here the 44100 for clock-rate should match a 24 in the config. 48000 should match a 23, 64000 match a 22 etc. Regards Peter
rtpmp4apay/depay could do that, yes
Still valid. Someone should provide a patch
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/273.