GNOME Bugzilla – Bug 726579
Receiving opus audio over RTP sometimes include static in the right channel
Last modified: 2016-03-03 17:49:51 UTC
Created attachment 272206 [details] Dump of receiver pipeline In 1.2.3 I randomly get a static in the right channel of audio received via RTP if it's encoded as opus. I was not able to find a pattern, but I have never reproduced this using e.g. vorbis so I guess it is related to opus codec. Please see attached pipeline and sample recordings. For streaming I use avconv although I was able to reproduce this by using gst-launch as a sender. Sender command: $ avconv -f pulse -i default -acodec libopus -ab 128k -f rtp rtp://localhost:10000 avconv version 9.11-6:9.11-2ubuntu1, Copyright (c) 2000-2013 the Libav developers built on Mar 2 2014 20:32:58 with gcc 4.8 (Ubuntu 4.8.2-16ubuntu4) [pulse @ 0xcf5100] Estimating duration from bitrate, this may be inaccurate Guessed Channel Layout for Input Stream #0.0 : stereo Input #0, pulse, from 'default': Duration: N/A, start: 0.005766, bitrate: N/A Stream #0.0: Audio: pcm_s16le, 48000 Hz, stereo, s16, 1536 kb/s Output #0, rtp, to 'rtp://localhost:10000': Metadata: encoder : Lavf54.20.3 Stream #0.0: Audio: libopus, 48000 Hz, stereo, s16, 128 kb/s Stream mapping: Stream #0:0 -> #0:0 (pcm_s16le -> libopus) SDP: v=0 o=- 0 0 IN IP4 127.0.0.1 s=No Name c=IN IP4 127.0.0.1 t=0 0 a=tool:libavformat 54.20.3 m=audio 10000 RTP/AVP 97 b=AS:128 a=rtpmap:97 opus/48000 Press ctrl-c to stop encoding ^Csize= 1446kB time=112.17 bitrate= 105.6kbits/s
Created attachment 272207 [details] Sample recording of received audio
I was able to capture GST_DEBUG=*:4 output and the only suspicious messages are 0:02:57.384555751 22025 0x7f54b8003720 WARN audioresample gstaudioresample.c:998:gst_audio_resample_check_discont:<resampler> encountered timestamp discontinuity of 1669 samples = 0:00:00.034770833 0:02:57.400888624 22025 0x7f54b8003720 WARN audioresample gstaudioresample.c:998:gst_audio_resample_check_discont:<resampler> encountered timestamp discontinuity of 1641 samples = 0:00:00.034187500 0:02:57.451372489 22025 0x7f54b8003720 WARN audioresample gstaudioresample.c:998:gst_audio_resample_check_discont:<resampler> encountered timestamp discontinuity of 1544 samples = 0:00:00.032166667 0:02:57.511207611 22025 0x7f54b8003720 WARN audioresample gstaudioresample.c:998:gst_audio_resample_check_discont:<resampler> encountered timestamp discontinuity of 1556 samples = 0:00:00.032416667 0:02:57.561562897 22025 0x7f54b8003720 WARN audioresample gstaudioresample.c:998:gst_audio_resample_check_discont:<resampler> encountered timestamp discontinuity of 1737 samples = 0:00:00.036187500 0:02:57.631957957 22025 0x7f54b8003720 WARN audioresample gstaudioresample.c:998:gst_audio_resample_check_discont:<resampler> encountered timestamp discontinuity of 1624 samples = 0:00:00.033833333 I've also changed alsasink to pulsesink - no difference, still able to reproduce so it is rather not a sink problem.
Could you please share the source gst-launch command line you mentioned to test with ? The avconv I have here does not support opus. The receiver pipeline looks awfully complicated too. Does it also happen with straight gst-launch-1.0 rtsp://localhost:10000 ? Last, how repeatable is this, roughly ?
Hey, recently I've encountered very similar problems with MP3 file decoded via filesrc. The pipeline was roughly the same, my app basically takes some audio sources, and mix them with adder. Adder always has at least one fake source (audiotestsrc playing silence) to keep pipeline running and it whole pipeline is forced to use 44.1KHz wherever possible via capsfilters. That problem encountered only with MP3 files that were recorded in 48KHz. Opus uses 48KHz, too. So combining all this information plus beforementioned log I am starting to think that this bug is not related to Opus, but audioresample. Or combination of different sources where some are 44.1KHz, some are 48KHz, resampled to 44.1, and then mixed by adder. Is it possible be due to the fact that adder don't care about timestamps?
> Is it possible be due to the fact that adder don't care about timestamps? You could try with the new 'audiomixer' element instead.
Do you think it is mature enough to use it in the production app, that has to be very stable?
That obviously depends on your requirements and what exactly you're doing. Try it and see if it makes a difference, then see if it runs stable enough for you. If it isn't stable enough for you we need to fix that.
Can you please give me any guidance on how to backport this to 1.2.3 and compile as a separate package? I have to rely on stable branch available in ubuntu 14.04, if I introduce extra element it must be packed as a separate .deb that I will ship to my clients.
Created attachment 274195 [details] Recording of two filesrcs that are affected
I have caught an example of the same problem with two filesrcs. First file (the one with static) is 48KHz (Audio file with ID3 version 2.3.0, contains: MPEG ADTS, layer III, v1, 320 kbps, 48 kHz, JntStereo). This file is starting to fade out around 0:40-0:45. You can hear around 0:44 that static is faded out along with music. For me it's another indication that audioresample that is located in this branch of the pipeline is causing problems, not the adder itself. I suggest renaming bug to "audioresample randomly adds static on timestamp discontinuity". Unfortunately GST_DEBUG no logs this time, as it was caught in the production environment.
As a data point, [1] seems to produce good audio (so far) and does not trigger valgrind warnings (apart from leaks). One source is 44.1 kHz and the other 48 kHz. However, [2] produces [3], which might have been a plausible reason if you hadn't tried pulsesink too, as I don't see it happen with pulsesink (which AFAIK uses ALSA under the hood). [1] gst-launch-1.0 filesrc location=/home/v/Samples/8.wav ! wavparse ! audioconvert ! audioresample ! 'audio/x-raw,format=S32LE,rate=48000,channels=2' ! queue name=q48 ! adder name=mix ! queue ! audioconvert ! alsasink filesrc location=/home/v/Samples/6.wav ! wavparse ! audioconvert ! audioresample ! 'audio/x-raw,format=S32LE,rate=48000,channels=2' ! queue name=q44 ! mix. [2] gst-launch-1.0 filesrc location=/home/v/Samples/8.wav ! wavparse ! audioconvert ! audioresample ! 'audio/x-raw,format=S32LE,rate=44100,channels=2' ! queue name=q48 ! adder name=mix ! queue ! audioconvert ! alsasink filesrc location=/home/v/Samples/6.wav ! wavparse ! audioconvert ! audioresample ! 'audio/x-raw,format=S32LE,rate=44100,channels=2' ! queue name=q44 ! mix. [3] ==24984== Thread 8: ==24984== Source and destination overlap in memcpy(0x11e58bc0, 0x11e58bc0, 8192) ==24984== at 0x4C2CFA0: memcpy@@GLIBC_2.14 (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==24984== by 0x93145EA: snd_pcm_area_copy (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x93147CC: snd_pcm_areas_copy (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x9352D8D: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x932458B: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x932EDA0: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x932EE61: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x932EF3B: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x93226EC: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x93181CC: ??? (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x93229D9: snd_pcm_mmap_writei (in /usr/lib/x86_64-linux-gnu/libasound.so.2.0.0) ==24984== by 0x90BA6DB: gst_alsasink_write (gstalsasink.c:1041) ==24984== by 0x7F89997: audioringbuffer_thread_func (gstaudiosink.c:252) ==24984== by 0x53FD304: g_thread_proxy (gthread.c:764) ==24984== by 0x56C7E99: start_thread (pthread_create.c:308) ==24984==
Might be related to https://bugzilla.gnome.org/show_bug.cgi?id=739800 (static on decoding high sample rate files).
> For me it's another indication that audioresample that is located in this > branch of the pipeline is causing problems, not the adder itself. > > I suggest renaming bug to "audioresample randomly adds static on timestamp > discontinuity". There were quite a few audioresample fixes recently, some of which caused issues like the one described. I'd say let's close this, since it doesn't look like there's a lot we can do without a way to reproduce it anyway. If it's still a problem, someone will hopefully file a new bug. Please re-open if you can still reproduce it with git master.