GNOME Bugzilla – Bug 646167
rtpjitterbuffer: stops processing when mode=buffer and drop-on-latency=1
Last modified: 2018-11-03 14:43:46 UTC
I was doing some tests with gstrtpjitterbuffer, basically generating audio with audiotestsrc and sending it using rtp (ptime must be 20ms on my tests). The pipeline used to generate audio is: gst-launch -m audiotestsrc is_live=true ! alawenc ! audio/x-alaw,rate=8000,channels=1 ! rtppcmapay min-ptime=20000000 max-ptime=20000000 ! udpsink host=127.0.0.1 port=5000 And on the receive side i have an application that does some measurements using pad probing on the gstrtpjitterbuffer element. Everything was ok until i started to set the gstrtpjitterbuffer "mode" property to "buffer". I got a little audio and them silence. First thing i thought was "my test must be broken". But when i started to debug to find out what i was doing wrong i found that what was making my test getting locked was: g_object_set(G_OBJECT (jitter_buffer), "drop-on-latency", TRUE, NULL); so i tried to reproduce the problem using gst-launch and it really stops processing, (the "buffering" message gets on 71% and stops, no audio is heard). If i don't set drop-on-latency=1, this don't happens, or if the mode is not buffer it works with drop-on-latency=1. Basically: works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=none drop-on-latency=1 ! decodebin ! autoaudiosink works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=none drop-on-latency=0 ! decodebin ! autoaudiosink works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=slave drop-on-latency=1 ! decodebin ! autoaudiosink works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=slave drop-on-latency=0 ! decodebin ! autoaudiosink works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=buffer drop-on-latency=0 ! decodebin ! autoaudiosink stop processing on 71%: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=buffer drop-on-latency=1 ! decodebin ! autoaudiosink i didn't find anything saying that the drop-on-latency property must not be set to true when the buffering mode is "buffer", so i thought this must be a bug (it is pretty simple to reproduce and the pipelines are quite simple too). The problem happened on Ubuntu 10.10/Gstreamer 0.10.30.
Testing a little more i found out that if you run the sender pipeline first, and them the receiver pipeline, it wont block. But if you run the receiver pipeline first and then the sender pipeline, it blocks. to reproduce the problem, run: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=buffer drop-on-latency=1 ! decodebin ! autoaudiosink and them: gst-launch -m audiotestsrc is_live=true ! alawenc ! audio/x-alaw,rate=8000,channels=1 ! rtppcmapay min-ptime=20000000 max-ptime=20000000 ! udpsink host=127.0.0.1 port=5000 it will block on buffering 71%. it is interesting that changing the sink changes the way the problem happens. with alsasink the buffering message blocks on 53%, pulsesink blocks on 71%, and if you use a fakesink, the problem dont happens at all. can someone confirm if this problem really exist or if im doing something stupid here?
Just confirming that the problem still occurs on Ubuntu 11.04 / Gstreamer 0.10.32. The start of the sound can be heard but them the pipeline get stuck on 71% (for pulsesink) or 53% (for alsasink). Any other buffer mode seems to work, only mode=buffer drop-on-latency=1 seems to generate this problem.
I can reproduce this on the 1.1.x master, I don't even need to set drop-on-latency to make it fail, it seems that buffer mode is completely broken now. Receiver (start first): gst-launch-1.0 udpsrc port=5000 caps='application/x-rtp,clock-rate=8000,payload=8,encoding-name=PCMA' ! rtpjitterbuffer mode=buffer drop-on-latency=0 ! rtppcmadepay ! alawdec ! pulsesink Sender: gst-launch-1.0 -v audiotestsrc is_live=true ! alawenc ! rtppcmapay ! udpsink host=127.0.0.1 port=5000
Still the case one year later, it starts buffering and never stops.
The buffer mode still seems to be broken on 1.4.0. If I run the commands Olivier posted in comment 3, the receiver buffers to 100% and then keeps on adding incoming packets to the jitterbuffer forever (according to GST_DEBUG=rtpjitterbuffer:5). However, the behavior seems to change with the latency value. If I e.g. change the latency to 700 ms, I actually get audio through and the buffer is limited to 5 packets. It only works if I start the sender first, and only with drop-on-latency=false.
Wim: If no one fixes, we may as well remove it and all of the complexities it adds.
It appears, using Olivier's test above, that the push thread goes to wait on the event g_cond after the deadline timer expires and never comes back. It waits because at that time, the jitterbuffer is buffering. if (priv->blocked || !priv->active || rtp_jitter_buffer_is_buffering (priv->jbuf)) There are a few ways to kick it out (event, reset, etc.), but in normal flow, none of them happen. To prove this out I just added JBUF_SIGNAL_EVENT (priv) at the end of the chain, though that definitely isn't a fix. With that, buffers start flowing again. It seems the chain fn needs to signal if the jitterbuffer was buffering last time the push thread went to wait.
Created attachment 295500 [details] [review] Proposed fix This patch (against master) fixes the jitterbuffer in that, if the push thread is waiting, we wake it up even if the head hasn't changed. Commit 3cd0e8ae8 introduced the change related to the head, and that commit makes sense, but has negative effects for this mode. That said, if you start the receiver in either case above, you still won't hear audio, even though the buffer is pushing. The reason is the timestamps (audiobasesink drops them). If the receiver is started after, then the first test results in a smooth tone, while the second (on my machine) is a bit choppy. All plugins good jitterbuffer tests pass against master (there was a soup failure), but I did not add a new one. I will if this patch looks good upon review.
Created attachment 295609 [details] [review] Proposed fix v2 The previous patch did not check the active flag. Additionally, to preserve the logic associated with actions when the head changes, I did not modify the code inside the if block, so there is some duplication.
Is it possible to only wake up the pushing thread when the buffer is full, instead of for every new buffer?
Good point, and I don't see why not. Querying the buffer (similar to what handle_next_buffer()) does would work. I'll code up a patch tonight, try it out, and resubmit.
Created attachment 295716 [details] [review] Proposed patch v3 This patch addresses the case when drop-on-latency=0. Per earlier discussions, it should only wake the thread up for the buffer-mode related condition if the jbuf mode is buffer, it isn't buffering, the jbuf is active, and the push thread is waiting. This does not address the case of drop-on-latency=1. I will (try to) address those next.
Created attachment 295717 [details] [review] Proposed patch drop-on-latency The drop-on-latency case is slightly more involved than the drop-on-latency=0 case. First, the latency, as noted above, will affect the results, but it still eventually fails. It passes buffers for a time because most of the early items in the jbuf are not droppable. This allows the jbuf a chance to finish buffering. However, once the buffer drains, it will likely never push again. The reason, from what I've seen, is the call rtp_jitter_buffer_get_ts_diff(). It appears the logic is just reversed, i.e. the accounting for wrap is done when the head < tail, not the other way around. Because of this, the dropping code always drops a buffer, which means the buffer never fills.
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/42.