After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 646167 - rtpjitterbuffer: stops processing when mode=buffer and drop-on-latency=1
rtpjitterbuffer: stops processing when mode=buffer and drop-on-latency=1
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-good
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2011-03-29 22:13 UTC by tiagokatcipis
Modified: 2018-11-03 14:43 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Proposed fix (1.06 KB, patch)
2015-01-27 04:26 UTC, Jason Litzinger
none Details | Review
Proposed fix v2 (1.10 KB, patch)
2015-01-28 04:44 UTC, Jason Litzinger
none Details | Review
Proposed patch v3 (1.41 KB, patch)
2015-01-29 05:04 UTC, Jason Litzinger
none Details | Review
Proposed patch drop-on-latency (1.14 KB, patch)
2015-01-29 05:12 UTC, Jason Litzinger
none Details | Review

Description tiagokatcipis 2011-03-29 22:13:19 UTC
I was doing some tests with gstrtpjitterbuffer, basically generating audio with audiotestsrc and sending it using rtp (ptime must be 20ms on my tests).

The pipeline used to generate audio is:

gst-launch -m audiotestsrc is_live=true ! alawenc ! audio/x-alaw,rate=8000,channels=1 ! rtppcmapay min-ptime=20000000 max-ptime=20000000 ! udpsink host=127.0.0.1 port=5000

And on the receive side i have an application that does some measurements using pad probing on the gstrtpjitterbuffer element. Everything was ok until i started to set the gstrtpjitterbuffer "mode" property to "buffer". I got a little audio and them silence. First thing i thought was "my test must be broken". But when i started to debug to find out what i was doing wrong i found that what was making my test getting locked was:

g_object_set(G_OBJECT (jitter_buffer), "drop-on-latency", TRUE, NULL);

so i tried to reproduce the problem using gst-launch and it really stops processing, (the "buffering"  message gets on 71% and stops, no audio is heard). If i don't set drop-on-latency=1, this don't happens, or if the mode is not buffer it works with drop-on-latency=1.

Basically:


works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=none drop-on-latency=1 ! decodebin ! autoaudiosink

works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=none drop-on-latency=0 ! decodebin ! autoaudiosink

works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=slave drop-on-latency=1 ! decodebin ! autoaudiosink

works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=slave drop-on-latency=0 ! decodebin ! autoaudiosink

works: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=buffer drop-on-latency=0 ! decodebin ! autoaudiosink

stop processing on 71%: gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=buffer drop-on-latency=1 ! decodebin ! autoaudiosink


i didn't find anything saying that the drop-on-latency property must not be set to true when the buffering mode is "buffer", so i thought this must be a bug (it is pretty simple to reproduce and the pipelines are quite simple too).

The problem happened on Ubuntu 10.10/Gstreamer 0.10.30.
Comment 1 tiagokatcipis 2011-04-18 14:18:54 UTC
Testing a little more i found out that if you run the sender pipeline first, and them the receiver pipeline, it wont block. But if you run the receiver pipeline first and then the sender pipeline, it blocks.

to reproduce the problem, run:

gst-launch udpsrc port=5000 caps=application/x-rtp,clock-rate=8000 ! gstrtpptdemux ! gstrtpjitterbuffer mode=buffer drop-on-latency=1 ! decodebin ! autoaudiosink

and them:

gst-launch -m audiotestsrc is_live=true ! alawenc ! audio/x-alaw,rate=8000,channels=1 ! rtppcmapay min-ptime=20000000 max-ptime=20000000 ! udpsink host=127.0.0.1 port=5000

it will block on buffering 71%.

it is interesting that changing the sink changes the way the problem happens. with alsasink the buffering message blocks on 53%, pulsesink blocks on 71%, and if you use a fakesink, the problem dont happens at all.

can someone confirm if this problem really exist or if im doing something stupid here?
Comment 2 tiagokatcipis 2011-05-02 14:04:43 UTC
Just confirming that the problem still occurs on Ubuntu 11.04 / Gstreamer 0.10.32.

The start of the sound can be heard but them the pipeline get stuck on 71% (for pulsesink) or 53% (for alsasink).

Any other buffer mode seems to work, only mode=buffer drop-on-latency=1 seems to generate this problem.
Comment 3 Olivier Crête 2013-01-29 00:09:50 UTC
I can reproduce this on the 1.1.x master, I don't even need to set drop-on-latency to make it fail, it seems that buffer mode is completely broken now.

Receiver (start first):
gst-launch-1.0 udpsrc port=5000 caps='application/x-rtp,clock-rate=8000,payload=8,encoding-name=PCMA' ! rtpjitterbuffer mode=buffer drop-on-latency=0 ! rtppcmadepay ! alawdec ! pulsesink

Sender:
gst-launch-1.0 -v audiotestsrc is_live=true ! alawenc ! rtppcmapay ! udpsink host=127.0.0.1 port=5000
Comment 4 Sebastian Dröge (slomo) 2014-01-26 17:30:28 UTC
Still the case one year later, it starts buffering and never stops.
Comment 5 Jeppe Ledet-Pedersen 2014-07-30 14:49:17 UTC
The buffer mode still seems to be broken on 1.4.0.

If I run the commands Olivier posted in comment 3, the receiver buffers to 100% and then keeps on adding incoming packets to the jitterbuffer forever (according to GST_DEBUG=rtpjitterbuffer:5).

However, the behavior seems to change with the latency value. If I e.g. change the latency to 700 ms, I actually get audio through and the buffer is limited to 5 packets. It only works if I start the sender first, and only with drop-on-latency=false.
Comment 6 Olivier Crête 2014-08-01 22:28:09 UTC
Wim: If no one fixes, we may as well remove it and all of the complexities it adds.
Comment 7 Jason Litzinger 2015-01-26 05:24:04 UTC
It appears, using Olivier's test above, that the push thread goes to wait on the event g_cond after the deadline timer expires and never comes back.  It waits because at that time, the jitterbuffer is buffering.  

  if (priv->blocked || !priv->active ||
      rtp_jitter_buffer_is_buffering (priv->jbuf))

There are a few ways to kick it out (event, reset, etc.), but in normal flow, none of them happen.  

To prove this out I just added JBUF_SIGNAL_EVENT (priv) at the end of the chain, though that definitely isn't a fix. With that, buffers start flowing again.

It seems the chain fn needs to signal if the jitterbuffer was buffering last time the push thread went to wait.
Comment 8 Jason Litzinger 2015-01-27 04:26:45 UTC
Created attachment 295500 [details] [review]
Proposed fix

This patch (against master) fixes the jitterbuffer in that, if the push thread is waiting, we wake it up even if the head hasn't changed.  Commit 3cd0e8ae8 introduced the change related to the head, and that commit makes sense, but has negative effects for this mode.

That said, if you start the receiver in either case above, you still won't hear audio, even though the buffer is pushing.  The reason is the timestamps (audiobasesink drops them).  If the receiver is started after, then the first test results in a smooth tone, while the second (on my machine) is a bit choppy.

All plugins good jitterbuffer tests pass against master (there was a soup failure), but I did not add a new one.  I will if this patch looks good upon review.
Comment 9 Jason Litzinger 2015-01-28 04:44:09 UTC
Created attachment 295609 [details] [review]
Proposed fix v2

The previous patch did not check the active flag. Additionally, to preserve the logic associated with actions when the head changes, I did not modify the code inside the if block, so there is some duplication.
Comment 10 Wim Taymans 2015-01-28 09:32:51 UTC
Is it possible to only wake up the pushing thread when the buffer is full, instead of for every new buffer?
Comment 11 Jason Litzinger 2015-01-28 15:20:19 UTC
Good point, and I don't see why not.  Querying the buffer (similar to what handle_next_buffer()) does would work.  I'll code up a patch tonight, try it out, and resubmit.
Comment 12 Jason Litzinger 2015-01-29 05:04:23 UTC
Created attachment 295716 [details] [review]
Proposed patch v3

This patch addresses the case when drop-on-latency=0.  Per earlier discussions, it should only wake the thread up for the buffer-mode related condition if the jbuf mode is buffer, it isn't buffering, the jbuf is active, and the push thread is waiting.

This does not address the case of drop-on-latency=1.  I will (try to) address those next.
Comment 13 Jason Litzinger 2015-01-29 05:12:11 UTC
Created attachment 295717 [details] [review]
Proposed patch drop-on-latency

The drop-on-latency case is slightly more involved than the drop-on-latency=0 case.

First, the latency, as noted above, will affect the results, but it still eventually fails.  It passes buffers for a time because most of the early items in the jbuf are not droppable.  This allows the jbuf a chance to finish buffering.  However, once the buffer drains, it will likely never push again.  The reason, from what I've seen, is the call rtp_jitter_buffer_get_ts_diff().  It appears the logic is just reversed, i.e. the accounting for wrap is done when the head < tail, not the other way around.

Because of this, the dropping code always drops a buffer, which means the buffer never fills.
Comment 14 GStreamer system administrator 2018-11-03 14:43:46 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/42.