GNOME Bugzilla – Bug 667796
playback: Use bitrate estimation from downstream for buffering
Last modified: 2018-11-03 11:20:38 UTC
When listening to internet radio streams (http, doesn't matter which codec) at one point (sometimes within a few seconds, sometimes after 30mins) it stops playing. Sometimes is just stops playing back and never resumes, sometimes it stops playing back, buffers up some more data, and then resumes playback. After digging a bit deeper it seems that for the stops-and-never-resumes scenario the issue is that souphttpsrc just stops reading (i.e. the server doesn't return anything). After having a look at various graphs of what is being read, my gutt feeling (tm) is that we are playing back (and therefore reading from the server) slightly slower/faster than the remote server, resulting in the following: * We connect to server and start downloading data (initially it might give us a burst from the previous X seconds of stream, which corresponds to the size of the ringbuffer the server keeps for providing encoded content) * We read/playback the stream slightly slower/faster * Eventually we end up wanting data which the server either no longer has in its queue (slower local playback) or hasn't encoded yet (faster local playback). Taking the following values: Server buffer : 5s Readout speed : 0.2% slower than server encoding rate We end up with: Time after which we'll end up wanting to read data *past* the 5s buffer size: 5s / 0.002 => 2500s => 41m40s If we were reading faster, then it's the opposite problem: The server gives a burst of 5 seconds of data, after 41m40s we end up having caught up with live playback and need to buffer up some more data. So essentially ... with the current implementation, unless your soundcard is at *exactly* the same playback rate as the server's encoding system ... you'll always end up with either: * Your playback just stops (because we're playing back slightly slower and eventually have no more data to read from the server). * Your playback pauses until there is more data to read (because we're playing back slightly faster). Note : There might *also* be an issue with queue2 throttling the input too aggressively, will attach more findings after.
Shouldn't it just go back to "buffering" ? That said, the same problem happens if you do plain mpegts streaming, so maybe we should have a generic solution doing something liek what rtpjitterbuffer does.
tbh the server should close the connection if you try to read past the ringbuffer limit. And yes, it does go to buffering (queue2 drains), but never gets filled again. As for mpegts streaming ... in what situation ? over udp ? over tcp ? dvb ? The problem is that this kind of streaming aren't "live" as per the GStreamer definition of it (i.e. no data gets dropped if you're a tiny bit late getting it). That being said, it would be nice to consider them as "live but with massive jitter and latency" so that we can apply some very-big-window (think 30s-1min) rate estimation to have the playback gradually adjust itself to the overall rate and avoid this issue.
Moving to playbin.
This is still valid but should be fixed another way (in playbin3 obviously) by having bitrate estimation provided by downstream elements (such as parsers). The root cause is that queue2 figures out how much to buffer (in time) completely wrong (by guess-timating wrongly the input bitrate). * Add a new GST_QUERY_BITRATE to ask for nominal (and potentialy minimum and maximum bitrate) * Implement the query handling in parsers and demuxers * Have queue2 query downstream if upstream didn't provide it * You know the *actual* amount of bytes to buffer for given buffering time limits
*** Bug 776491 has been marked as a duplicate of this bug. ***
> The root cause is that queue2 figures out how much to buffer (in time) > completely wrong (by guess-timating wrongly the input bitrate). > > * Add a new GST_QUERY_BITRATE to ask for nominal (and potentialy minimum and > maximum bitrate) About this,Is there any further hints? perhaps this is a urgent need improvement for playbin3, Everything else is fine,but buffering does not perform as well when playing some streaming case type streams(mpegts over http,etc) compared to playbin2.:(
The proposed implementation is being worked on and should have someone to review soon
(In reply to Edward Hervey from comment #7) > The proposed implementation is being worked on and should have someone to > review soon Wow,really looking forward to it.Thanks for this information :) In addition, if bitrate can be exposed in playbin3/or urisourbin, perhaps upper app can dynamically set some parameters based on the runtime bitrate, and more different types of applications will benefit from it.
Created attachment 372842 [details] [review] query: add a new bitrate query Basic guts of the implementation.
Very helpful.^^ Do you have a plan to submit a patch that downstream plugins(parser?demuxer?) implements query_bitrate ? :)
Tracer tool points out that there's a leak here, gstqueue2.c query_downstream_bitrate(),missing "gst_query_unref(query);" :)
Created attachment 373624 [details] [review] query: add a new bitrate query
Created attachment 373625 [details] [review] urisourcebin: limit the byte size of the queue based on the bitrate reported by queue2
Created attachment 373626 [details] [review] urisourcebin: add low/high watermark properties
Created attachment 373627 [details] [review] urisourcebin: update forwarded properties on change
Created attachment 373628 [details] [review] urisourcebin: add a statistics property for queuing
Created attachment 373629 [details] [review] tests/urisourcebin: add get/set watermark test
Created attachment 373630 [details] [review] isompt: implement support for the bitrate query
Created attachment 373631 [details] [review] matroska: implement support for the bitrate query
Created attachment 373632 [details] [review] tsdemux: implement support for the bitrate query
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/60.