After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 742979 - adaptivedemux: Adaptive bitrate algorithm does not react fast enough to declining bitrate
adaptivedemux: Adaptive bitrate algorithm does not react fast enough to decli...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other All
: Normal normal
: 1.5.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
: 737703 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2015-01-15 16:02 UTC by A Ashley
Modified: 2015-02-12 00:34 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
adaptivedemux: Use a Simple Moving Average to estimate the bitrate. (3.88 KB, patch)
2015-01-20 16:43 UTC, Mathieu Duponchelle
none Details | Review
adaptivedemux: Use a Simple Moving Average to estimate the bitrate. (3.59 KB, patch)
2015-01-20 19:36 UTC, Mathieu Duponchelle
rejected Details | Review
adaptivedemux: track per-fragment bitrates. (8.82 KB, patch)
2015-01-27 16:12 UTC, Mathieu Duponchelle
committed Details | Review
adaptivedemux log on a line restricted to 2Mbps (258.51 KB, text/plain)
2015-02-03 11:16 UTC, A Ashley
  Details
Another GStreamer log, with --gst-debug=adaptivedemux:5,dashdemux:5,qtdemux:2,3 (129.26 KB, text/plain)
2015-02-03 15:08 UTC, A Ashley
  Details
Log of each HTTP request (2.52 KB, text/plain)
2015-02-03 15:14 UTC, A Ashley
  Details

Description A Ashley 2015-01-15 16:02:44 UTC
The logic used by GstAdaptiveDemux is to accumulate the total number of bytes downloaded and the time it takes to perform those downloads:
 
    bitrate =
         (stream->download_total_bytes * 8) /
         ((double) stream->download_total_time / G_GUINT64_CONSTANT(1000000));

The problem with this approach is that this algorithm does not allow GstAdaptiveDemux to rapidly react to a declining throughput, because prior downloads disproportionately impact the bitrate calculation.

I think we need something that reacts rapidly to a drop in bandwidth but is more cautious when increasing bandwidth - a bit like the way the TCP congestion window works.

Here's my starter for ten:

When selecting the first fragment upon starting to play content, GstAdaptiveDemux selects the Representation with the lowest bitrate for each active AdaptationSet.

GstAdaptiveDemux calculates the effective download rate of each fragment by monitoring the amount of time required to download each fragment and the size of this fragment. It maintains a record of the download rate of the last three fragments for each active AdaptationSet.

GstAdaptiveDemux maintains a record of the amount of compressed data that has been downloaded and has not been rendered, in terms of the amount of time that this compressed data would take to render at normal playback speed. 

NOTE 1: Not sure if that is even possible - would a buffering query do the job?

GstAdaptiveDemux should suspend downloading of fragments if the amount of compressed data that has been downloaded and has not been rendered exceeds buffer_limit seconds.

NOTE 2: buffer_limit would be a configurable property, with a suggested default of 30 seconds

NOTE 3: For MPEG DASH content that uses the live profile, other factors such as availability start time restrictions might mean that the amount of buffered data never reaches the buffer_limit.

Prior to downloading each fragment, GstAdaptiveDemux would use the following rules to select a Representation from an AdaptationSet:
* If less than three fragments have been downloaded, select the Representation with the lowest bitrate, 
* otherwise:
    a) Calculate the mean download rate using the recorded download rates and define this as mrate
    b) Select the Representation that has a bitrate that is less than or equal to bandwidth_usage% of mrate
    c) Calculate the estimated time to download a fragment using the selected Representation:
        est = fragment->duration * representation->bitrate / mrate
    d) If est is greater than the amount of data buffered for this AdaptationSet, select the Representation with the next lowest bitrate. If no suitable Representation is available, select the Representation with the lowest bitrate, otherwise repeat from step (c)
Comment 1 Olivier Crête 2015-01-15 16:51:22 UTC
I don't think it's right to rely on the amount of data buffered by downstream elements, they may be buffering it for other reasons than the "jitter" from the download.

I don't think the buffer_limit mechanism is needed, when the queues are full, then pushing the buffers downstream will block and it should stop downloading more than the window. This should already be there?

Wouldn't he problem be solved by using the same algo as now, but only relying on the average download speed of the last "N" fragments instead of the download speed of the entire video?

I'm not sure I understand why you want to compare the download time of the next fragment to the amount of data that is actually buffered, do you assume that fragments have varying lengths? I would have assume that their lengths are quite stable.
Comment 2 A Ashley 2015-01-19 08:33:52 UTC
(In reply to comment #1)
> 
> I don't think the buffer_limit mechanism is needed, when the queues are full,
> then pushing the buffers downstream will block and it should stop downloading
> more than the window. This should already be there?

Good point. You are right, there's no need for adaptivedemux to enforce a download limit.
> 
> Wouldn't the problem be solved by using the same algo as now, but only relying
> on the average download speed of the last "N" fragments instead of the download
> speed of the entire video?

That would be a good start.

> 
> I'm not sure I understand why you want to compare the download time of the next
> fragment to the amount of data that is actually buffered, do you assume that
> fragments have varying lengths? I would have assume that their lengths are
> quite stable.

The problem that I am trying to avoid is underflow, where the decoder runs out of data.

For example, if fragments are 3 seconds long and there are 3 seconds of video waiting to be decoded+displayed, adaptivedemux should not select a Representation that causes it to spend 6 seconds downloading a 3 second fragment.

adaptivedemux should be able to react to a pending underflow by selecting a lower bitrate Representation.

An alternative to using a downstream query would be for adaptivedemux to detect when it had downloaded a fragment slower than realtime and reacting appropriately by switching to a lower bitrate Representation for the next fragment.
Comment 3 Thiago Sousa Santos 2015-01-19 14:50:54 UTC
I agree that we need a better algorithm. We could have a running window average for the last N fragments (configurable via a property, maybe) and we could have it only go up a bitrate if the average is good enough. Otherwise to go down a bitrate it could just rely on the last fragment to be able to quickly react to a network congestion.

Ultimately we might need to provide different algorithms for different usage and a hook so applications could also plug their own custom algorithms.
Comment 4 Sebastian Dröge (slomo) 2015-01-19 14:58:02 UTC
I think in hlsdemux I implemented some kind of running average once upon a time :) Something like old = (new * 3 + old) / 4. Still not great and something like described above would probably work much better.
Comment 5 Thiago Sousa Santos 2015-01-19 15:48:27 UTC
(In reply to comment #4)
> I think in hlsdemux I implemented some kind of running average once upon a time
> :) Something like old = (new * 3 + old) / 4. Still not great and something like
> described above would probably work much better.

This was unintentionally removed on the port to adaptivedemux (so it is a bug/regression)
Comment 6 A Ashley 2015-01-20 11:24:28 UTC
Sebastian's algorithm would certainly make it more responsive to declining bitrate without the need to keep track of previous fragments. We're currently using that logic, but with the additional tweak of short-circuiting the three fragment pre-load if the download speed is sufficiently high.

I still think it should try to avoid intentional underflow before starting a download.

I wonder if we could do something like keep a track of the GstClockTime when the most recently sent buffer will end (using PTS + duration?). A check of (gst_clock_get_time() + estimated_download_time) <= last_buffer_end_time could be used to detect potential underflow. However, I'm not sure if this logic holds true if sync=false for the sink element.
Comment 7 Thiago Sousa Santos 2015-01-20 12:43:09 UTC
(In reply to comment #6)
> Sebastian's algorithm would certainly make it more responsive to declining
> bitrate without the need to keep track of previous fragments. We're currently
> using that logic, but with the additional tweak of short-circuiting the three
> fragment pre-load if the download speed is sufficiently high.
> 
> I still think it should try to avoid intentional underflow before starting a
> download.
> 
> I wonder if we could do something like keep a track of the GstClockTime when
> the most recently sent buffer will end (using PTS + duration?). A check of
> (gst_clock_get_time() + estimated_download_time) <= last_buffer_end_time could
> be used to detect potential underflow. However, I'm not sure if this logic
> holds true if sync=false for the sink element.

This kind of check makes sense. And I wouldn't worry if the sink is on sync=false because that means that it doesn't care about synchronization, so it shouldn't be worried about underflows.
Comment 8 Mathieu Duponchelle 2015-01-20 16:43:49 UTC
Created attachment 295021 [details] [review]
adaptivedemux: Use a Simple Moving Average to estimate the bitrate.
Comment 9 Mathieu Duponchelle 2015-01-20 16:55:39 UTC
This patch uses the moving average bitrate in every case, which means that subclasses will also go up a bitrate based on it. Using the total average bitrate in case the moving average bitrate is superior to it seems a bit too conservative to me, as it will get less and less reactive with time.

I considered using an exponential moving average, which would have simplified the implementation, but the interval as calculated by g_get_monotonic_time () - stream->download_start_time fluctuates too wildly.

The length of the "lookback window" is hardcoded to 100 buffers for now, should I just #define it or expose it as a property?

As for testing, one can use the RangeHttpServer implemented in gst-devtools/validate/launcher :

Apply this diff: http://www.fpaste.org/172014/21772678/

Then run the server at the root of the directory from which you wish to serve files, in my case :

[ptv] [meh@meh medias]$ pwd
/home/meh/gst-validate/gst-integration-testsuites/medias
[ptv] [meh@meh medias]$ ls
big  defaults  extras-ges
[ptv] [meh@meh medias]$ python /home/meh/devel/pitivi-git/gst-devtools/validate/launcher/RangeHTTPServer.py

And in another terminal :

gst-launch-1.0 playbin uri=http://127.0.0.1:8079/defaults/watermelon/Manifest.mpd

You can then input new bandwidth values, in bytes per second.
Comment 10 Mathieu Duponchelle 2015-01-20 19:36:49 UTC
Created attachment 295043 [details] [review]
adaptivedemux: Use a Simple Moving Average to estimate the bitrate.
Comment 11 Mathieu Duponchelle 2015-01-20 19:37:27 UTC
This patch is rebased on the latest master.
Comment 12 A Ashley 2015-01-22 17:34:53 UTC
100 fragments seems a little large, as that would take up to 5 minutes (using 3 second fragments) to react to a reduction in bitrate.
Comment 13 Mathieu Duponchelle 2015-01-22 17:37:08 UTC
The "time to download" and sizes values are per-buffer, not per-fragment.
Comment 14 Thiago Sousa Santos 2015-01-26 22:34:09 UTC
The problem with doing per-buffer is that, for example, we would be only using the last 409600 bytes for the usual 4096 chunk size and the default 100 chunks.

This would most likely only be using the last part of the downloaded fragment to compute the bitrate as the switches only happen after a fragment is downloaded. Moreover it would be ignoring the latency to set up the connection that happens on the beginning of the fragment download and it could be giving slightly better bitrate than the real one.

So I'd prefer to have this moving average being computed on the full fragment download bitrate, for the last 3 fragments (or other small number). The size of the window could be configurable with a property. We could use the last value to detect congestion and reduce the bitrate immediately and use the average to decide if the bitrate should be increased, for example.
Comment 15 Mathieu Duponchelle 2015-01-27 16:12:15 UTC
Created attachment 295549 [details] [review]
adaptivedemux: track per-fragment bitrates.

And use the average to go up in resolution, and the last fragment
bitrate to go down.

This allows the demuxer to react rapidly to bitrate loss, and
be conservative for bitrate improvements.

+ Add a construct only property to define the number of fragments
  to consider when calculating the average moving bitrate.
Comment 16 Mathieu Duponchelle 2015-01-27 16:12:55 UTC
Review of attachment 295043 [details] [review]:

Obsoleted by next proposed patch
Comment 17 Thiago Sousa Santos 2015-01-28 11:54:07 UTC
Let's wait for other people to comment on this heuristic
Comment 18 Matthieu Bouron 2015-01-28 12:56:33 UTC
*** Bug 737703 has been marked as a duplicate of this bug. ***
Comment 19 A Ashley 2015-02-03 11:04:38 UTC
Review of attachment 295549 [details] [review]:

> static void
>+gst_adaptive_demux_set_property (GObject * object, guint prop_id,
>+    const GValue * value, GParamSpec * pspec)
>+{
>+  GstAdaptiveDemux *demux = GST_ADAPTIVE_DEMUX (object);
>+
>+  switch (prop_id) {
>+    case PROP_LOOKBACK_FRAGMENTS:
>+      demux->num_lookback_fragments = g_value_get_uint (value);
>+      break;
>+    default:
>+      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
>+      break;
>+  }
>+}
>+

Probably should check if demux has already been started. Given the current
code, increasing the buffer size after start will cause a buffer overflow.
I think that either the property should not be changeable after start, or the
buffer needs to re-allocated if the property is changed.

> static guint64
>-gst_adaptive_demux_stream_update_current_bitrate (GstAdaptiveDemuxStream *
>-    stream)
>+_update_average_bitrate (GstAdaptiveDemux * demux,
>+    GstAdaptiveDemuxStream * stream, guint64 new_bitrate)
>+{
>+  gint index = stream->moving_index % demux->num_lookback_fragments;
>+
>+  stream->moving_bitrate -= stream->fragment_bitrates[index];
>+  stream->fragment_bitrates[index] = new_bitrate;
>+  stream->moving_bitrate += new_bitrate;
>+
>+  stream->moving_index += 1;
>+
>+  if (stream->moving_index > demux->num_lookback_fragments)
>+    return stream->moving_bitrate / demux->num_lookback_fragments;
>+  return stream->moving_bitrate / stream->moving_index;
>+}

Should this be reset after a seek?
Comment 20 A Ashley 2015-02-03 11:16:51 UTC
Created attachment 296007 [details]
adaptivedemux log on a line restricted to 2Mbps

I am seeing problems at start up when using a link that has been limited to 2Mbps. It switches between playing and buffering for almost a minute before finally entering a steady state.

I am testing with the stream:
http://rdmedia.bbc.co.uk/dash/ondemand/bbb/avc3/1/client_manifest-common_init.mpd

The bitrates in this stream are:
    V1  347Kbps
    V2  604Kbps
    V3 1299Kbps
    V4 2442Kbps
    V5 4382Kbps

The log of every fragment downloaded (the @204800 indicates the throughput):
2015-02-03T11:06:49.859Z/dash/ondemand/bbb/avc3/1/client_manifest-common_init.mpd @ 204800
2015-02-03T11:06:49.897Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:06:49.900Z/dash/ondemand/bbb/avc3/1/A2-aud.mp4/IS @ 204800
2015-02-03T11:06:50.101Z/dash/ondemand/bbb/avc3/1/V1.mp4/1 @ 204800
2015-02-03T11:06:50.111Z/dash/ondemand/bbb/avc3/1/A2-aud.mp4/1 @ 204800
2015-02-03T11:06:50.321Z/dash/ondemand/bbb/avc3/1/A2-aud.mp4/IS @ 204800
2015-02-03T11:06:50.407Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:06:50.532Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/2 @ 204800
2015-02-03T11:06:50.613Z/dash/ondemand/bbb/avc3/1/V2.mp4/2 @ 204800
2015-02-03T11:06:50.842Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/3 @ 204800
2015-02-03T11:06:51.156Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/4 @ 204800
2015-02-03T11:06:51.464Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/5 @ 204800
2015-02-03T11:06:51.622Z/dash/ondemand/bbb/avc3/1/V2.mp4/3 @ 204800
2015-02-03T11:06:51.774Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/6 @ 204800
2015-02-03T11:06:52.084Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/7 @ 204800
2015-02-03T11:06:52.334Z/dash/ondemand/bbb/avc3/1/V2.mp4/4 @ 204800
2015-02-03T11:06:52.393Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/8 @ 204800
2015-02-03T11:06:52.640Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:06:52.702Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/9 @ 204800
2015-02-03T11:06:52.852Z/dash/ondemand/bbb/avc3/1/V3.mp4/5 @ 204800
2015-02-03T11:06:53.011Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/10 @ 204800
2015-02-03T11:06:53.320Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/11 @ 204800
2015-02-03T11:06:53.627Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/12 @ 204800
2015-02-03T11:06:53.937Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/13 @ 204800
2015-02-03T11:06:54.246Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/14 @ 204800
2015-02-03T11:06:54.555Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/15 @ 204800
2015-02-03T11:06:54.871Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/16 @ 204800
2015-02-03T11:06:55.181Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/17 @ 204800
2015-02-03T11:06:55.459Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:06:55.494Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/18 @ 204800
2015-02-03T11:06:55.677Z/dash/ondemand/bbb/avc3/1/V2.mp4/6 @ 204800
2015-02-03T11:06:55.803Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/19 @ 204800
2015-02-03T11:06:56.111Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/20 @ 204800
2015-02-03T11:06:56.286Z/dash/ondemand/bbb/avc3/1/V2.mp4/7 @ 204800
2015-02-03T11:06:56.419Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/21 @ 204800
2015-02-03T11:06:56.697Z/dash/ondemand/bbb/avc3/1/V2.mp4/8 @ 204800
2015-02-03T11:06:56.734Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/22 @ 204800
2015-02-03T11:06:57.003Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:06:57.050Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/23 @ 204800
2015-02-03T11:06:57.212Z/dash/ondemand/bbb/avc3/1/V3.mp4/9 @ 204800
2015-02-03T11:06:57.365Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/24 @ 204800
2015-02-03T11:06:57.675Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/25 @ 204800
2015-02-03T11:06:57.984Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/26 @ 204800
2015-02-03T11:06:58.293Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/27 @ 204800
2015-02-03T11:06:58.602Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/28 @ 204800
2015-02-03T11:06:58.910Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/29 @ 204800
2015-02-03T11:06:59.221Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/30 @ 204800
2015-02-03T11:06:59.529Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/31 @ 204800
2015-02-03T11:06:59.837Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/32 @ 204800
2015-02-03T11:07:00.118Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:00.148Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/33 @ 204800
2015-02-03T11:07:00.322Z/dash/ondemand/bbb/avc3/1/V2.mp4/10 @ 204800
2015-02-03T11:07:00.459Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/34 @ 204800
2015-02-03T11:07:00.771Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/35 @ 204800
2015-02-03T11:07:01.032Z/dash/ondemand/bbb/avc3/1/V2.mp4/11 @ 204800
2015-02-03T11:07:01.081Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/36 @ 204800
2015-02-03T11:07:01.390Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/37 @ 204800
2015-02-03T11:07:01.642Z/dash/ondemand/bbb/avc3/1/V2.mp4/12 @ 204800
2015-02-03T11:07:01.699Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/38 @ 204800
2015-02-03T11:07:02.007Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/39 @ 204800
2015-02-03T11:07:02.325Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/40 @ 204800
2015-02-03T11:07:02.552Z/dash/ondemand/bbb/avc3/1/V2.mp4/13 @ 204800
2015-02-03T11:07:02.635Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/41 @ 204800
2015-02-03T11:07:02.944Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/42 @ 204800
2015-02-03T11:07:03.253Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/43 @ 204800
2015-02-03T11:07:03.363Z/dash/ondemand/bbb/avc3/1/V2.mp4/14 @ 204800
2015-02-03T11:07:03.563Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/44 @ 204800
2015-02-03T11:07:03.771Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:03.872Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/45 @ 204800
2015-02-03T11:07:03.984Z/dash/ondemand/bbb/avc3/1/V3.mp4/15 @ 204800
2015-02-03T11:07:04.181Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/46 @ 204800
2015-02-03T11:07:04.490Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/47 @ 204800
2015-02-03T11:07:04.802Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/48 @ 204800
2015-02-03T11:07:05.109Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/49 @ 204800
2015-02-03T11:07:05.418Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/50 @ 204800
2015-02-03T11:07:05.728Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/51 @ 204800
2015-02-03T11:07:05.993Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:06.039Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/52 @ 204800
2015-02-03T11:07:06.203Z/dash/ondemand/bbb/avc3/1/V2.mp4/16 @ 204800
2015-02-03T11:07:06.349Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/53 @ 204800
2015-02-03T11:07:06.658Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/54 @ 204800
2015-02-03T11:07:06.915Z/dash/ondemand/bbb/avc3/1/V2.mp4/17 @ 204800
2015-02-03T11:07:06.968Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/55 @ 204800
2015-02-03T11:07:07.277Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/56 @ 204800
2015-02-03T11:07:07.587Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/57 @ 204800
2015-02-03T11:07:07.724Z/dash/ondemand/bbb/avc3/1/V2.mp4/18 @ 204800
2015-02-03T11:07:07.935Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/58 @ 204800
2015-02-03T11:07:08.246Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/59 @ 204800
2015-02-03T11:07:08.335Z/dash/ondemand/bbb/avc3/1/V2.mp4/19 @ 204800
2015-02-03T11:07:08.544Z/dash/ondemand/bbb/avc3/1/V2.mp4/20 @ 204800
2015-02-03T11:07:08.555Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/60 @ 204800
2015-02-03T11:07:08.864Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/61 @ 204800
2015-02-03T11:07:09.174Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/62 @ 204800
2015-02-03T11:07:09.485Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/63 @ 204800
2015-02-03T11:07:09.794Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/64 @ 204800
2015-02-03T11:07:09.856Z/dash/ondemand/bbb/avc3/1/V2.mp4/21 @ 204800
2015-02-03T11:07:10.104Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/65 @ 204800
2015-02-03T11:07:10.413Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/66 @ 204800
2015-02-03T11:07:10.721Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/67 @ 204800
2015-02-03T11:07:10.962Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:11.030Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/68 @ 204800
2015-02-03T11:07:11.174Z/dash/ondemand/bbb/avc3/1/V3.mp4/22 @ 204800
2015-02-03T11:07:11.340Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/69 @ 204800
2015-02-03T11:07:11.651Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/70 @ 204800
2015-02-03T11:07:11.961Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/71 @ 204800
2015-02-03T11:07:12.269Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/72 @ 204800
2015-02-03T11:07:12.580Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/73 @ 204800
2015-02-03T11:07:12.889Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/74 @ 204800
2015-02-03T11:07:12.981Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:13.193Z/dash/ondemand/bbb/avc3/1/V2.mp4/23 @ 204800
2015-02-03T11:07:13.198Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/75 @ 204800
2015-02-03T11:07:13.509Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/76 @ 204800
2015-02-03T11:07:13.603Z/dash/ondemand/bbb/avc3/1/V2.mp4/24 @ 204800
2015-02-03T11:07:13.819Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/77 @ 204800
2015-02-03T11:07:13.914Z/dash/ondemand/bbb/avc3/1/V2.mp4/25 @ 204800
2015-02-03T11:07:14.127Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/78 @ 204800
2015-02-03T11:07:14.437Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/79 @ 204800
2015-02-03T11:07:14.624Z/dash/ondemand/bbb/avc3/1/V2.mp4/26 @ 204800
2015-02-03T11:07:14.745Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/80 @ 204800
2015-02-03T11:07:15.054Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/81 @ 204800
2015-02-03T11:07:15.131Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:15.348Z/dash/ondemand/bbb/avc3/1/V3.mp4/27 @ 204800
2015-02-03T11:07:15.364Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/82 @ 204800
2015-02-03T11:07:15.674Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/83 @ 204800
2015-02-03T11:07:15.983Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/84 @ 204800
2015-02-03T11:07:16.293Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/85 @ 204800
2015-02-03T11:07:16.605Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/86 @ 204800
2015-02-03T11:07:16.914Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/87 @ 204800
2015-02-03T11:07:17.223Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/88 @ 204800
2015-02-03T11:07:17.537Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/89 @ 204800
2015-02-03T11:07:17.847Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/90 @ 204800
2015-02-03T11:07:18.055Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:18.157Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/91 @ 204800
2015-02-03T11:07:18.267Z/dash/ondemand/bbb/avc3/1/V2.mp4/28 @ 204800
2015-02-03T11:07:18.475Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/92 @ 204800
2015-02-03T11:07:18.784Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/93 @ 204800
2015-02-03T11:07:19.094Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/94 @ 204800
2015-02-03T11:07:19.279Z/dash/ondemand/bbb/avc3/1/V2.mp4/29 @ 204800
2015-02-03T11:07:19.407Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/95 @ 204800
2015-02-03T11:07:19.718Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/96 @ 204800
2015-02-03T11:07:20.027Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/97 @ 204800
2015-02-03T11:07:20.192Z/dash/ondemand/bbb/avc3/1/V2.mp4/30 @ 204800
2015-02-03T11:07:20.336Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/98 @ 204800
2015-02-03T11:07:20.645Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/99 @ 204800
2015-02-03T11:07:20.953Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/100 @ 204800
2015-02-03T11:07:21.207Z/dash/ondemand/bbb/avc3/1/V2.mp4/31 @ 204800
2015-02-03T11:07:21.261Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/101 @ 204800
2015-02-03T11:07:21.571Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/102 @ 204800
2015-02-03T11:07:21.882Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/103 @ 204800
2015-02-03T11:07:22.192Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/104 @ 204800
2015-02-03T11:07:22.220Z/dash/ondemand/bbb/avc3/1/V2.mp4/32 @ 204800
2015-02-03T11:07:22.502Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/105 @ 204800
2015-02-03T11:07:22.812Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/106 @ 204800
2015-02-03T11:07:22.931Z/dash/ondemand/bbb/avc3/1/V2.mp4/33 @ 204800
2015-02-03T11:07:23.121Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/107 @ 204800
2015-02-03T11:07:23.431Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/108 @ 204800
2015-02-03T11:07:23.438Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:23.643Z/dash/ondemand/bbb/avc3/1/V3.mp4/34 @ 204800
2015-02-03T11:07:23.742Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/109 @ 204800
2015-02-03T11:07:24.054Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/110 @ 204800
2015-02-03T11:07:24.363Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/111 @ 204800
2015-02-03T11:07:24.671Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/112 @ 204800
2015-02-03T11:07:24.979Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/113 @ 204800
2015-02-03T11:07:25.288Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/114 @ 204800
2015-02-03T11:07:25.596Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/115 @ 204800
2015-02-03T11:07:25.909Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/116 @ 204800
2015-02-03T11:07:26.054Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:26.219Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/117 @ 204800
2015-02-03T11:07:26.265Z/dash/ondemand/bbb/avc3/1/V2.mp4/35 @ 204800
2015-02-03T11:07:26.531Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/118 @ 204800
2015-02-03T11:07:26.774Z/dash/ondemand/bbb/avc3/1/V2.mp4/36 @ 204800
2015-02-03T11:07:26.839Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/119 @ 204800
2015-02-03T11:07:27.150Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/120 @ 204800
2015-02-03T11:07:27.459Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/121 @ 204800
2015-02-03T11:07:27.486Z/dash/ondemand/bbb/avc3/1/V2.mp4/37 @ 204800
2015-02-03T11:07:27.701Z/dash/ondemand/bbb/avc3/1/V2.mp4/38 @ 204800
2015-02-03T11:07:27.769Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/122 @ 204800
2015-02-03T11:07:28.080Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/123 @ 204800
2015-02-03T11:07:28.208Z/dash/ondemand/bbb/avc3/1/V2.mp4/39 @ 204800
2015-02-03T11:07:28.391Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/124 @ 204800
2015-02-03T11:07:28.701Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/125 @ 204800
2015-02-03T11:07:28.914Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:29.010Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/126 @ 204800
2015-02-03T11:07:29.129Z/dash/ondemand/bbb/avc3/1/V3.mp4/40 @ 204800
2015-02-03T11:07:29.320Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/127 @ 204800
2015-02-03T11:07:29.631Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/128 @ 204800
2015-02-03T11:07:29.942Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/129 @ 204800
2015-02-03T11:07:30.136Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:30.252Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/130 @ 204800
2015-02-03T11:07:30.344Z/dash/ondemand/bbb/avc3/1/V2.mp4/41 @ 204800
2015-02-03T11:07:30.564Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/131 @ 204800
2015-02-03T11:07:30.874Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/132 @ 204800
2015-02-03T11:07:31.053Z/dash/ondemand/bbb/avc3/1/V2.mp4/42 @ 204800
2015-02-03T11:07:31.187Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/133 @ 204800
2015-02-03T11:07:31.496Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/134 @ 204800
2015-02-03T11:07:31.665Z/dash/ondemand/bbb/avc3/1/V2.mp4/43 @ 204800
2015-02-03T11:07:31.806Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/135 @ 204800
2015-02-03T11:07:32.116Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/136 @ 204800
2015-02-03T11:07:32.424Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/137 @ 204800
2015-02-03T11:07:32.575Z/dash/ondemand/bbb/avc3/1/V2.mp4/44 @ 204800
2015-02-03T11:07:32.736Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/138 @ 204800
2015-02-03T11:07:33.059Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/139 @ 204800
2015-02-03T11:07:33.369Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/140 @ 204800
2015-02-03T11:07:33.382Z/dash/ondemand/bbb/avc3/1/V5.mp4/IS @ 204800
2015-02-03T11:07:33.600Z/dash/ondemand/bbb/avc3/1/V3.mp4/45 @ 204800
2015-02-03T11:07:33.679Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/141 @ 204800
2015-02-03T11:07:33.989Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/142 @ 204800
2015-02-03T11:07:34.298Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/143 @ 204800
2015-02-03T11:07:34.607Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/144 @ 204800
2015-02-03T11:07:34.916Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/145 @ 204800
2015-02-03T11:07:35.228Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/146 @ 204800
2015-02-03T11:07:35.536Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/147 @ 204800
2015-02-03T11:07:35.845Z/dash/ondemand/bbb/avc3/1/A4-aud.mp4/148 @ 204800
Comment 21 Mathieu Duponchelle 2015-02-03 11:41:14 UTC
(In reply to comment #19)
> Review of attachment 295549 [details] [review]:
> 
> > static void
> >+gst_adaptive_demux_set_property (GObject * object, guint prop_id,
> >+    const GValue * value, GParamSpec * pspec)
> >+{
> >+  GstAdaptiveDemux *demux = GST_ADAPTIVE_DEMUX (object);
> >+
> >+  switch (prop_id) {
> >+    case PROP_LOOKBACK_FRAGMENTS:
> >+      demux->num_lookback_fragments = g_value_get_uint (value);
> >+      break;
> >+    default:
> >+      G_OBJECT_WARN_INVALID_PROPERTY_ID (object, prop_id, pspec);
> >+      break;
> >+  }
> >+}
> >+
> 
> Probably should check if demux has already been started. Given the current
> code, increasing the buffer size after start will cause a buffer overflow.
> I think that either the property should not be changeable after start, or the
> buffer needs to re-allocated if the property is changed.

The propery is indeed declared as CONSTRUCT_ONLY

> > static guint64
> >-gst_adaptive_demux_stream_update_current_bitrate (GstAdaptiveDemuxStream *
> >-    stream)
> >+_update_average_bitrate (GstAdaptiveDemux * demux,
> >+    GstAdaptiveDemuxStream * stream, guint64 new_bitrate)
> >+{
> >+  gint index = stream->moving_index % demux->num_lookback_fragments;
> >+
> >+  stream->moving_bitrate -= stream->fragment_bitrates[index];
> >+  stream->fragment_bitrates[index] = new_bitrate;
> >+  stream->moving_bitrate += new_bitrate;
> >+
> >+  stream->moving_index += 1;
> >+
> >+  if (stream->moving_index > demux->num_lookback_fragments)
> >+    return stream->moving_bitrate / demux->num_lookback_fragments;
> >+  return stream->moving_bitrate / stream->moving_index;
> >+}
> 
> Should this be reset after a seek?

I would think this info is still significant after a seek.
Comment 22 Mathieu Duponchelle 2015-02-03 11:44:07 UTC
(In reply to comment #20)
> Created an attachment (id=296007) [details]
> adaptivedemux log on a line restricted to 2Mbps
> 
> I am seeing problems at start up when using a link that has been limited to
> 2Mbps. It switches between playing and buffering for almost a minute before
> finally entering a steady state.
> 
> I am testing with the stream:
> http://rdmedia.bbc.co.uk/dash/ondemand/bbb/avc3/1/client_manifest-common_init.mpd
> 

I'll have a look, how exactly do you restrict the bandwidth?
Comment 23 A Ashley 2015-02-03 15:08:07 UTC
Created attachment 296028 [details]
Another GStreamer log, with --gst-debug=adaptivedemux:5,dashdemux:5,qtdemux:2,3

Another log file, this time including dashdemux DEBUG level.
Comment 24 A Ashley 2015-02-03 15:14:48 UTC
Created attachment 296029 [details]
Log of each HTTP request

Log of each HTTP request that was made when adaptive-demux-abr-log-v2.txt was created.
Comment 25 Mathieu Duponchelle 2015-02-03 15:44:50 UTC
OK there's a bunch of interesting things in this log, first thing I notice is :

0:00:00.148947972  3618  0x8b17430 DEBUG          adaptivedemux gstadaptivedemux.c:653:gst_adaptive_demux_expose_stream:<dashdemux0> Adding srcpad '':video_00 with caps audio/x-m4a

which is a bit confusing :)

I also notice this :

0:00:01.793368579  [...]Last 3 fragments average bitrate is 898837
0:00:01.793655216  [...] mediaURL = V2.mp4/3
0:00:01.793727944  [...]Downloading uri: http://rdmedia.bbc.co.uk/dash/ondemand/bbb/avc3/1/V5.mp4/IS, range:0 - -1

This shows that the patch correctly calculates the bitrate, and the subclass selects an appropriate representation, but I'm not sure why the V5 representation is downloaded anyway.

I'll look into these issues, but it would still be helpful to know how you limit the bandwidth on your end so I can reproduce Alex ?
Comment 26 Mathieu Duponchelle 2015-02-03 17:19:37 UTC
(In reply to comment #23)
> Created an attachment (id=296028) [details]
> Another GStreamer log, with --gst-debug=adaptivedemux:5,dashdemux:5,qtdemux:2,3
> 
> Another log file, this time including dashdemux DEBUG level.

OK, I just realized that both of the requests for V5.mp4 are header requests, the actual fragments that are downloaded are always V1 / V2, which is correct. There is no issue here with my patch, which correctly selects the appropriate representations for the available bandwidth.

Apart from that, I unfortunately can't reproduce the buffering issue that you describe.
Comment 27 A Ashley 2015-02-06 10:25:37 UTC
I think we should accept Mathieu's patch and close this ticket. His proposed patch is a lot better than what is currently in adaptivedemux and provides a fairly conservative estimate for available bandwidth.

If/when I am able to create a stand alone test that reproduces the buffering issue, I will create a new ticket.
Comment 28 Thiago Sousa Santos 2015-02-11 22:58:31 UTC
Agreed. Just reviewed the patch and it looks good enough.

Mathieu, can you merge it, please?
Comment 29 Mathieu Duponchelle 2015-02-11 23:08:17 UTC
Review of attachment 295549 [details] [review]:

commit c98348c1417fbe6cabe0fd4dfcb3d65acd8d5c81
Author: Mathieu Duponchelle <mathieu.duponchelle@opencreed.com>
Date:   Tue Jan 27 13:48:42 2015 +0100

    adaptivedemux: track per-fragment bitrates.
    
    And use the average to go up in resolution, and the last fragment
    bitrate to go down.
    
    This allows the demuxer to react rapidly to bitrate loss, and
    be conservative for bitrate improvements.
    
    + Add a construct only property to define the number of fragments
      to consider when calculating the average moving bitrate.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=742979