After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 758960 - New GstStream API and decodebin3/playbin3
New GstStream API and decodebin3/playbin3
Product: GStreamer
Classification: Platform
Component: gstreamer (core)
git master
Other Linux
: Normal enhancement
: 1.9.1
Assigned To: GStreamer Maintainers
GStreamer Maintainers
: 703846 733235 (view as bug list)
Depends on:
Blocks: 667217 755220
Reported: 2015-12-02 16:20 UTC by Edward Hervey
Modified: 2017-02-01 14:43 UTC
See Also:
GNOME target: ---
GNOME version: ---

Description Edward Hervey 2015-12-02 16:20:55 UTC
This is to track the proposal to get the new GstStream API and decodebin3/playbin3 into upstream.

The code is currently here:

The best place to look for more info are:
* the design docs in core (for the streams API) and base (for decodebin3/playbin3)
* the recording of the talk presenting it :  along with the slides
* and obviously the code :)

Fast-track to start using it: compile both core/base from the streams branch, you can activate playbin3/decodebin3 on any gstreamer application by using the USE_PLAYBIN3=1 env variable.

The goal is *NOT* to push this upstream for 1.8. While the code does work, there are still a lot of small features that still need to be thought/implemented before.
Comment 1 Carlos Rafael Giani 2016-02-22 15:54:32 UTC
I'd add as something for the new uridecodebin3 .

Also, currently, GstURIHandler only accepts a URI string in its set_uri() vfunc. As a consequence, the current uridecodebin and playbin elements only have a "uri" property. However, sometimes, additional media specific data needs to be passed to the source element. One example would be credentials.

Two methods of accomplishing this is:

1) serialize the additional data as a string and append it to the URI
2) pass on the data in the "source-setup" signal

Method #1 adds extra overhead by the serialization, which can be significant depending on the size of the extra data. Also, some data sets simply cannot be serialized.
Method #2 requires one source-setup callback to contain if-elseif-blocks to cover all of the sources that need extra data, which doesn't scale well.

Therefore, a better option would be to add a GstStructure alongside the URI:

* GstURIHandler would get a new "set_uri_full()" vfunc, with an additional "GstStructure *extra_data" argument.

* A new gst_element_make_from_uri_full() would be added, with an additional "GstStructure *extra_data" argument right after the "uri" argument.

* uridecodebin3, urisourcebin, and playbin3 would get "extra-data" properties, which would allow for passing a GstStructure pointer.

gst_element_make_from_uri_full() would call set_uri() if set_uri_full is NULL. This maintains backwards compatibility with older elements that implement GstURIHandler and set the set_uri vfunc. (Perhaps the src_uri() vfunc should also be marked as deprecated.)
gst_element_make_from_uri() would internally just call gst_element_make_from_uri_full() with extra_data set to NULL.

It is currently unclear if any of these functions/properties above would take ownership over the GstStructure.
Comment 2 Tim-Philipp Müller 2016-02-22 17:44:48 UTC
As I see it, all these uri handling propositions are completely orthogonal to decodebin3/playbin3 and the stream API (it's just an extension of some existing mechanism, it could be added to playbin just like playbin3, there's nothing in it that relates to the Stream API nor anything that relates to playbin3 specifically..)
Comment 3 Carlos Rafael Giani 2016-02-22 21:22:45 UTC
The same applies to my linked entry 762125. The point is to introduce this to playbin3/uridecodebin3/urisourcebin right from the start. But you are right, this could also be added to existing playbin/uridecodebin.
Comment 4 Carlos Rafael Giani 2016-05-24 10:53:00 UTC
Next topic: How will playbin3/decodebin3 handle gapless playback? I don't just mean the immediate switch from the next to the current track. Some form of preparing/prebuffering is absolutely vital to gapless.

The way playbin currently does it is insufficient in many cases. For example, I've observed that when trying to play a FLAC HD 192 kHz 24 bit gapless album froma  DLNA server on some embedded platforms, prebuffering had to start 10 seconds before the current track ended, otherwise the the network buffer would not be 100% full by the time the switch happened. playbin relied on ring buffers covering the switch period.

Also, during prebuffering, the pipeline is essentially running 2 decoder chains in parallel, which of course increases CPU usage, and the next track's prebuffering runs as fast as it can, further increasing CPU%. So, some form of throttling may be good. Also, with prebuffering mechanisms, parsers have to be able to respond to duration&position queries to know that a track is about to end in N seconds.

Generally I strongly recommend to develop this on single-core embedded platforms like a BeagleBoard to catch as many corner cases as possible. Do not rely on PC figures.
Comment 5 Nicolas Dufresne (ndufresne) 2016-05-24 11:29:39 UTC
I believe in decodebin3, we buffer before the decoder. We should not have to run multiple decoder in parallel. This is particularly needed on embedded, where you may not have multiple instances of the same decoder.
Comment 6 Carlos Rafael Giani 2016-05-24 12:03:50 UTC
So you mean prebuffering without decoding. Yes, this is the ideal. I think a source -> networkbuffer -> parser -> queue -> probe. chain would be enough for prebuffering. The parser is necessary, otherwise there will be no bitrate estimate, and without a bitrate, the network buffer's size cannot be readjusted properly. The probe -> queue chain is necessary in case the parser does not figure out the bitrate right away (some parsers need a couple of frames until they produce a bitrate tag). In this case, the probe would block until the bitrate is seen. Once it is seen, it no longer blocks, and the queue's maximum buffer count is set to 1.
Comment 7 Edward Hervey 2016-06-02 07:43:12 UTC
*** Bug 703846 has been marked as a duplicate of this bug. ***
Comment 8 Edward Hervey 2016-06-30 12:51:18 UTC
commit d514e79beed868067e1f936bc0159a4a6a8564e4
Author: Edward Hervey <>
Date:   Wed Jun 29 18:14:51 2016 +0200

    playback: New elements
    With contributions from Jan Schmidt <>
    * decodebin3 and playbin3 have the same purpose as the decodebin and
    playbin elements, except make usage of more 1.x features and the new
    GstStream API. This allows them to be more memory/cpu efficient.
    * parsebin is a new element that demuxers/depayloads/parses an incoming
    stream and exposes elementary streams. It is used by decodebin3.
    It also automatically creates GstStream and GstStreamCollection for
    elements that don't natively create them and sends the corresponding
    events and messages
    * Any application using playbin can use playbin3 by setting the env
    variable USE_PLAYBIN3=1 without reconfiguration/recompilation.

commit 63f6f05d66537877365a1099ac176dc3afbb9f6b
Author: Edward Hervey <>
Date:   Fri Jun 12 10:53:23 2015 +0200

    gst: New Stream listing/selection system
    * GstStream
    * GstStreamCollection
Comment 9 Edward Hervey 2017-02-01 14:43:28 UTC
*** Bug 733235 has been marked as a duplicate of this bug. ***