After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 598533 - [decodebin2] Post element message with the stream topology on the bus
[decodebin2] Post element message with the stream topology on the bus
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal normal
: 0.10.26
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2009-10-15 11:19 UTC by Sebastian Dröge (slomo)
Modified: 2009-10-28 13:16 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
decodebin2: Store for every chain, which pad resulted in it's creation (3.05 KB, patch)
2009-10-15 11:20 UTC, Sebastian Dröge (slomo)
committed Details | Review
decodebin2: Store the "endcaps" of a chain (3.68 KB, patch)
2009-10-15 11:20 UTC, Sebastian Dröge (slomo)
committed Details | Review
decodebin2: Post a element message on the bus with the stream topology (4.77 KB, patch)
2009-10-15 11:20 UTC, Sebastian Dröge (slomo)
committed Details | Review

Description Sebastian Dröge (slomo) 2009-10-15 11:19:42 UTC
Attached patches to what the summary says, please review. Edward can explain why we want this ;)
Comment 1 Sebastian Dröge (slomo) 2009-10-15 11:20:17 UTC
Created attachment 145500 [details] [review]
decodebin2: Store for every chain, which pad resulted in it's creation
Comment 2 Sebastian Dröge (slomo) 2009-10-15 11:20:25 UTC
Created attachment 145501 [details] [review]
decodebin2: Store the "endcaps" of a chain

This are the caps that either resulted in a deadend if
no plugin for them could be found or raw caps.
Comment 3 Sebastian Dröge (slomo) 2009-10-15 11:20:32 UTC
Created attachment 145502 [details] [review]
decodebin2: Post a element message on the bus with the stream topology
Comment 4 Edward Hervey 2009-10-15 11:25:33 UTC
Apart from the fact that the 3rd patch could be speeded up by using Quarks and avoiding all the structure copies in gst_structure_set ... it looks good to me !

Commit ! :)
Comment 5 Wim Taymans 2009-10-15 11:36:17 UTC
Isn't there a more generic way to get this info? what exactly is it doing?
Comment 6 Sebastian Dröge (slomo) 2009-10-15 11:38:26 UTC
commit efcca84bac5e5351040a9557321041348450216f
Author: Sebastian Dröge <sebastian.droege@collabora.co.uk>
Date:   Thu Oct 15 13:13:43 2009 +0200

    decodebin2: Post a element message on the bus with the stream topology
    
    Fixes bug #598533.

commit 50fdbcd9ea9ba058d695b2826fb97690536faebc
Author: Sebastian Dröge <sebastian.droege@collabora.co.uk>
Date:   Thu Oct 15 13:01:01 2009 +0200

    decodebin2: Store the "endcaps" of a chain
    
    This are the caps that either resulted in a deadend if
    no plugin for them could be found or raw caps.

commit 366aaae8257e4c934f2be152aa9a15f3386925a1
Author: Sebastian Dröge <sebastian.droege@collabora.co.uk>
Date:   Thu Oct 15 11:38:14 2009 +0200

    decodebin2: Store for every chain, which pad resulted in its creation
Comment 7 Wim Taymans 2009-10-15 11:44:59 UTC
It looks like it iterates the elements from the srcpads and enumerates the caps in a tree. Surely this can be done on any pipeline with a helper method, no?
Comment 8 Stefan Sauer (gstreamer, gtkdoc dev) 2009-10-27 13:09:52 UTC
I agree with wim. I'd also like to know about the usecase here, instead of pinging Edward on IRC.
Comment 9 Edward Hervey 2009-10-28 11:47:25 UTC
Right now, applications who want to know the following information about a URI need to write a lot of code:
* How many streams does it have ?
* What kind of streams (audio/video/text/private/...) does it have ?
* What are the various properties of those streams (ex : for video, what's the width/height/framerate/par/interlacing/native-colorspace/...)
* What's the container format (in GstCaps and not in mystical string tags coming from somewhere) ?
* What are the codecs used in each stream (same comment as for containers) ? 
* What's the duration of the URI ?
* What are the tags used ?

In PiTiVi and some other gst-python-based applications, we have at least the discoverer which helps a little bit, but it's not usable from non-python applications.

All that information can be used for many different use-cases (ranging from simple to more advanced):
* Desktop information (hover/right-click over the file to get the multimedia info)
* Playback systems (in order to show a lot more information)
* Application doing decision making based on the type of streams available and/or require the duration (tools using gnonlin that need to know which streams are available and can use, and what the duration is)
* Trans-coding/trans-muxing so you can use the information to create the proper encoding/muxing elements behind

Other side effects:
* Have a *unique* way of describing the container format and the codec formats being used, since you get the *actual* caps and you can then use gstpbutils, instead on relying on elements to provide a string (which they might not do, or they might do differently, ...).
* If you have such a topology... you can then create a pipeline that can encode/mux files that will end up being (closely) similar, by taking that topology tree and reverting it (i.e. source becomes sink, demuxer becomes muxer, ...).


Right now, there is only one thing which is non-trivial to get from the list at the top of this comment, and that is the stream topology of a URI.

I thought that since decodebin2 is already constructing a consequent part of the pipeline, we could just have it store that information and emit it as a message.
The problem is:
* people are complaining about the uglyness of this
* GstStructure is a serious PITA to work with, and even more so when you start having nested structures


So what I propose instead is a generic API to be able to get the stream topology of a GstBin (and therefore a GstPipeline).
The idea is to end up with a tree-like structure/object that applications or other libraries can then re-use to accomplish the goals stated at the top of this comment.

I'd like to state that this can not be done for all pipelines, but only for pipelines that have branching-out-only topologies, or mixing-in-topology (i.e. pipelines like src ! demuxer ! many decoders !..., or many streams ! muxer ! sink). So any pipeline whose stream structure can be represented as a tree. Trying to achieve this stream topology for any kind of pipeline would mean creating a system that allows branching-then-mixing-back structures :(

Why STREAM topology (and not just topology) ?
* Because it would only contain the chain of identical caps, and where they split-up/aggregate, but NOT the elements. If you want the topology of the elements... you can just iterate the elements in a pipeline.

------
A proposed API could be the following:

struct GstStreamInformation {
  GstStreamType         streamtype;  // simple stream or container
  GstCaps              *caps;        // the caps of the stream
  GstStreamInformation *previous;    // The upstream stream (can be NULL)
  GstStreamInformation *next;        // The downstream stream (can be NULL)
};

struct GstStreamContainerInformation {
  GstStreamInformation  parent;
  GHashTable           *streams;     // the streams this container provides
                                     // or aggregates
                                     // key : unique key (can be pad name)
                                     // value: GstStreamInformation*
}

/**
 * gst_bin_get_topology:
 * @bin: a #GstBin
 * @topology: Will contain a pointer to the topology if successfull
 *
 * Gets the topology of the bin.
 * 
 * Returns: TRUE if the topology could be determined, else FALSE
 */
gboolean gst_bin_get_topology (GstBin * bin, GstStreamInformation **topology);
------
Comment 10 Tim-Philipp Müller 2009-10-28 12:20:13 UTC
I think you should concentrate on the non-playback application use cases here.

Because no playback application in the world is going to be interested in all this cr^Hinformation. It's simply a case of information overload. For a playback app aggregation of information is key, and topology is completely uninteresting (besides a basic notion of 'streams' as in 'streams that can be selected or audio/video).
Comment 11 Wim Taymans 2009-10-28 12:47:09 UTC
To me it sounds like (at the lowest level) you want an API to walk the pipeline graph so that you can collect information about each element/pad you encounter.

This is all pretty straightforward when using the internally linked pads of an element. Problems arrise when you come to an element that splits or merges 'streams' because then you have to decide which path to follow or that streams end.

The proposed API seems to implement a subset of this functionality. It only provides the caps (no elements, no pads). This might be enough for your current use case but I don't think we can call this a generic API. 

I think you should focus on making decodebin2 provide you with the info you need right now instead of trying to solve this problem generically. I believe that's why I was wondereding about but not too opposed to the message that decodebin2 now posts.
Comment 12 Sebastian Dröge (slomo) 2009-10-28 13:16:46 UTC
Let's keep the original bug closed as fixed then please ;)

Because a generic solution is not easy and can't provide the info needed for your use cases, back then we decided to leave this bug closed as is and let decodebin2 post its messages.

Also a generic solution needs a lot more thinking, we should have a separate bug for this. Something like Wim's abstract idea of a tree walk API would definitely be nice to have.