GNOME Bugzilla – Bug 701430
For audio / video mixing in gst-editing-services, we need to use metadata.
Last modified: 2013-06-04 18:25:26 UTC
Context : Sebastian Dröge just implemented a volume property on adder pads. We have an adder for each audio track, and each track element has a volume element upstream. We would need volume to negotiate a Meta if adder is present downstream, and let adder do the volume modifications in that case. Ideally, we would like to have the same system for videomixer, to let it handle alpha and [x, y, z] coordinates. The problem is that we don't know which element could be placed upstream to negotiate these metas. Videobox, as far as I understand, does not do exactly that, it handles alpha but doesn't handle [x, y, z] the same way videomixer does. What do you think would be the best way to expose these properties ?
I think we agreed on IRC that this has nothing to do in GStreamer directly but should be fixed in gnonlin/GES as it's only related directly to the design of these.
I think the description of the issue is not 100% clear here. I will try to clarify it: We need to expose the x-y-alpha properties of some gnlsource so that the information will be used by the videomixer that is inside the pipeline. But those gnlobject are not linked to the mixer at the time the user want to set those properties (as the linking is dynamical), and thus there is no pad to set the properties on. We could have dummy properties but then making the properties controllable will make the code really ugly (we have gone that way first and it is just not a solution). Having an element in the source itself to which to set those properties on and that attaches metas on buffer make the code way simpler/cleaner in our case, and I believe this is not a design issue on our part (we have beeen thinking a lot about the various solutions and this on seem to be the cleanest/simplest one to me). We ended up implementing exactly that in GES[0] itself and the result in terme of code is quite nice to me. I believe the same issue will have to be solve in other projects at some point, but anyway. [0] https://github.com/MathieuDuponchelle/PitiviGes/commits/video_mixing
Probably but having metas for that in GStreamer does not make sense conceptionally. It's very specific to your use case, from a GStreamer point of view this is all completely transparent and generally usable.
I believe this same videomixer meta will be usefull to be able to delegate the compositing to the sink anyway.