GNOME Bugzilla – Bug 633776
Video Texture streaming interface
Last modified: 2013-07-18 09:22:49 UTC
Created attachment 173678 [details] [review] Patch to plugins-base with the proposed interface This interface is intended for video sinks that can either render to a video overlay or produce frames that can be map to GL textures without any additional copy. See documentation in the patch. This is a draft version
I would factor out the "Seamless overlay switching" to even another iface. Then one can implement texture streaming on elements just supporting it and xvoverlay on those which support that. The switching iface could be implemented on sinks implementing both texture streaming and overlay.
Some spec for a eglsink where this interface would come handy: https://wiki.linaro.org/WorkingGroups/Middleware/Multimedia/Specs/1105/GstreamerEglSink
What problem are you trying to solve here? The patch above makes no sense as an interface. We currently have both a gl and egl sink in gst-plugins-gl. How are these not sufficient?
I guess glimagesink is the name of the gl sink, but I can't found egl sink. Maybe I'm not looking at the right place ?
Created attachment 175702 [details] [review] Updated patch David, first of all a disclaimer, this is the firs interface ever that I implement in gstreamer, so forgive any dumb errors. Regarding the purpose of the interface you can see it in the documentation comments I have added to the videotexture.c Regarding gstreamer-gl, they way I see it is that we want the applications to control the scene composition where video is just another texture. This interface allows video sinks to temporarily allow the applications to do fancy transitions and then resume seamlessly the normal rendering to the HW overlay without having to change the pipeline dynamically. Since using the HW overlay is more energy efficient. This would be like having in one single element the functionality of gl-upload and a video sink. We would like to have a gstreamer interface for any HW vendors to implement and leave the egl details hidden under. if we split the functionality as Stefan proposed we could have one interface only to provide the textures and that could be implemented also by the gl-upload and then the other one for the seamless transition. The point is that a single elemetn can implement them both. This is the kind of flexible sink that we want to use in Meego. BTW, we have a working prototype of a sink implementing partially this i/f (the seamless transition is not working yet) and this patch will evolve as we finalize this work.
The following 3 patches contain a new version of the interface: - it is rebased to plugins-bad - it removes all the unnecessary properties and focuses on the mechanism to exchange textures. In order to add new interfaces to plugins-bad I decided to add new subdirectories to keep the existing library (libgstphotography) with the same name, rather than creating a libgstinterfacesbad The part needed for seamless transition is added to a separate interface. I attach a sample application using a sink with that interface (testapp)
Created attachment 176722 [details] [review] Patch 1/3 : move photography interface to own dir
Created attachment 176723 [details] [review] Patch 2/3: new simplified video texture interface
Created attachment 176724 [details] [review] Patch 3/3: extra gtk-doc text added to describe multi-process case Extra documentation to describe the use case for multi-process rendering scenario.
Created attachment 176725 [details] example of a simple test application using a sink with this interface
The following three attachments provide a new version of the interface: The first patch is a new version of the patch to allow multiple interfaces to coexists. first proposal based on moving the interfaces to it's own directories introduced lot's of issues in the compilation of the whole -plugins-bad tree. The other patch is the videotexture interface, Main diference is the addition of a new property for the application to specify the EGL-context in use The patches are rebased to the latest version of -plugins-bad (0.10.21)
Created attachment 179364 [details] [review] Patch 1 - allow for multiple interfaces
Created attachment 179365 [details] [review] patch with new interface- egl context property added
Created attachment 179366 [details] Updated test app
Isn't this deprecated now that we have eglessink ?
This should be obsolete now by GstContext, GstVideoTextureUploadMeta, gst-plugins-gl and the EGL library in gst-plugins-bad or a combination of these. Don't think anything is still missing conceptionally. (In reply to comment #15) > Isn't this deprecated now that we have eglessink ? Note that I plan to deprecated that in favour of gst-plugins-gl.
Agreed, this is now obsolete .