After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 633776 - Video Texture streaming interface
Video Texture streaming interface
Status: RESOLVED OBSOLETE
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2010-11-02 07:21 UTC by Miguel Verdu
Modified: 2013-07-18 09:22 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Patch to plugins-base with the proposed interface (18.15 KB, patch)
2010-11-02 07:21 UTC, Miguel Verdu
none Details | Review
Updated patch (18.02 KB, patch)
2010-12-02 11:42 UTC, Miguel Verdu
none Details | Review
Patch 1/3 : move photography interface to own dir (75.44 KB, patch)
2010-12-19 22:06 UTC, Miguel Verdu
none Details | Review
Patch 2/3: new simplified video texture interface (18.95 KB, patch)
2010-12-19 22:07 UTC, Miguel Verdu
none Details | Review
Patch 3/3: extra gtk-doc text added to describe multi-process case (6.16 KB, patch)
2010-12-19 22:09 UTC, Miguel Verdu
none Details | Review
example of a simple test application using a sink with this interface (4.76 KB, application/x-gzip)
2010-12-19 22:10 UTC, Miguel Verdu
  Details
Patch 1 - allow for multiple interfaces (3.39 KB, patch)
2011-01-26 12:08 UTC, Miguel Verdu
none Details | Review
patch with new interface- egl context property added (21.64 KB, patch)
2011-01-26 12:09 UTC, Miguel Verdu
none Details | Review
Updated test app (4.80 KB, application/zip)
2011-01-26 12:12 UTC, Miguel Verdu
  Details

Description Miguel Verdu 2010-11-02 07:21:03 UTC
Created attachment 173678 [details] [review]
Patch to plugins-base with the proposed interface

This interface is intended for video sinks that can either render to a video overlay or produce frames that can be map to GL textures without any additional copy.
See documentation in the patch.
This is a draft version
Comment 1 Stefan Sauer (gstreamer, gtkdoc dev) 2010-11-05 11:54:18 UTC
I would factor out the "Seamless overlay switching" to even another iface. Then one can implement texture streaming on elements just supporting it and xvoverlay on those which support that. The switching iface could be implemented on sinks implementing both texture streaming and overlay.
Comment 2 Stefan Sauer (gstreamer, gtkdoc dev) 2010-12-01 14:59:22 UTC
Some spec for a eglsink where this interface would come handy:
https://wiki.linaro.org/WorkingGroups/Middleware/Multimedia/Specs/1105/GstreamerEglSink
Comment 3 David Schleef 2010-12-02 06:09:22 UTC
What problem are you trying to solve here?  The patch above makes no sense as an interface.

We currently have both a gl and egl sink in gst-plugins-gl.  How are these not sufficient?
Comment 4 Benjamin Gaignard 2010-12-02 11:06:38 UTC
I guess glimagesink is the name of the gl sink, but I can't found egl sink. Maybe I'm not looking at the right place ?
Comment 5 Miguel Verdu 2010-12-02 11:42:20 UTC
Created attachment 175702 [details] [review]
Updated patch

David,
first of all a disclaimer, this is the firs interface ever that I implement in gstreamer, so forgive any dumb errors.

Regarding the purpose of the interface you can see it in the documentation comments I have added to the videotexture.c

Regarding gstreamer-gl, they way I see it is that we want the applications to control the scene composition where video is just another texture.
This interface allows video sinks to temporarily allow the applications to do fancy transitions and then resume seamlessly the normal rendering to the HW overlay without having to change the pipeline dynamically. Since using the HW overlay is more energy efficient.
This would be like having in one single element the functionality of gl-upload  and a video sink. 

We would like to have a gstreamer interface for any HW vendors to implement and leave the egl details hidden under.

if we split the functionality as Stefan proposed we could have one interface only to provide the textures and that could be implemented also by the gl-upload
and then the other one for the seamless transition. The point is that a single elemetn can implement them both.

This is the kind of flexible sink that we want to use in Meego.
BTW, we have a working prototype of a sink implementing partially this i/f (the seamless transition is not working yet) and this patch will evolve as we finalize this work.
Comment 6 Miguel Verdu 2010-12-19 22:04:29 UTC
The following 3 patches contain a new version of the interface:
- it is rebased to plugins-bad
- it removes all the  unnecessary properties and focuses on the mechanism to exchange textures.

In order to add new interfaces to plugins-bad I decided to add new subdirectories to keep the existing library (libgstphotography) with the same name, rather than creating a libgstinterfacesbad

The part needed for seamless transition is added to a separate interface.
I attach a sample application using a sink with that interface (testapp)
Comment 7 Miguel Verdu 2010-12-19 22:06:05 UTC
Created attachment 176722 [details] [review]
Patch 1/3 : move photography interface to own dir
Comment 8 Miguel Verdu 2010-12-19 22:07:23 UTC
Created attachment 176723 [details] [review]
Patch 2/3: new simplified video texture interface
Comment 9 Miguel Verdu 2010-12-19 22:09:14 UTC
Created attachment 176724 [details] [review]
Patch 3/3: extra gtk-doc text added to describe multi-process case

Extra documentation to describe the use case for multi-process rendering scenario.
Comment 10 Miguel Verdu 2010-12-19 22:10:13 UTC
Created attachment 176725 [details]
example of a simple test application using a sink with this interface
Comment 11 Miguel Verdu 2011-01-26 12:06:58 UTC
The following three attachments provide a new version of the interface:

The first patch is a new  version of the patch to allow multiple interfaces to coexists.
first proposal based on moving the interfaces to it's own directories introduced lot's of issues in the compilation of the whole -plugins-bad tree.

The other  patch is the videotexture interface, Main diference is the addition of a new property for the application to specify the EGL-context in use

The patches are rebased to the latest version of -plugins-bad (0.10.21)
Comment 12 Miguel Verdu 2011-01-26 12:08:25 UTC
Created attachment 179364 [details] [review]
Patch 1 - allow for multiple interfaces
Comment 13 Miguel Verdu 2011-01-26 12:09:25 UTC
Created attachment 179365 [details] [review]
patch with new interface- egl context property added
Comment 14 Miguel Verdu 2011-01-26 12:12:59 UTC
Created attachment 179366 [details]
Updated test app
Comment 15 Edward Hervey 2013-07-18 05:58:26 UTC
Isn't this deprecated now that we have eglessink ?
Comment 16 Sebastian Dröge (slomo) 2013-07-18 08:51:21 UTC
This should be obsolete now by GstContext, GstVideoTextureUploadMeta, gst-plugins-gl and the EGL library in gst-plugins-bad or a combination of these. Don't think anything is still missing conceptionally.

(In reply to comment #15)
> Isn't this deprecated now that we have eglessink ?

Note that I plan to deprecated that in favour of gst-plugins-gl.
Comment 17 Miguel Verdu 2013-07-18 09:22:49 UTC
Agreed, this is now obsolete .