GNOME Bugzilla – Bug 744698
VDPAU interoperability with GstGL
Last modified: 2018-05-05 14:13:21 UTC
Use "GL_NV_vdpau_interop", i.e. vdpausurface to gltexture, to have vdpauvideodec ! glimagesink that avoids downloading/uploading the frame in between. Mesa's Gallium3D drivers has VDPAU state trackers. On ubuntu 14, you need to install libvdpau-dev and mesa-vdpau-drivers packages. https://www.opengl.org/registry/specs/NV/vdpau_interop.txt http://nouveau.freedesktop.org/wiki/VideoAcceleration/
On ubuntu you may also need to install nouveau-firmware package as mentioned in link above.
Note, I had this working in 0.10 (in some lost branch, probably on git.collabora.co.uk). Though I had some issue which prevented me from merging. The display sink at the time was cluttersink, and there was no way to convince COGL that the external texture has been updated. My patches to workaround have never been reviewed. I think this is no longer an issue as we most likely target glimagesink which does not have this kind of optimization. The second issue with VDPAU intereop, was that there is a race (at least on proprietary driver). Our scheme to track when a buffer/texture can be freed used to be (and is still I think) that we release the previous buffer when the next comes in. This didn't work with proprietary driver. Keeping 2 buffers would workaround, obviously a fence object is the right approach, no doubt. Fence for tracking texture life time is still a todo in libgstgl. Let me know if you need these branch, the extension is very simple to use.
This is the mentionned change to convince COGL to update the texture: http://cgit.collabora.com/git/user/nicolas/clutter-gst.git/commit/?h=hw-accel&id=6f41850e442c5faf6140858f1da7f925f2b1af4c This one is the implementation of the converter (in the ancient form of gl upload support: http://cgit.collabora.com/git/user/nicolas/gst-plugins-bad.git/commit/?h=hw-accel&id=c81594423194375c45c3ce8125b3533d82c3fed6 And this was from a larger VPDAU branch (which is most likely 100% outdated now): http://cgit.collabora.com/git/user/nicolas/gst-plugins-bad.git/log/?h=hw-accel
Created attachment 309011 [details] [review] vdpau: clean Makefile.am
Created attachment 309012 [details] [review] vpdmpegdec: force GST_CAPS_FEATURE_MEMORY_VDPAU In order to test vdpinterop the output buffers have GST_CAPS_FEATURE_MEMORY_VDPAU feature
Created attachment 309013 [details] [review] Add gstvdpinterop element
WARNING: these patches are not for pushing, just a request for comments. This weekend, moved by curiosity , I poke on this. It is a work-in-progress, and actually it does not work. Do not fail either:) ... and leaks as hell. So far, my conclusions are: * Since we need to keep tracking the output surface id and registered surface id, the GstGLMemory doesn't look enough, so the GLTextureUpload meta might be more natural. * Perhaps a GstVdpContext might be a good idea. * Perhaps a buffer pool of OutputSurface has more sense than VideoSurface
Nicolas thx for the links and feedbacks I am sure this will help. Victor that's cool that you are looking at this. You are right I think you should define a GstVideoGLTextureUpload callback. Here is what I suggest: You need first to port gstvdpoutputbuffer.c (renaming to gstvdpoutputmemory.c) and gstvdpoutputbufferpool.c to gst1.x Define this GstVideoGLTextureUpload cb in gstvdpoutputbufferpool.c And attach it to all allocated buffers. Then gstvdpvideopostprocess.c needs to be ported. Maybe for now just to convert from memory:vdpau to meta:GstVideoGLTextureUploadMeta. Also I think the vtable construction in your patch can be done in gstgl directly. Indeed GL_NV_vdpau_interop is defined in glext.h and part of the khronos specs/regitry. Also vdpdevice is void* in this header so no need to include vdpau.h in gstgl.
vdpau is not really that interesting anymore with nvdec support.