GNOME Bugzilla – Bug 561225
Add support for VDPAU (PureVideo)
Last modified: 2018-11-03 13:03:51 UTC
A way to accelerate display/decoding for VC-1, MPEG-1, MPEG-2, H264.
See: http://www.nvnews.net/vbulletin/showthread.php?t=123091 MPlayer patches: ftp://download.nvidia.com/XFree86/vdpau/mplayer-vdpau-3076399.tar.bz2
"Video Decode and Presentation API for Unix" ftp://download.nvidia.com/XFree86/vdpau/doxygen/html/index.html Although Xine, VLC and MythTV all now have various stages of VDPAU implementations, I was only able to dig up the commits for MythTV[1] (using VDPAU to render their interface onscreen). The only information about the VLC implementation so far[2] is that "Two students are working on it". The only mention I can find of support in Xine is from a post to xine-devel[3] where a developer states he "will sure try to write a VDPAU vo driver". However, this post is from November, and according to a Phoronix.com article, Xine supports VDPAU starting Dec 17th[4]. Anyway, I started writing this bug comment in the hopes of linking you guys some other implementations to use as a reference instead of "just" the one for mplayer/ffmpeg released by NVIDIA themselves. Perhaps there would be something to be learned from Xine/MythTV/VLC's implementations. But as you can see I only managed to dig up actual code for one of them - MythTV. 1: http://svn.mythtv.org/trac/changeset/19360 - Initial commit http://svn.mythtv.org/trac/changeset/19364 - And some minor touch ups http://www.phoronix.com/scan.php?page=news_item&px=Njg4Ng - Phoronix article 2: http://forum.videolan.org/viewtopic.php?f=13&t=53928&p=176923#p176923 http://www.phoronix.com/scan.php?page=news_item&px=Njk1NA - Phoronic article 3: http://sourceforge.net/mailarchive/forum.php?thread_name=200811212026.20166.julian%40jusst.de&forum_name=xine-devel 4: http://www.phoronix.com/scan.php?page=news_item&px=Njk0MA
A few recommendations of how to pursue this: * Create a sys/vdpau/ path in gst-plugins-bad * Write vdpau configure.ac/Makefile.am code to build a blank plugin against it. (Proof of concept of detection/linking of the vdpau library). * Write 2 simple plugins to convert RAW Video GstBuffer <=> VdpVideoSurface/VdpBitmapSurface. (Proof of concept of RAM <=> GPU communication). You might have to create a gstreamer 'mime' type for these buffers (video/x-vdpau-video-surface, video/x-vdpau-bitmap-surface, .??). * Write a sink using the VdpDevice that takes VdpVideoSurface or VdpBitmapSurface. (Proof of concept of synchronous VDPAU display). * Wrap the VdpVideoMixer feature (de-interlacing, colorspace conversion, noise reduction, sharpness reduction, ...). (Proof of concept of using simple VDPAU processing features). * FINALLY : Wrap the VdpDecoder. At this point, if you've done all the steps above, you will have narrowed down your source of issues to a small range (You already know vdpau works, you can send/receive data from it, you can display it, and you can do some processing on it). You will most likely have to do some format-specific parsing of the encoded data buffers before passing them on to the VDPAU code (according to my reading of the specs).
I also had the though that'd it'd be nice for the decoder element(s) to support negotiation on the src pad - if downstream supports the video/x-vdpau-*-surface output that, otherwise allow negotiation to a video/x-raw-(yuv|rgb) buffer and pull the frames back from the graphics card. That way, we can get VDPAU decoding in playbin with the existing pipeline building.
Proposing this for SoC 2009.
If VDPAU is a problem, there's also VA-API and XvBA: http://www.phoronix.com/scan.php?page=article&item=xorg_vaapi_mplayer&num=1 (Should I file separate bugs?)
VDPAU isn't a 'problem', it's a 'challenge' :) Anyway, yes, please file separate bugs for those.
I've been doing some initial work on this which can be found at http://github.com/satis/gst-plugins-bad/tree/master. I've implemented 3 elements. 1. vdpauyuvvideo -- takes video/x-raw-yuv and outputs video/x-vdpau-video 2. vdpauvideoyuv -- takes video/x-vdpau-video and outputs video/x-raw-yuv 3. vdpaumpegdec -- proof of concept mpeg decoder which is capable of outputting "recognizable" video from some files :). The mpeg decoder (which is kindof useless anyway) is based pretty much on guesswork and therefore it's a wonder it even gives "recognizable" output. To test the "vdpaumpegdec" element you can try the following pipeline: "gst-launch filesrc location=/path/to/mpeg/file ! mpegvideoparse ! vdpaumpegdec ! vdpauvideoyuv ! xvimagesink".
So the work on vdpaumpegdec has been slowly progressing and is now in a slightly better shape. It works on most mpeg streams I've tested and gets correctly autoplugged into playbin.
Playing with what's been implemented so far, I think it's mature enough that we should merge it into -bad, so more people can easily play with it. I noticed, however, that all of the commits in your tree have bogus email addresses in the author/committer info, which isn't cool for the main repositories. We should fix those before importing the patches into the main -bad. After that, we can pull updates from your tree directly (if you start committing with proper contact details)
I've now fixed the emails (I hope) and also rebased the branches.
Thanks! I pushed it upstream, along with a few commits of my own
(In reply to comment #8) > I've been doing some initial work on this which can be found at > http://github.com/satis/gst-plugins-bad/tree/master. > > I've implemented 3 elements. > > 1. vdpauyuvvideo -- takes video/x-raw-yuv and outputs video/x-vdpau-video > 2. vdpauvideoyuv -- takes video/x-vdpau-video and outputs video/x-raw-yuv > 3. vdpaumpegdec -- proof of concept mpeg decoder which is capable of > outputting "recognizable" video from some files :). > > The mpeg decoder (which is kindof useless anyway) is based pretty much on > guesswork and therefore it's a wonder it even gives "recognizable" output. > > To test the "vdpaumpegdec" element you can try the following pipeline: > "gst-launch filesrc location=/path/to/mpeg/file ! mpegvideoparse ! vdpaumpegdec > ! vdpauvideoyuv ! xvimagesink". after building and installing the bad plugins, I still don't get vdpauyuvvideo, vdpaumpegdec elements, I tried rebuilding the registry by deleting the old one and issuing gst-inspect. Do I need to do anything special?
(In reply to comment #13) > > after building and installing the bad plugins, I still don't get vdpauyuvvideo, > vdpaumpegdec elements, I tried rebuilding the registry by deleting the old one > and issuing gst-inspect. Do I need to do anything special? Check the configure output. It should say that it's detected your installed vdpau library and is configured to build the GStreamer elements.
What's the status of this one? It sounds like it it got into git about a year ago... has it made it into a release? This would be a killer app on these fancy little ion-powered HTPC boxes that are getting so popular... What about the va-api or XvBA support? Anything happening with those ones?
Gwenole Beauchesne is working on VA-API: http://www.splitted-desktop.com/~gbeauchesne/gstreamer-vaapi/
is there a technical reason why Theora or VP8 is not concerned bu this bug report ?
(In reply to comment #17) > is there a technical reason why Theora or VP8 is not concerned bu this bug > report ? They are not supported by any current GPU (VDAPU, XbMC and VA-API only expose the capacities of the underlying hardware).
I have tested divx h264 mpeg4_part2 vc1 wmv3 1080P video formats which my card(nvidia GT240m) supports, only divx failed. shall I file a new bug?
(In reply to comment #19) > I have tested divx h264 mpeg4_part2 vc1 wmv3 1080P video formats which my > card(nvidia GT240m) supports, only divx failed. > > shall I file a new bug? I'm sorry, vc1 and wmv3 are not support by gst-plugins-bad, two more bugs?
https://bugzilla.gnome.org/show_bug.cgi?id=611032 is assigned, and seems to have more life than this bug, and applies to nvidia, amd/ati, and even some intel cards.
Any reason not to support both?
Should this bug be closed ? It has been done during gsoc 2010 here http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau . Though some features like h264 has been disabled when porting to gst1.x.
(In reply to Julien Isorce from comment #23) > Should this bug be closed ? It has been done during gsoc 2010 here > http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau . > Though some features like h264 has been disabled when porting to gst1.x. I have been working on this, but still is a WIP. It is stalled on the frame parser for mpeg4 since I had to attend other tasks. Still, I just pushed my branch if somebody is willing to give a look: https://gitlab.com/vjaquez-gstreamer/gst-plugins-bad/commits/vdp
(In reply to Víctor Manuel Jáquez Leal from comment #24) > I have been working on this, but still is a WIP. It is stalled on the frame > parser for mpeg4 since I had to attend other tasks. Still, I just pushed my > branch if somebody is willing to give a look: > > https://gitlab.com/vjaquez-gstreamer/gst-plugins-bad/commits/vdp Hi Victor, do you plan to finish that work ? (and bug #744698). Maybe it is worth to attach for review what you got already ? I cannot access the gitlab link anymore. I would say the most interesting part would be to have gstvdpauh264dec and gstvdpauvideopostprocess back (eventually gstvdpausink if you already ported it). I do not think anybody use the current gst-vdpau since it only supports mpeg2 so what you made is really valuable and would be a pity to not have it upstream. I am chasing this because today Christian Koenig (from AMD) submitted a patch series for Mesa/Gallium to extend Gallium vdpau backend to support potential new VDPAU functionality/API: VdpVideoSurfaceDMABuf and VdpOutputSurfaceDMABuf. (Similar to vaAcquireBufferHandle in VAAPI). It aallow to export a surface as a dma buffer and to use in conjunction with EXT_image_dma_buf_import. See one of the patches: https://patchwork.freedesktop.org/patch/76129/ So Victor if you work is merged I might be interested to add support for dmabuf.
(In reply to Julien Isorce from comment #25) Hi Julien, > Hi Victor, do you plan to finish that work ? (and bug #744698). Maybe it is > worth to attach for review what you got already ? I cannot access the gitlab > link anymore. I work on this when I find time (which is not a lot). Managed to have a mpeg2 decoding since the mpegvideo parser sends the parsing data as a buffer meta. But no other parser does that (bug 691712). That means that we need to replicate what gstreamer-vaapi does: do its own frame parsing to extract the slice's properties. Nonetheless, I developed it using the proprietary NVIDIA driver, and the mpeg2 decoding to ximagesink works fine. Last week I setup VDPAU decoding with nouveau and the decoder doesn't show the frames correctly! (though no error messages are throw). Very disappointing. Though I need to verify if my setup is correct. I can clean up the patches and upload them here as soon as I find time (most probably not this month :S) > I would say the most interesting part would be to have gstvdpauh264dec and > gstvdpauvideopostprocess back (eventually gstvdpausink if you already ported > it). Simultaneously I started working on a gstvdaumixer (postproc) and the gstvdpauinterop (bug 744698) with the idea of grouping them in a bin. But also they are not functional yet, almost pure boilerplate. > I do not think anybody use the current gst-vdpau since it only supports > mpeg2 so what you made is really valuable and would be a pity to not have it > upstream. Agree. > I am chasing this because today Christian Koenig (from AMD) submitted a > patch series for Mesa/Gallium to extend Gallium vdpau backend to support > potential new VDPAU functionality/API: VdpVideoSurfaceDMABuf and > VdpOutputSurfaceDMABuf. > (Similar to vaAcquireBufferHandle in VAAPI). It aallow to export a surface > as a dma buffer and to use in conjunction with EXT_image_dma_buf_import. > See one of the patches: https://patchwork.freedesktop.org/patch/76129/ Yes, I saw that API patch in the mailing list. > So Victor if you work is merged I might be interested to add support for > dmabuf. :$
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/issues/7.