GNOME Bugzilla – Bug 720174
vaapipostproc: add support for "download" capability
Last modified: 2014-08-22 16:54:09 UTC
QA need interface to download video data directly from vaapipostproc, which is used to do validation for testing features like CSC
Hi, (In reply to comment #0) > QA need interface to download video data directly from vaapipostproc, which is > used to do validation for testing features like CSC For this usage, vaapipostproc is not the appropriate element since you could also perform additional color conversion, thus invalidating your tests. What should be done instead is to use the "fakesink" element for instance and track the "handoff" signal so that you could map the incoming buffer, get the underlying VA surface id, and use vaDeriveImage() to read the exact pixels from the surface. What kind of pipeline did you have in mind? Thanks.
Having said that, the usage model I described requires bug #720311 to be fixed first. That will be the supported way. I still wondering about practical use cases for the "download" capability.
It seems your suggested way is a good way to go, a 'fakesrc' or 'fakesink' is a good thing to have. I haven't learnt up enough knowledge to implement it yet. but let's consider one way to go at first. We'll learn more to see how it would work
It may be related, but by using the vaapipostproc format=x I was expecting (maybe I'm completly wrong about this) to have a stream format as video/raw-x, format=x .. The result was : video/x-raw(memory:VASurface) format=x and this stream isn't "convenant" to use with other gstreamer plugins. (for example : videomixer). I think that a download plugin is very important for the flexibility gain by a "standard" stream format. Apparently, the vaapidownload was implemented in 0.10, but to my knownledge, there isn't an equivalent plugin in newer version. Exemple of pipeline : ... ! vaapidecode ! vaapipostproc ! videoconvert ! videomixer ! videoconvert ! xvimagesink
Ah, yes, support of pipepline like 'vaapidecode ! vaapipostproc ! videoconvert ! videomixer ! videoconvert ! xvimagesink' would also help us to do side by side comparison with other render method when there's failure in case that rendering is not correctly working for vaapisink
(In reply to comment #1) > Hi, > > (In reply to comment #0) > > QA need interface to download video data directly from vaapipostproc, which is > > used to do validation for testing features like CSC > > For this usage, vaapipostproc is not the appropriate element since you could > also perform additional color conversion, thus invalidating your tests. What > should be done instead is to use the "fakesink" element for instance and track > the "handoff" signal so that you could map the incoming buffer, get the > underlying VA surface id, and use vaDeriveImage() to read the exact pixels from > the surface. This can work, but not convenient for users. this is similar like hook vaPutSurface with preload.just like some hack ways. It's better to have a download feature in vaapipostproc. then QA or others can easy use it. e.g. vaapi decode and save raw yuv to files, or connect with software rendering. > > What kind of pipeline did you have in mind? Thanks.
Hi, (In reply to comment #6) > (In reply to comment #1) > > (In reply to comment #0) > > > QA need interface to download video data directly from vaapipostproc, which is > > > used to do validation for testing features like CSC > > > > For this usage, vaapipostproc is not the appropriate element since you could > > also perform additional color conversion, thus invalidating your tests. What > > should be done instead is to use the "fakesink" element for instance and track > > the "handoff" signal so that you could map the incoming buffer, get the > > underlying VA surface id, and use vaDeriveImage() to read the exact pixels from > > the surface. > This can work, but not convenient for users. this is similar like hook > vaPutSurface with preload.just like some hack ways. I was not talking about convenience but about correctness of the end result that you would want to compare. That's different things, and for QA/validation purposes, an explicit use of vaDeriveImage() for a custom plug-in for a specific platform is the way to go.
(In reply to comment #4) > It may be related, but by using the vaapipostproc format=x I was expecting > (maybe I'm completly wrong about this) to have a stream format as video/raw-x, > format=x .. The result was : video/x-raw(memory:VASurface) format=x and this > stream isn't "convenant" to use with other gstreamer plugins. (for example : > videomixer). vaapipostproc, like any other VA-API related plug-ins, will always prefer the native representation first. > I think that a download plugin is very important for the flexibility gain by a > "standard" stream format. Well, there was a strong opposition to the vaapiupload/vaapidownload plug-ins. That's why I stuffed all the things I needed at this time into vaapipostproc. :) > Apparently, the vaapidownload was implemented in 0.10, but to my knownledge, > there isn't an equivalent plugin in newer version. > > Exemple of pipeline : > ... ! vaapidecode ! vaapipostproc ! videoconvert ! videomixer ! videoconvert ! > xvimagesink SW rendering is exactly what I don't want to expose because this hurts performance. However, the videomixer based usages can be a valid reason. Having said that, the HW accelerated path shall always be the preferred path. It's possible to have plain video/x-raw exposed, but I want to be sure that this path is not selected first for auto-plugging purposes. vaapipostproc is not auto-plugged yet, though.
if vaapidownload can do vaDeriveImage, there's no need for QA to write a specific plugin. and convert surface to raw yuv should be gstvaapi's work.
(In reply to comment #9) > if vaapidownload can do vaDeriveImage, there's no need for QA to write a > specific plugin. and convert surface to raw yuv should be gstvaapi's work. vaapidownload can do vaDeriveImage(), but not in a deterministic way. For validation purposes, you have to make sure you always get the raw YUV data, as contained in the original VA surface. Alternative is also to fix the VA driver so that to not involve scaler through vaGetImage() when VA image size == VA surface size.
Just wanted to comment that we did use the vaapidownload element and would love to see it make a re-appearance for GStreamer 1.0 in some form, whether that be in vaapipostproc or again as a separate element. We do pipelines that involve creating jpeg thumbnails, video mixer stuff, all the usual GStreamer things. We found vaapidownload to be sufficiently fast for our needs.
*** Bug 724817 has been marked as a duplicate of this bug. ***
commit 493337c4b154a789b5f5e9d86f95b4015d07b57e Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com> Date: Fri Aug 22 15:22:32 2014 +0200 vaapipostproc: add support for "download" capability. Allow implicit conversions to raw video formats, while still keeping VA surfaces underneath. This allows for chaining the vaapipostproc element to a software-only element that takes care of maps/unmaps. e.g. xvimagesink. https://bugzilla.gnome.org/show_bug.cgi?id=720174