GNOME Bugzilla – Bug 712558
add GstVideoGLTextureUploadMeta inside vaapi_video_buffer_pool_alloc_buffer
Last modified: 2013-12-11 14:32:13 UTC
Currently, the upload meta is added by vaapidecode (from a buffer that comes from the vaapivideobufferpool). When the next element attempts to upload that buffer, it results in the creation of the vaapitexture used to vaCopySurface. When done, the buffer returns to the pool and the upload meta is removed from the buffer and the vaapitexture is destroyed. The process starts again with vaapidecode obtaining a buffer and adding the upload meta. This happens for every single buffer and is extremely inefficient. By simply adding the upload meta inside the buffer pool instead of after the fact (in vaapidecode) I go from ~80% cpu usage to < 10% in the following pipeline. ... ! vaapidecode ! glimagesink
Right. Thanks.
BTW, could you please upload the GstVideoGLTextureUploadMeta support code for glimagesink somewhere? Thanks.
https://github.com/ystreet/gst-plugins-gl/tree/upload_meta
Created attachment 260411 [details] [review] vaapidecode: request GLTextureUpload meta on buffers in the bufferpool From: Matthieu Bouron <matthieu.bouron@collabora.com> Requesting the GLTextureUpload meta on buffers in the bufferpool prevents such metas from being de-allocated when buffers are released in the sink. This is particulary useful in terms of performance when using the GLTextureUploadMeta API since the VaapiTexture associated with the target texture is stored in the meta. The GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META option added here might be implemented in gstreamer later on.
Alternatively, one could check for the GLTextureUploadMeta existence in the GstCapsFeatures of the caps set on the pool. I'm cool with either the bufferpool option or the capsfeatures approach but it might be wise to ask some core gstreamer developers.
Ok, so after a discussion on irc, the idea is to proceed with the buffer pool option as it relates to configuration of the allocation whereas the capsfeatures are for negotiation. See also https://bugzilla.gnome.org/show_bug.cgi?id=712798 for upstream inclusion
I think I will prefer the approach whereby VA/GLX texture is allocated and attached to the GstVaapiVideoMeta. Besides, that texture ought to be stored in another GstVaapiVideoPool. Rationale: we don't want to allocate as many VA/GLX textures as VA surfaces used for decoding because this un-necessarily increases memory requirements. i.e. most of the times, only 2 VA/GLX textures would be needed.
Any updates on this ? Should the patch be reworked ?
Created attachment 262970 [details] [review] [PATCH 1/1] vaapidecode: request GLTextureUpload meta on buffers in the bufferpool Rebased on master and only declare GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META if gstreamer < 1.2.2
Maybe is it enough to just check for GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META definition by its name instead of the specific GStreamer version? i.e. that could help distributions that backport features without necessarily upgrading the version. Anyhow, the patch looks fine too. Thanks.
Created attachment 263122 [details] [review] [PATCH 1/1] vaapidecode: request GLTextureUpload meta on buffers in the bufferpool Patch updated, check presence of GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META instead of gstreamer version.
Review of attachment 263122 [details] [review]: ::: gst/vaapi/gstvaapivideobufferpool.c @@ +221,3 @@ +#if GST_CHECK_VERSION(1,1,0) + if (priv->has_texture_upload_meta) + gst_buffer_add_texture_upload_meta(buffer); I moved that down, after the if (priv->has_video_meta), assuming that you could be supporting the GLTextureUpload meta but not the raw video meta(?).
commit 6b6c10d94d927602f7d47a0bec57cc64431a893f Author: Matthieu Bouron <matthieu.bouron@collabora.com> Date: Wed Nov 20 17:20:07 2013 +0000 plugins: request GLTextureUpload meta on buffers in the buffer pool. Requesting the GLTextureUpload meta on buffers in the bufferpool prevents such metas from being de-allocated when buffers are released in the sink. This is particulary useful in terms of performance when using the GLTextureUploadMeta API since the GstVaapiTexture associated with the target texture is stored in the meta. https://bugzilla.gnome.org/show_bug.cgi?id=712558 Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>