GNOME Bugzilla – Bug 729196
omxvideodec: no memory:EGLImage feature in the caps when using eglimage allocator
Last modified: 2014-07-23 08:14:25 UTC
Currently on RPI, the negotiated caps are "video/x-raw, format=(string)RGBA" for the 3 following cases 1- playbin uri=movie.mp4 2- playbin uri=movie.mp4 flags=0x00000003 (video but no deinterlace neither videobalance) 3- playbin uri=movie.mp4 flags=0x00000040 (native video) whereas omxvideodec selects the GstEGLImageAllocator in all thoses 3 cases.
Created attachment 275427 [details] [review] omxvideodec: negotiate caps with memory:EGLImage feature when using EGLImage allocator With this patch, 2- and 3- are negotiated with video/x-raw(memory:EGLImage), format=RGBA
So the omxvideodec acts as previously except it first tries to negotiate with the caps feature. If it fails then it tries without caps feature but can still selects the GstEGLImageAllocator like before. I left the fallback to not bother existing applications.
I wonder if we should remove "video/x-raw(memory:EGLImage)" in libgstgl, and so use "video/x-raw(meta:GstVideoGLTextureUploadMeta)" here in the attached patches. See also https://bugzilla.gnome.org/show_bug.cgi?id=728940#c8
(In reply to comment #3) > I wonder if we should remove "video/x-raw(memory:EGLImage)" in libgstgl, and so > use "video/x-raw(meta:GstVideoGLTextureUploadMeta)" No, you need memory:EGLImage if you explicitly need EGLImage and nothing else.
Correct me if I'm wrong, but meta:GstVideoGLTextureUploadMeta is something upstream offers, while memory:EGLImage comes from downstream offer. So in term of caps, memory:EGLImage is an upstream (decoder) requirement, while meta:GstVideoGLTextureUploadMeta is a downstream (videosink) requirement.
memory:EGLImage can be both
(In reply to comment #5) > Correct me if I'm wrong, but meta:GstVideoGLTextureUploadMeta is something > upstream offers, while memory:EGLImage comes from downstream offer. So in term > of caps, memory:EGLImage is an upstream (decoder) requirement, while > meta:GstVideoGLTextureUploadMeta is a downstream (videosink) requirement. (In reply to comment #6) > memory:EGLImage can be both Thx, it's more clear when saying like this :) So I guess the attached patch are correct ?
Review of attachment 275427 [details] [review]: We should also be able to use EGLImage if it is *not* in the capsfeatures, but the allocation query returns us (among other buffer pools/allocators) a EGLImage buffer pool/allocator. Is this the case? ::: omx/gstomxvideodec.c @@ +126,3 @@ klass->cdata.type = GST_OMX_COMPONENT_TYPE_FILTER; + klass->cdata.default_src_template_caps = GST_VIDEO_CAPS_MAKE_WITH_FEATURES + (GST_CAPS_FEATURE_MEMORY_EGL_IMAGE, "RGBA") "; " We only support EGLImage on RPI right now... and also you'll need to add it to the gstomx.conf probably @@ +940,3 @@ + gst_caps_replace (&state->caps, NULL); + + /* fallback or fix videobalance + deinterlace */ videobalance and deinterlace are irrelevant here, there can be million other reason for it failing
(In reply to comment #8) > Review of attachment 275427 [details] [review]: > > We should also be able to use EGLImage if it is *not* in the capsfeatures, but > the allocation query returns us (among other buffer pools/allocators) a > EGLImage buffer pool/allocator. Is this the case? This is what I call "fallback" case. First it tries to use EGLImage if its in capsfeature. If fails it fallback to try using EGLImage if it is *not* in the capsfeatures like it does prior this patch. And if it fails again it does like before, i.e. try to negotiate with usual RAW like I420 through OMX_AllocateBuffer like before. > > ::: omx/gstomxvideodec.c > @@ +126,3 @@ > klass->cdata.type = GST_OMX_COMPONENT_TYPE_FILTER; > + klass->cdata.default_src_template_caps = GST_VIDEO_CAPS_MAKE_WITH_FEATURES > + (GST_CAPS_FEATURE_MEMORY_EGL_IMAGE, "RGBA") "; " > > We only support EGLImage on RPI right now... and also you'll need to add it to > the gstomx.conf probably Ah right thx! > > @@ +940,3 @@ > + gst_caps_replace (&state->caps, NULL); > + > + /* fallback or fix videobalance + deinterlace */ > > videobalance and deinterlace are irrelevant here, there can be million other > reason for it failing ok.
Created attachment 275559 [details] [review] omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator Address remarks
sounds good ?
Review of attachment 275559 [details] [review]: Don't be so impatient, other bugs are waiting weeks for a review :P (be part of the solution :) ) Looks generally good but: ::: omx/gstomxvideodec.c @@ +932,3 @@ + state->caps = gst_video_info_to_caps (&state->info); + gst_caps_set_features (state->caps, 0, + gst_caps_features_new ("memory:EGLImage", NULL)); Don't we have a #define for that somewhere? @@ +2417,3 @@ + /* if try to negotiate with caps feature memory:EGLImage + * and if allocator is not of type memory EGLImage then fails */ + if (feature && gst_caps_features_contains (feature, "memory:EGLImage") and here
Was because the review has started :) Do you mean GST_CAPS_FEATURE_MEMORY_EGL_IMAGE ? (http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/egl/gsteglimagememory.h#n37)
Yes
Created attachment 275676 [details] [review] omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator
Comment on attachment 275676 [details] [review] omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator commit bdec8c05954b048d5e21c5e954a7d56f96231797 Author: Julien Isorce <julien.isorce@collabora.co.uk> Date: Tue Apr 29 15:16:16 2014 +0100 omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator Previously when using gst EGLImage allocator the caps was video/x-raw, format=RGBA instead of video/x-raw(memory:EGLImage), format=RGBA Kepp previous behavior in case negotiation fails with caps feature. It means it will still have a chance to use EGLImage even if the feature is not in the caps. https://bugzilla.gnome.org/show_bug.cgi?id=729196
(In reply to comment #7) > (In reply to comment #5) > > Correct me if I'm wrong, but meta:GstVideoGLTextureUploadMeta is something > > upstream offers, while memory:EGLImage comes from downstream offer. So in term > > of caps, memory:EGLImage is an upstream (decoder) requirement, while > > meta:GstVideoGLTextureUploadMeta is a downstream (videosink) requirement. > > (In reply to comment #6) > > memory:EGLImage can be both > > Thx, it's more clear when saying like this :) > Once the caps are negotiated and "set_caps" is called, if the feature in the negotiated caps is GstVideoGLTextureUploadMeta, it means the underlying memory of the gstbuffer (to which the meta is attached) can be anything. And this anything memory is going to be uploaded to a gl texture. Am I right ?
Yes, "uploaded" using the accelerated upload() method.