After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 729196 - omxvideodec: no memory:EGLImage feature in the caps when using eglimage allocator
omxvideodec: no memory:EGLImage feature in the caps when using eglimage alloc...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-omx
git master
Other Linux
: Normal normal
: 1.2.0
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2014-04-29 14:31 UTC by Julien Isorce
Modified: 2014-07-23 08:14 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
omxvideodec: negotiate caps with memory:EGLImage feature when using EGLImage allocator (4.79 KB, patch)
2014-04-29 14:32 UTC, Julien Isorce
needs-work Details | Review
omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator (4.16 KB, patch)
2014-05-01 17:16 UTC, Julien Isorce
accepted-commit_now Details | Review
omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator (4.24 KB, patch)
2014-05-02 17:07 UTC, Julien Isorce
committed Details | Review

Description Julien Isorce 2014-04-29 14:31:08 UTC
Currently on RPI, the negotiated caps are "video/x-raw, format=(string)RGBA" for the 3 following cases 

1- playbin uri=movie.mp4
2- playbin uri=movie.mp4 flags=0x00000003  (video but no deinterlace neither videobalance)
3- playbin uri=movie.mp4 flags=0x00000040  (native video)

whereas omxvideodec selects the GstEGLImageAllocator in all thoses 3 cases.
Comment 1 Julien Isorce 2014-04-29 14:32:56 UTC
Created attachment 275427 [details] [review]
omxvideodec: negotiate caps with memory:EGLImage feature when using EGLImage allocator

With this patch, 2- and 3- are negotiated with video/x-raw(memory:EGLImage), format=RGBA
Comment 2 Julien Isorce 2014-04-29 14:37:17 UTC
So the omxvideodec acts as previously except it first tries to negotiate with the caps feature. If it fails then it tries without caps feature but can still selects the GstEGLImageAllocator like before.

I left the fallback to not bother existing applications.
Comment 3 Julien Isorce 2014-05-01 13:52:03 UTC
I wonder if we should remove "video/x-raw(memory:EGLImage)" in libgstgl, and so use "video/x-raw(meta:GstVideoGLTextureUploadMeta)" here in the attached patches. 

See also https://bugzilla.gnome.org/show_bug.cgi?id=728940#c8
Comment 4 Sebastian Dröge (slomo) 2014-05-01 14:34:01 UTC
(In reply to comment #3)
> I wonder if we should remove "video/x-raw(memory:EGLImage)" in libgstgl, and so
> use "video/x-raw(meta:GstVideoGLTextureUploadMeta)" 

No, you need memory:EGLImage if you explicitly need EGLImage and nothing else.
Comment 5 Nicolas Dufresne (ndufresne) 2014-05-01 14:54:26 UTC
Correct me if I'm wrong, but meta:GstVideoGLTextureUploadMeta is something upstream offers, while memory:EGLImage comes from downstream offer. So in term of caps, memory:EGLImage is an upstream (decoder) requirement, while meta:GstVideoGLTextureUploadMeta is a downstream (videosink) requirement.
Comment 6 Sebastian Dröge (slomo) 2014-05-01 15:08:10 UTC
memory:EGLImage can be both
Comment 7 Julien Isorce 2014-05-01 15:52:35 UTC
(In reply to comment #5)
> Correct me if I'm wrong, but meta:GstVideoGLTextureUploadMeta is something
> upstream offers, while memory:EGLImage comes from downstream offer. So in term
> of caps, memory:EGLImage is an upstream (decoder) requirement, while
> meta:GstVideoGLTextureUploadMeta is a downstream (videosink) requirement.

(In reply to comment #6)
> memory:EGLImage can be both

Thx, it's more clear when saying like this :)

So I guess the attached patch are correct ?
Comment 8 Sebastian Dröge (slomo) 2014-05-01 16:02:46 UTC
Review of attachment 275427 [details] [review]:

We should also be able to use EGLImage if it is *not* in the capsfeatures, but the allocation query returns us (among other buffer pools/allocators) a EGLImage buffer pool/allocator. Is this the case?

::: omx/gstomxvideodec.c
@@ +126,3 @@
   klass->cdata.type = GST_OMX_COMPONENT_TYPE_FILTER;
+  klass->cdata.default_src_template_caps = GST_VIDEO_CAPS_MAKE_WITH_FEATURES
+      (GST_CAPS_FEATURE_MEMORY_EGL_IMAGE, "RGBA") "; "

We only support EGLImage on RPI right now... and also you'll need to add it to the gstomx.conf probably

@@ +940,3 @@
+          gst_caps_replace (&state->caps, NULL);
+
+        /* fallback or fix videobalance + deinterlace */

videobalance and deinterlace are irrelevant here, there can be million other reason for it failing
Comment 9 Julien Isorce 2014-05-01 16:57:18 UTC
(In reply to comment #8)
> Review of attachment 275427 [details] [review]:
> 
> We should also be able to use EGLImage if it is *not* in the capsfeatures, but
> the allocation query returns us (among other buffer pools/allocators) a
> EGLImage buffer pool/allocator. Is this the case?

This is what I call "fallback" case. First it tries to use EGLImage if its in capsfeature. If fails it fallback to try using EGLImage if it is *not* in the capsfeatures like it does prior this patch. And if it fails again it does like before, i.e. try to negotiate with usual RAW like I420 through OMX_AllocateBuffer like before.

> 
> ::: omx/gstomxvideodec.c
> @@ +126,3 @@
>    klass->cdata.type = GST_OMX_COMPONENT_TYPE_FILTER;
> +  klass->cdata.default_src_template_caps = GST_VIDEO_CAPS_MAKE_WITH_FEATURES
> +      (GST_CAPS_FEATURE_MEMORY_EGL_IMAGE, "RGBA") "; "
> 
> We only support EGLImage on RPI right now... and also you'll need to add it to
> the gstomx.conf probably

Ah right thx!

> 
> @@ +940,3 @@
> +          gst_caps_replace (&state->caps, NULL);
> +
> +        /* fallback or fix videobalance + deinterlace */
> 
> videobalance and deinterlace are irrelevant here, there can be million other
> reason for it failing

ok.
Comment 10 Julien Isorce 2014-05-01 17:16:11 UTC
Created attachment 275559 [details] [review]
omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator

Address remarks
Comment 11 Julien Isorce 2014-05-02 14:15:18 UTC
sounds good ?
Comment 12 Sebastian Dröge (slomo) 2014-05-02 14:35:34 UTC
Review of attachment 275559 [details] [review]:

Don't be so impatient, other bugs are waiting weeks for a review :P (be part of the solution :) )

Looks generally good but:

::: omx/gstomxvideodec.c
@@ +932,3 @@
+      state->caps = gst_video_info_to_caps (&state->info);
+      gst_caps_set_features (state->caps, 0,
+          gst_caps_features_new ("memory:EGLImage", NULL));

Don't we have a #define for that somewhere?

@@ +2417,3 @@
+      /* if try to negotiate with caps feature memory:EGLImage
+       * and if allocator is not of type memory EGLImage then fails */
+      if (feature && gst_caps_features_contains (feature, "memory:EGLImage")

and here
Comment 13 Julien Isorce 2014-05-02 15:41:55 UTC
Was because the review has started :)
Do you mean GST_CAPS_FEATURE_MEMORY_EGL_IMAGE ? (http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/egl/gsteglimagememory.h#n37)
Comment 14 Sebastian Dröge (slomo) 2014-05-02 15:54:49 UTC
Yes
Comment 15 Julien Isorce 2014-05-02 17:07:31 UTC
Created attachment 275676 [details] [review]
omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator
Comment 16 Julien Isorce 2014-05-02 17:08:12 UTC
Comment on attachment 275676 [details] [review]
omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator

commit bdec8c05954b048d5e21c5e954a7d56f96231797
Author: Julien Isorce <julien.isorce@collabora.co.uk>
Date:   Tue Apr 29 15:16:16 2014 +0100

    omxvideodec: can negotiate caps with memory:EGLImage feature when using EGLImage allocator
    
    Previously when using gst EGLImage allocator the caps was
    video/x-raw, format=RGBA instead of
    video/x-raw(memory:EGLImage), format=RGBA
    
    Kepp previous behavior in case negotiation fails with caps feature.
    It means it will still have a chance to use EGLImage even if the
    feature is not in the caps.
    
    https://bugzilla.gnome.org/show_bug.cgi?id=729196
Comment 17 Julien Isorce 2014-05-08 08:36:11 UTC
(In reply to comment #7)
> (In reply to comment #5)
> > Correct me if I'm wrong, but meta:GstVideoGLTextureUploadMeta is something
> > upstream offers, while memory:EGLImage comes from downstream offer. So in term
> > of caps, memory:EGLImage is an upstream (decoder) requirement, while
> > meta:GstVideoGLTextureUploadMeta is a downstream (videosink) requirement.
> 
> (In reply to comment #6)
> > memory:EGLImage can be both
> 
> Thx, it's more clear when saying like this :)
> 

Once the caps are negotiated and "set_caps" is called, if the feature in the negotiated caps is GstVideoGLTextureUploadMeta, it means the underlying memory of the gstbuffer (to which the meta is attached) can be anything. And this anything memory is going to be uploaded to a gl texture. Am I right ?
Comment 18 Nicolas Dufresne (ndufresne) 2014-05-08 13:43:59 UTC
Yes, "uploaded" using the accelerated upload() method.