After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 767801 - Failed to negotiate RGBA for EGLImage on Raspberry Pi 3
Failed to negotiate RGBA for EGLImage on Raspberry Pi 3
Status: RESOLVED INCOMPLETE
Product: GStreamer
Classification: Platform
Component: gst-omx
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2016-06-17 21:09 UTC by zacattackcit
Modified: 2018-08-28 05:03 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Log file showing omx error (941.48 KB, text/x-log)
2016-06-17 21:09 UTC, zacattackcit
Details
Master debug log, freezes after first frame (2.54 MB, text/x-log)
2016-06-21 19:26 UTC, zacattackcit
Details

Description zacattackcit 2016-06-17 21:09:20 UTC
Created attachment 329974 [details]
Log file showing omx error

I've built gst-omx from source and am trying to run on a Raspberry Pi 3.  The versions in the Raspbian repo (version 1.4.4) works fine, but when using my compiled version, I hit the following error:

"Failed to negotiate RGBA for EGLImage"

and the CPU gets railed instead of the minimal 5-10% usage I expect.  A minimal pipeline that reproduces the error is this:

gst-launch-1.0 -vv -e rpicamsrc ! "video/x-h264,width=640,height=480" ! h264parse ! omxh264dec ! videoconvert ! glimagesink

A snippet of a log with GST_DEBUG=6 is attached.  I kept a couple thousand lines on either side of the error--the log is very verbose!  Let me know if there's anything else that would be useful to debug this.

Thanks!
Comment 1 Matthew Waters (ystreet00) 2016-06-19 02:52:45 UTC
Which version of gst-omx and gst-plugins-bad do you have?

Currently, gst-omx master requires gst-plugins-bad master for zerocopy support.

If you compile gst-omx from before https://cgit.freedesktop.org/gstreamer/gst-omx/commit/?id=27d2cdd45dcf0bbd58c753e4b33058607153c4c1 you can use gst-plugins-bad 1.4.x up until 1.8.x for zerocopy support.
Comment 2 zacattackcit 2016-06-19 21:25:01 UTC
I was using gst-omx from this commit (actually, one of yours by the looks of it!):

https://cgit.freedesktop.org/gstreamer/gst-omx/commit/?id=705d7722fbe05da17f2bedfdc34476382687c95f

and gst-plugins-bad from this commit:

https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=b9cfa34118e023efb046a9eac78f1b5c275bf839

Seems like from your explanation that these two commits should have been compatible, but I'll try updating gst-omx to 91dd92964d2171c6e43878f93cb0980101795cf2 (the dates on the commits are a bit closer).  Updating all of gstreamer to master or 1.8.2 would be possible, but painful since building for my target (RPi3) takes quite some time... does it seem like I'm on the right track?

Thanks!
Comment 3 Matthew Waters (ystreet00) 2016-06-20 06:09:16 UTC
Those commits look fine.

I have noticed there's something odd about that error when it pops up although haven't been through the code enough to find out what happens.  Forcing EGLImage with a capsfilter makes omxh264dec output memory:EGLImage anyway which makes it work.
Comment 4 Julien Isorce 2016-06-20 08:07:30 UTC
It seems that in your case the block https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n2543 does not lead to have "found" as TRUE.
(triggered by line 962: https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n962)
Comment 5 zacattackcit 2016-06-20 14:17:49 UTC
(In reply to Matthew Waters (ystreet00) from comment #3)
> Those commits look fine.
> 
> I have noticed there's something odd about that error when it pops up
> although haven't been through the code enough to find out what happens. 
> Forcing EGLImage with a capsfilter makes omxh264dec output memory:EGLImage
> anyway which makes it work.

I tried your workaround by adding a caps filter after omxh264dec like this:

gst-launch-1.0 -vv -e rpicamsrc ! "video/x-h264,width=640,height=480" ! h264parse ! omxh264dec ! "video/x-raw(memory:EGLImage),format=RGBA,width=640,height=480" ! videoconvert ! glimagesink

but still get the same error.  Can you post the command that made it work for you?
Comment 6 Matthew Waters (ystreet00) 2016-06-20 14:39:38 UTC
The error will still appear, just that it's actually outputting EGLImage's instead of falling back.
Comment 7 zacattackcit 2016-06-20 22:42:52 UTC
I get an image displayed, the problem is that the CPU jumps up to ~270% which causes stuttering and artifacts.  In my earlier testing of version 1.4.4 I had low (~15%) CPU usage, which made sense given the use of hardware acceleration on both the source and decoder.

I'm trying to use master now...will see if all the EGL stuff in there helps.
Comment 8 zacattackcit 2016-06-21 19:23:51 UTC
Reporting back about the status of master: still get the same error, but the behavior is different.  I only get 1 frame of video in the glimagesink, no new frames come in even though the camera is still sending them out.  CPU stays low, but that might just be because nothing is actually going through the pipeline...

Here is what I'm using to test now:

gst-launch-1.0 -vv -e rpicamsrc preview=false ! "video/x-h264,width=640,height=480" ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=pithree port=5000

gst-launch-1.0 -vv -e udpsrc port=5000 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! glimagesink


I'll attach another debug.log file with the results of testing on master.
Comment 9 zacattackcit 2016-06-21 19:26:46 UTC
Created attachment 330167 [details]
Master debug log, freezes after first frame

I removed the first 0.5 seconds of the log.  At GST_DEBUG=6, we get to 3600 KB quickly!
Comment 10 zacattackcit 2016-07-11 15:36:26 UTC
Update on this: I went allll the way back to gstreamer 1.4.4 (the version in the raspbian repos) and still had problems with my test pipeline.  So, I can't confirm whether or not this bug is actually a problem with master, or with my build flags for gst-omx.

If anyone has a cerbero recipe for gst-omx that is known to work on the RPi, that would be a big help.
Comment 11 Philippe Normand 2016-12-01 17:38:35 UTC
I can't reproduce this issue on my RPi2 with gst git master (1.11).
Comment 12 minfrin 2016-12-11 01:54:21 UTC
The origin of this error message is in the following code:

      /* try to negotiate with caps feature */
      GST_DEBUG_OBJECT (self,
          "Try to negotiate with feature %s",
          GST_CAPS_FEATURE_MEMORY_GL_MEMORY);
      if (!gst_video_decoder_negotiate (GST_VIDEO_DECODER (self))) {

        GST_DEBUG_OBJECT (self,
            "Failed to negotiate with feature %s",
            GST_CAPS_FEATURE_MEMORY_GL_MEMORY);

        if (state->caps)
          gst_caps_replace (&state->caps, NULL);

        /* fallback: try to use EGLImage even if it is not in the caps feature */
        if (!gst_video_decoder_negotiate (GST_VIDEO_DECODER (self))) {
          gst_video_codec_state_unref (state);
          GST_ERROR_OBJECT (self, "Failed to negotiate RGBA for EGLImage");
          GST_VIDEO_DECODER_STREAM_UNLOCK (self);
          goto no_egl;
        }
      }

What is happening here is that first an attempt is made to negotiate downstream with GLMemory turned on and the format hard coded to RGBA. If this negotiation attempt fails, we then attempt to negotite downstream a second time without GLMemory, and the format hard coded to RGBA. If this second attempt to negotiate fails, we get "Failed to negotiate RGBA for EGLImage".

The question here is why did both negotiations fail. In my case, I get a failure on a pipeline that is effectively "omxh264dec ! omxh264enc" because omxh264enc doesn't support RGBA. In your case, examine the debug logs of the videoconvert and glimagesink elements to monitor how their caps are negotiated.
Comment 13 Tom Mitchell 2017-01-25 09:16:53 UTC
I also have the same problem with this pipeline. I use the latest sources from git 1.11.01.

gst-launch-1.0 souphttpsrc location=<MY_URL> ! image/jpeg,framerate=10/1 ! jpegparse ! queue2 use-buffering=true ! omxmjpegdec ! videoconvert ! videoscale add-borders=true ! video/x-raw,format=RGBA,width=800,height=480 ! glimagesink

I get the same error and I also have artifacts and stuttering video. 
CPU usage here is 90-110%.

On the version in the raspberry pi repository the same pipeline has to be used with 'format=RGB', 'format=RGBA' doesn't work.

I have to use: 
gst-launch-1.0 souphttpsrc location=<MY_URL> ! image/jpeg,framerate=10/1 ! jpegparse ! queue2 use-buffering=true ! omxmjpegdec ! videoconvert ! videoscale add-borders=true ! video/x-raw,format=RGB,width=800,height=480 ! glimagesink

CPU usage here is 23 - 30% and the video is displayed perfect.
Comment 14 Robert Mast 2017-02-11 21:30:26 UTC
Trying to get a hardware-transcoder running on my RPi3 I have several issues, including this one.
0:00:03.423813823 21763  0x1983d50 ERROR            omxvideodec gstomxvideodec.c:974:gst_omx_video_dec_reconfigure_output_port:<omxmpeg2videodec-omxmpeg2videodec0> Failed to negotiate RGBA for EGLImage


gst-launch-1.0 --gst-debug omx*:5 souphttpsrc location="http://127.0.0.1:9981/stream/channel/41ceb7b6ef96f869d7d6c78b6b3a27a6?ticket=24A946694272F14A9994F5F2280E466AB495F745?profile=pass" num-buffers=-1 ! tsdemux name=demux demux. ! queue ! mpegvideoparse ! omxmpeg2videodec  >Filebug.txt 2>&1

All gstreamer-builds are 1.10.3, libvpx-1.5.0.
Comment 15 minfrin 2017-02-11 22:09:22 UTC
Can you try this patch: https://bugzilla.gnome.org/show_bug.cgi?id=775948
Comment 16 Robert Mast 2017-02-12 13:06:51 UTC
By the way, there were also all kind of EGL-errors, and a negotiation about EGL Memory also went wrong.

OSMC gave a bad smiley (X couldn't start), and 

sudo apt-get remove libegl1-mesa*

restored the EGL-version that enables OSMC to run again, discussed here:

https://www.raspberrypi.org/forums/viewtopic.php?p=482554

So probably a dll-hell issue in Raspbian/OSMC concerning EGL is also disturbing the case.
Comment 17 Robert Mast 2017-02-12 16:08:44 UTC
(In reply to minfrin from comment #15)
> Can you try this patch: https://bugzilla.gnome.org/show_bug.cgi?id=775948

With that patch at least the error is now gone and the pipeline stays alive, so my long lasting adventure of finally getting this transcoding to work can continue. 

Perhaps someone with more experience can confirm it really does what it has to do.
Comment 18 Julien Isorce 2017-05-22 09:23:57 UTC
Hi minfrin@sharp.fm , in comment 12 it is not clear, it looks the pipeline reported in comment 0 works for you, was it on rpi3 ?

Hi Matthew, in comment 3 should it memory:GLMemory instead ? (since https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/gst-libs/gst/gl/egl?id=5498e97a11c2627af22560cdc949d8f95883210e and some other commits)

Hi Tom, in comment 13 I think the high cpu usage is because forcing video/x-raw makes the decoder to output to system memory. Also if the frame size or color is not the same , then videoconvert ! videoscale won't work in passthrough, so forcing cpu mode instead of zero (system memory) copy.

Hi Robert, Hi minfrin, let's continue the transcoding issue on https://bugzilla.gnome.org/show_bug.cgi?id=775948 because it looks a different problem than this current issue tracker.

To sum-up according to comment 11 the pipeline reported on comment 0 works on rpi2. 

The bug being reported 1 year ago, can any one with a rpi3 try the exact pipeline reported on comment 0 with a recent gst ? Worth to try different pipelines, like without videoconvert, and from a media file too, thx.
Comment 19 Matthew Waters (ystreet00) 2017-05-22 09:36:44 UTC
(In reply to Julien Isorce from comment #18)
> Hi Matthew, in comment 3 should it memory:GLMemory instead ?

Yes
Comment 20 Matthew Waters (ystreet00) 2018-05-06 13:24:24 UTC
Does this still occur with sufficient GPU memory?