GNOME Bugzilla – Bug 767801
Failed to negotiate RGBA for EGLImage on Raspberry Pi 3
Last modified: 2018-08-28 05:03:42 UTC
Created attachment 329974 [details] Log file showing omx error I've built gst-omx from source and am trying to run on a Raspberry Pi 3. The versions in the Raspbian repo (version 1.4.4) works fine, but when using my compiled version, I hit the following error: "Failed to negotiate RGBA for EGLImage" and the CPU gets railed instead of the minimal 5-10% usage I expect. A minimal pipeline that reproduces the error is this: gst-launch-1.0 -vv -e rpicamsrc ! "video/x-h264,width=640,height=480" ! h264parse ! omxh264dec ! videoconvert ! glimagesink A snippet of a log with GST_DEBUG=6 is attached. I kept a couple thousand lines on either side of the error--the log is very verbose! Let me know if there's anything else that would be useful to debug this. Thanks!
Which version of gst-omx and gst-plugins-bad do you have? Currently, gst-omx master requires gst-plugins-bad master for zerocopy support. If you compile gst-omx from before https://cgit.freedesktop.org/gstreamer/gst-omx/commit/?id=27d2cdd45dcf0bbd58c753e4b33058607153c4c1 you can use gst-plugins-bad 1.4.x up until 1.8.x for zerocopy support.
I was using gst-omx from this commit (actually, one of yours by the looks of it!): https://cgit.freedesktop.org/gstreamer/gst-omx/commit/?id=705d7722fbe05da17f2bedfdc34476382687c95f and gst-plugins-bad from this commit: https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/?id=b9cfa34118e023efb046a9eac78f1b5c275bf839 Seems like from your explanation that these two commits should have been compatible, but I'll try updating gst-omx to 91dd92964d2171c6e43878f93cb0980101795cf2 (the dates on the commits are a bit closer). Updating all of gstreamer to master or 1.8.2 would be possible, but painful since building for my target (RPi3) takes quite some time... does it seem like I'm on the right track? Thanks!
Those commits look fine. I have noticed there's something odd about that error when it pops up although haven't been through the code enough to find out what happens. Forcing EGLImage with a capsfilter makes omxh264dec output memory:EGLImage anyway which makes it work.
It seems that in your case the block https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n2543 does not lead to have "found" as TRUE. (triggered by line 962: https://cgit.freedesktop.org/gstreamer/gst-omx/tree/omx/gstomxvideodec.c#n962)
(In reply to Matthew Waters (ystreet00) from comment #3) > Those commits look fine. > > I have noticed there's something odd about that error when it pops up > although haven't been through the code enough to find out what happens. > Forcing EGLImage with a capsfilter makes omxh264dec output memory:EGLImage > anyway which makes it work. I tried your workaround by adding a caps filter after omxh264dec like this: gst-launch-1.0 -vv -e rpicamsrc ! "video/x-h264,width=640,height=480" ! h264parse ! omxh264dec ! "video/x-raw(memory:EGLImage),format=RGBA,width=640,height=480" ! videoconvert ! glimagesink but still get the same error. Can you post the command that made it work for you?
The error will still appear, just that it's actually outputting EGLImage's instead of falling back.
I get an image displayed, the problem is that the CPU jumps up to ~270% which causes stuttering and artifacts. In my earlier testing of version 1.4.4 I had low (~15%) CPU usage, which made sense given the use of hardware acceleration on both the source and decoder. I'm trying to use master now...will see if all the EGL stuff in there helps.
Reporting back about the status of master: still get the same error, but the behavior is different. I only get 1 frame of video in the glimagesink, no new frames come in even though the camera is still sending them out. CPU stays low, but that might just be because nothing is actually going through the pipeline... Here is what I'm using to test now: gst-launch-1.0 -vv -e rpicamsrc preview=false ! "video/x-h264,width=640,height=480" ! h264parse ! rtph264pay config-interval=1 pt=96 ! udpsink host=pithree port=5000 gst-launch-1.0 -vv -e udpsrc port=5000 ! application/x-rtp, payload=96 ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! glimagesink I'll attach another debug.log file with the results of testing on master.
Created attachment 330167 [details] Master debug log, freezes after first frame I removed the first 0.5 seconds of the log. At GST_DEBUG=6, we get to 3600 KB quickly!
Update on this: I went allll the way back to gstreamer 1.4.4 (the version in the raspbian repos) and still had problems with my test pipeline. So, I can't confirm whether or not this bug is actually a problem with master, or with my build flags for gst-omx. If anyone has a cerbero recipe for gst-omx that is known to work on the RPi, that would be a big help.
I can't reproduce this issue on my RPi2 with gst git master (1.11).
The origin of this error message is in the following code: /* try to negotiate with caps feature */ GST_DEBUG_OBJECT (self, "Try to negotiate with feature %s", GST_CAPS_FEATURE_MEMORY_GL_MEMORY); if (!gst_video_decoder_negotiate (GST_VIDEO_DECODER (self))) { GST_DEBUG_OBJECT (self, "Failed to negotiate with feature %s", GST_CAPS_FEATURE_MEMORY_GL_MEMORY); if (state->caps) gst_caps_replace (&state->caps, NULL); /* fallback: try to use EGLImage even if it is not in the caps feature */ if (!gst_video_decoder_negotiate (GST_VIDEO_DECODER (self))) { gst_video_codec_state_unref (state); GST_ERROR_OBJECT (self, "Failed to negotiate RGBA for EGLImage"); GST_VIDEO_DECODER_STREAM_UNLOCK (self); goto no_egl; } } What is happening here is that first an attempt is made to negotiate downstream with GLMemory turned on and the format hard coded to RGBA. If this negotiation attempt fails, we then attempt to negotite downstream a second time without GLMemory, and the format hard coded to RGBA. If this second attempt to negotiate fails, we get "Failed to negotiate RGBA for EGLImage". The question here is why did both negotiations fail. In my case, I get a failure on a pipeline that is effectively "omxh264dec ! omxh264enc" because omxh264enc doesn't support RGBA. In your case, examine the debug logs of the videoconvert and glimagesink elements to monitor how their caps are negotiated.
I also have the same problem with this pipeline. I use the latest sources from git 1.11.01. gst-launch-1.0 souphttpsrc location=<MY_URL> ! image/jpeg,framerate=10/1 ! jpegparse ! queue2 use-buffering=true ! omxmjpegdec ! videoconvert ! videoscale add-borders=true ! video/x-raw,format=RGBA,width=800,height=480 ! glimagesink I get the same error and I also have artifacts and stuttering video. CPU usage here is 90-110%. On the version in the raspberry pi repository the same pipeline has to be used with 'format=RGB', 'format=RGBA' doesn't work. I have to use: gst-launch-1.0 souphttpsrc location=<MY_URL> ! image/jpeg,framerate=10/1 ! jpegparse ! queue2 use-buffering=true ! omxmjpegdec ! videoconvert ! videoscale add-borders=true ! video/x-raw,format=RGB,width=800,height=480 ! glimagesink CPU usage here is 23 - 30% and the video is displayed perfect.
Trying to get a hardware-transcoder running on my RPi3 I have several issues, including this one. 0:00:03.423813823 21763 0x1983d50 ERROR omxvideodec gstomxvideodec.c:974:gst_omx_video_dec_reconfigure_output_port:<omxmpeg2videodec-omxmpeg2videodec0> Failed to negotiate RGBA for EGLImage gst-launch-1.0 --gst-debug omx*:5 souphttpsrc location="http://127.0.0.1:9981/stream/channel/41ceb7b6ef96f869d7d6c78b6b3a27a6?ticket=24A946694272F14A9994F5F2280E466AB495F745?profile=pass" num-buffers=-1 ! tsdemux name=demux demux. ! queue ! mpegvideoparse ! omxmpeg2videodec >Filebug.txt 2>&1 All gstreamer-builds are 1.10.3, libvpx-1.5.0.
Can you try this patch: https://bugzilla.gnome.org/show_bug.cgi?id=775948
By the way, there were also all kind of EGL-errors, and a negotiation about EGL Memory also went wrong. OSMC gave a bad smiley (X couldn't start), and sudo apt-get remove libegl1-mesa* restored the EGL-version that enables OSMC to run again, discussed here: https://www.raspberrypi.org/forums/viewtopic.php?p=482554 So probably a dll-hell issue in Raspbian/OSMC concerning EGL is also disturbing the case.
(In reply to minfrin from comment #15) > Can you try this patch: https://bugzilla.gnome.org/show_bug.cgi?id=775948 With that patch at least the error is now gone and the pipeline stays alive, so my long lasting adventure of finally getting this transcoding to work can continue. Perhaps someone with more experience can confirm it really does what it has to do.
Hi minfrin@sharp.fm , in comment 12 it is not clear, it looks the pipeline reported in comment 0 works for you, was it on rpi3 ? Hi Matthew, in comment 3 should it memory:GLMemory instead ? (since https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/commit/gst-libs/gst/gl/egl?id=5498e97a11c2627af22560cdc949d8f95883210e and some other commits) Hi Tom, in comment 13 I think the high cpu usage is because forcing video/x-raw makes the decoder to output to system memory. Also if the frame size or color is not the same , then videoconvert ! videoscale won't work in passthrough, so forcing cpu mode instead of zero (system memory) copy. Hi Robert, Hi minfrin, let's continue the transcoding issue on https://bugzilla.gnome.org/show_bug.cgi?id=775948 because it looks a different problem than this current issue tracker. To sum-up according to comment 11 the pipeline reported on comment 0 works on rpi2. The bug being reported 1 year ago, can any one with a rpi3 try the exact pipeline reported on comment 0 with a recent gst ? Worth to try different pipelines, like without videoconvert, and from a media file too, thx.
(In reply to Julien Isorce from comment #18) > Hi Matthew, in comment 3 should it memory:GLMemory instead ? Yes
Does this still occur with sufficient GPU memory?