GNOME Bugzilla – Bug 753886
Poor GLImageSink performance on Raspberry Pi
Last modified: 2016-05-15 08:47:12 UTC
(Following on from https://bugzilla.gnome.org/show_bug.cgi?id=752376 ) In the 1.6 Release Candidate (1.5.90), there seems to have been a regression in OpenGL performance with regard to the GLImageSink. In the 1.4.5 release, I could play back a normal 1080p video fine on my Raspberry Pi 2, but in the new 1.5.90 release candidate the video is constantly stalling and end playback is only managing about 1 frame per second. This is also reproducible using videotestsrc: $ gst-launch-1.0 videotestsrc ! video/x-raw,width=1920,height=1080 ! glimagesink The above will result in GStreamer printing the following over and over again: WARNING: from element /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: A lot of buffers are being dropped. Additional debug info: gstbasesink.c(2846): gst_base_sink_is_too_late (): /GstPlayBin:playbin0/GstPlaySink:playsink/GstBin:vbin/GstGLImageSinkBin:glimagesinkbin0/GstGLImageSink:sink: There may be a timestamping problem, or this computer is too slow. The video output is sped up a bit if I add "sync=false" to the glimagesink here, but it's still noticeably slower than the old release was, although it does stop complaining about timestamping problems. That being said, I don't actually know what the sync= attribute does, the online documentation for gst-plugins-bad appears to have gone missing? I'm currently running rpi-userland b834074, and I've confirmed that 1.4.5 worked correctly using this firmware.
I have the same problem with gstreamer 1.7.1
Created attachment 318921 [details] Caps for gst-launch-1.0 filesrc location =/path/ ! qtdemux ! h264parse ! omxh264dec ! glimagesink
The caps look correct. glimagesink cannot render EGLImage's directly so it's converted by glupload. That has not changed since glimagesink was introduced in 1.4 so that's not this problem.
I managed to get qmlglsink working without performance drops on rpi - in it's initial state (and well after enabling it to work on rpi by using android code for egl) it's as slow as glimagesink. I set caps to EGLimage bypassing glupload. Once I had created one texture and rendered every eglimage to that texture I started having no frame drops at 1080p. Just for reference this was my rendering code (modified ext/qt/gstqsgtexture.cpp). if (tex == 0) { gl->GenTextures(1, &tex); gl->BindTexture (GL_TEXTURE_2D, tex); gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST); gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE); gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); gl->TexImage2D ( GL_TEXTURE_2D, 0, GL_RGBA, 1920, 1080, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL ); } gl->BindTexture (GL_TEXTURE_2D, tex); gl->EGLImageTargetTexture2D (GL_TEXTURE_2D, gst_egl_image_memory_get_image (mem)); I don't know if this is any helpful but the only difference that I see compared to when I was using glupload is that glupload creates a lot of new textures (1 for every frame?) - I don't know if that's what impacts performance though.
(In reply to Sergey Borovkov from comment #4) > I managed to get qmlglsink working without performance drops on rpi - in > it's initial state (and well after enabling it to work on rpi by using > android code for egl) it's as slow as glimagesink. I set caps to EGLimage > bypassing glupload. Once I had created one texture and rendered every > eglimage to that texture I started having no frame drops at 1080p. Just for > reference this was my rendering code (modified ext/qt/gstqsgtexture.cpp). > if (tex == 0) { > gl->GenTextures(1, &tex); > gl->BindTexture (GL_TEXTURE_2D, tex); > gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST); > gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, > GL_NEAREST); > gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, > GL_CLAMP_TO_EDGE); > gl->TexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE); > gl->TexImage2D ( GL_TEXTURE_2D, 0, GL_RGBA, 1920, 1080, 0, GL_RGBA, > GL_UNSIGNED_BYTE, NULL ); > } > gl->BindTexture (GL_TEXTURE_2D, tex); > gl->EGLImageTargetTexture2D (GL_TEXTURE_2D, > gst_egl_image_memory_get_image (mem)); > > I don't know if this is any helpful but the only difference that I see > compared to when I was using glupload is that glupload creates a lot of new > textures (1 for every frame?) - I don't know if that's what impacts > performance though. That's correct. Current implementation of EGLImageMemory and GLUpload creates a texture and copies EGLImage's contents to the glUpload's texture for every frame. That makes huge performance drop. I did some job to remove those overheads, here is the related bugs: https://bugzilla.gnome.org/show_bug.cgi?id=760916 https://bugzilla.gnome.org/show_bug.cgi?id=760918
Thanks. I will test your patches next week.
Just tried your patches. Performance for glimagesink is very good. I had a couple of runs when it dropped couple of frames but was on a couple occasions only. There are some warning printed during playback though: invalid uninstantiatable type '<unknown>' in cast to 'GstGLContextEGL'
The "invalid uninstantiatable ..." error is due to the presence of the old egl plugin. Remove the cruft from you prefix or build and it should go away. If it worked it's because you have been lucky enough to load the newest one first. (it's an indicator of type clash in GObject registration).
Nicolas, I am kinda late with response, but I did a full rebuild of my buildroot with gstreamer and warnings never go away.
You may be picking up system libraries instead.
Has the speed issue been resolved? Until resolution, we are kind of stuck with 1.2.4 or alike for playing 1080p and even 720p on Raspberry Pi 2.
>You may be picking up system libraries instead. This is impossible, since I am running from chrooted environment. To make sure it was not on my side I rebuilt everything from scratch as well removing previous binaries. Yet I'm still getting this warning every time cast GstGLContextEGL happens.
The other option is that it's attempting to cast from a GLX context to an EGL context. If the warnings disappear when running with GST_GL_PLATFORM=egl then that's your issue. In any case, at backtrace at the error site (run with G_DEBUG=fatal-warnings inside gdb) would be useful.
Warning do not disappear when running with GST_GL_PLATFORM=egl. This is the backtrace. (For reference I am running following pipeline - filesrc ! decodebin ! glupload ! qmlglsink).
+ Trace 236003
The aforementioned patches increase the performance of omx*dec ! glimagesink *** This bug has been marked as a duplicate of bug 760916 ***