GNOME Bugzilla – Bug 751684
Defaults to glimagesink even when GL is software rendered
Last modified: 2018-05-06 11:04:34 UTC
On a crappy arm board, playbin defaults to glimagesink, even though there is no hardware GL present, which gives terrible performance. ximagesink works much better. We may want to make glimagesink fail to get to READY if the Mesa is in software rendering mode. If needed, we can add a glsoftimagesink which has a very low rank and doesn't do that.
It nice to be able to run gl stuff in Software GL when debugging. Can you think of something more complete, that would allow restoring the feature for testing and debugging ?
@stormer: I suggest create a glsoftimagesink that has rank 0 and doesn't enforce usable GL. Other suggestion is to just downrank glimagesink.
> It nice to be able to run gl stuff in Software GL when debugging. > Can you think of something more complete, that would allow > restoring the feature for testing and debugging ? You would still be able to use playbin video-sink=glimagesink or gst-play --video-sink=glimagesink etc.
(In reply to Tim-Philipp Müller from comment #3) > You would still be able to use > > playbin video-sink=glimagesink > > or gst-play --video-sink=glimagesink > > etc. Not if the solution is to fail the ready transition when in software.
True. Failing the state change seems like a bad idea to me if the sink can work fine.
Created attachment 306438 [details] [review] glimagesink: Downrank to marginal Failing the ->ready transition seems not be an option, as the GL context is opened much later. Instead, just downrank glimagesink. On desktop, where there is good OpenGL, xvimagesink will come up first, on other platforms, OpenGL can't be trusted because it's either software (like in a VM) or broken (like on embedded)., so let ximagesink come above.
A downgrade would probably be ok. autovideosink is a fallback mechanism to make something work anyway, and it's potluck what sink you get anyway. And people who want to use gl will usually hard-code it anyway in their application or pipeline.
Hi, Have you compared ximagesink and glimagesink with the same sink video caps format ? Because when using playbin I suspect glimagesink will get yuv directly and imagesink rgb. Mesa's softpipe driver will be slow to convert yuv to rgb (+ scaling) I suspect. You can try: playbin video-sink="video/x-raw, format=BGRx ! glimagesink". Same with ximagesink. In any case there are 2 versions of the software mesa driver swrast: softpipe and llvmpipe. I guess the package you use does not provide this gallium llvmpipe driver which is much faster. You can verify the presence with es2_info (not sure if it is in mesa-utils or another package) -> under GL_RENDERER field. Or with GST_DEBUG=*gl*:6 gst-launch-1.0 videotestsrc ! glimagesink. Mesa's llvmpipe driver should work on arm. Also see: http://lists.freedesktop.org/archives/mesa-dev/2015-April/081986.html If an arm based distribution provides llvmpipe driver, glimagesink should not be slower than ximagesink+orc. The softpipe driver will definitely be slow on arm in most of the cases. In GStreamer, at runtime we could check if glGetString (GL_RENDERER) contains "softpipe". You do not need a GstGLContext to call gst_gl_context_default_get_proc_address(GST_GL_API_GLES2, "glGetString"). We could call that in gst-plugins-bad/ext/gl/gstopengl.c::plugin_init and downgrade the rank only if GL_RENDERER string contains "softpipe". Mesa provides the HW driver vc4 for raspberry pi but I think it has not been released. So once available, glimagesink should be lower ranked than ximagesink.
-lower +higher in last sentence :)
(In reply to Julien Isorce from comment #8) > You do not need a GstGLContext to call > gst_gl_context_default_get_proc_address(GST_GL_API_GLES2, "glGetString"). default_get_proc_address only looks within the GL shared library and doesn't perform {glX,egl,wgl,etc}GetProcAddress. As a result there are cases where the default get_proc_address will fail to retrieve valid functions. > We could call that in gst-plugins-bad/ext/gl/gstopengl.c::plugin_init and > downgrade the rank only if GL_RENDERER string contains "softpipe". We need a GL context current to be able to even think of calling the returned function pointer which is another complication.
(In reply to Matthew Waters from comment #10) > We need a GL context current to be able to even think of calling the > returned function pointer which is another complication. Ah right, sorry my mistake.
Created attachment 306979 [details] [review] WIP: downgrade glimagesink rank when using mesa's softpipe renderer Draft patch that does not compile. It requires to make a bit of refactoring adding new helpers. Maybe having this block as a helper could be useful for us and applications. No context should be current by the time gst_init is called. Or we could check if there is one, keep the handle around and make it current again when done.
Review of attachment 306438 [details] [review]: seems ok to me
Review of attachment 306979 [details] [review]: I'm very hesitant to encourage this as the application might want to override display or context creation which cannot be done without GstContext and thus bus and pipeline. Not to mention this is EGL, GLESv2 only.
I have no other suggestions. I compared softpipe and llvmpipe on a slow x86_64. softpipe is really not praticable except if input is RGBA + small resolution. Even in this case ximagesink is faster. llvmpipe is praticable but always a bit slower than videoconvert+ximagesink no matter the input size and colorspace. So I suspect the result would be even more in favor of ximagesink on arm. I was wondering, there is this Driver section in xorg.conf or elsewhere depending on the distro. But maybe not generic and a bit overkill too.
Review of attachment 306438 [details] [review]: "so let ximagesink come above." we should at least only downrank on X11. If gst_gl_window_get_default_type return GST_GL_TYPE_WINDOW_X11 (this function can be extracted by splitting gst_gl_window_new, so without really creating a window). Because the downgrade only serve case where the video sink is selected automatically. So we are fine if one wants to provides its own gl display and context it will force using glimagesink anyway. Or even use another gl sink.
We've hit the same problem in the Yocto project: https://bugzilla.yoctoproject.org/show_bug.cgi?id=7787 so we'll be applying the rank downgrade patch until it (or something better) shows up in upstream.
Comment on attachment 306438 [details] [review] glimagesink: Downrank to marginal What's with this now?
Not sure, are we waiting for opinions on whether to downgrade unconditionally or to downgrade only in an X environment?
There's another case of this kind of thing occurring in bug 768586 where on the RPi glimagesink is used but doesn't support the X11 window handle. There are also other permutations of problems with the rank system that exist where one wants glimagesink over ximagesink (like on RPi for zerocopy decoding with playbin). A simple downgrade everywhere is not the best idea for this reason. If it's provable that xvimagesink is always available (and usable) where glimagesink is hardware accelerated, then the rank downgrade only on X systems is ok by me.
This probably isn't going to change. If you need a specific video sink, you need to select it rather than using autovideosink.