After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 751684 - Defaults to glimagesink even when GL is software rendered
Defaults to glimagesink even when GL is software rendered
Status: RESOLVED WONTFIX
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal normal
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2015-06-29 21:37 UTC by Olivier Crête
Modified: 2018-05-06 11:04 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
glimagesink: Downrank to marginal (1002 bytes, patch)
2015-06-30 21:10 UTC, Olivier Crête
reviewed Details | Review
WIP: downgrade glimagesink rank when using mesa's softpipe renderer (1.89 KB, patch)
2015-07-07 08:26 UTC, Julien Isorce
rejected Details | Review

Description Olivier Crête 2015-06-29 21:37:58 UTC
On a crappy arm board, playbin defaults to glimagesink, even though there is no hardware GL present, which gives terrible performance. ximagesink works much better. We may want to make glimagesink fail to get to READY if the Mesa is in software rendering mode. If needed, we can add a glsoftimagesink which has a very low rank and doesn't do that.
Comment 1 Nicolas Dufresne (ndufresne) 2015-06-30 01:56:43 UTC
It nice to be able to run gl stuff in Software GL when debugging. Can you think of something more complete, that would allow restoring the feature for testing and debugging ?
Comment 2 Olivier Crête 2015-06-30 19:45:30 UTC
@stormer: I suggest create a glsoftimagesink that has rank 0 and doesn't enforce usable GL. Other suggestion is to just downrank glimagesink.
Comment 3 Tim-Philipp Müller 2015-06-30 20:11:28 UTC
> It nice to be able to run gl stuff in Software GL when debugging.
> Can you think of something more complete, that would allow 
> restoring the feature for testing and debugging ?

You would still be able to use

  playbin video-sink=glimagesink

or gst-play --video-sink=glimagesink

etc.
Comment 4 Nicolas Dufresne (ndufresne) 2015-06-30 20:49:01 UTC
(In reply to Tim-Philipp Müller from comment #3)
> You would still be able to use
> 
>   playbin video-sink=glimagesink
> 
> or gst-play --video-sink=glimagesink
> 
> etc.

Not if the solution is to fail the ready transition when in software.
Comment 5 Tim-Philipp Müller 2015-06-30 20:58:06 UTC
True. Failing the state change seems like a bad idea to me if the sink can work fine.
Comment 6 Olivier Crête 2015-06-30 21:10:05 UTC
Created attachment 306438 [details] [review]
glimagesink: Downrank to marginal

Failing the ->ready transition seems not be an option, as the GL context is opened much later. Instead, just downrank glimagesink.

On desktop, where there is good OpenGL, xvimagesink will come up first,
on other platforms, OpenGL can't be trusted because it's either software (like
in a VM) or broken (like on embedded)., so let ximagesink come above.
Comment 7 Tim-Philipp Müller 2015-06-30 21:16:42 UTC
A downgrade would probably be ok. autovideosink is a fallback mechanism to make something work anyway, and it's potluck what sink you get anyway. And people who want to use gl will usually hard-code it anyway in their application or pipeline.
Comment 8 Julien Isorce 2015-07-01 21:40:57 UTC
Hi,

Have you compared ximagesink and glimagesink with the same sink video caps format ? Because when using playbin I suspect glimagesink will get yuv directly and imagesink rgb. Mesa's softpipe driver will be slow to convert yuv to rgb (+ scaling) I suspect.
You can try: playbin video-sink="video/x-raw, format=BGRx ! glimagesink". Same with ximagesink.

In any case there are 2 versions of the software mesa driver swrast: softpipe and llvmpipe. I guess the package you use does not provide this gallium llvmpipe driver which is much faster.

You can verify the presence with es2_info (not sure if it is in mesa-utils or another package) -> under GL_RENDERER field. Or with GST_DEBUG=*gl*:6 gst-launch-1.0 videotestsrc ! glimagesink.

Mesa's llvmpipe driver should work on arm. Also see: http://lists.freedesktop.org/archives/mesa-dev/2015-April/081986.html

If an arm based distribution provides llvmpipe driver, glimagesink should not be slower than ximagesink+orc.

The softpipe driver will definitely be slow on arm in most of the cases.

In GStreamer, at runtime we could check if glGetString (GL_RENDERER) contains "softpipe".

You do not need a GstGLContext to call gst_gl_context_default_get_proc_address(GST_GL_API_GLES2, "glGetString").

We could call that in gst-plugins-bad/ext/gl/gstopengl.c::plugin_init and downgrade the rank only if GL_RENDERER string contains "softpipe".

Mesa provides the HW driver vc4 for raspberry pi but I think it has not been released. So once available, glimagesink should be lower ranked than ximagesink.
Comment 9 Julien Isorce 2015-07-01 21:42:15 UTC
-lower +higher in last sentence :)
Comment 10 Matthew Waters (ystreet00) 2015-07-07 06:59:40 UTC
(In reply to Julien Isorce from comment #8)
> You do not need a GstGLContext to call
> gst_gl_context_default_get_proc_address(GST_GL_API_GLES2, "glGetString").

default_get_proc_address only looks within the GL shared library and doesn't perform {glX,egl,wgl,etc}GetProcAddress.  As a result there are cases where the default get_proc_address will fail to retrieve valid functions.

> We could call that in gst-plugins-bad/ext/gl/gstopengl.c::plugin_init and
> downgrade the rank only if GL_RENDERER string contains "softpipe".

We need a GL context current to be able to even think of calling the returned function pointer which is another complication.
Comment 11 Julien Isorce 2015-07-07 08:20:24 UTC
(In reply to Matthew Waters from comment #10)
> We need a GL context current to be able to even think of calling the
> returned function pointer which is another complication.

Ah right, sorry my mistake.
Comment 12 Julien Isorce 2015-07-07 08:26:19 UTC
Created attachment 306979 [details] [review]
WIP: downgrade glimagesink rank when using mesa's softpipe renderer

Draft patch that does not compile. It requires to make a bit of refactoring adding new helpers.
Maybe having this block as a helper could be useful for us and applications.
No context should be current by the time gst_init is called. Or we could check if there is one, keep the handle around and make it current again when done.
Comment 13 Matthew Waters (ystreet00) 2015-07-07 08:44:43 UTC
Review of attachment 306438 [details] [review]:

seems ok to me
Comment 14 Matthew Waters (ystreet00) 2015-07-07 08:48:12 UTC
Review of attachment 306979 [details] [review]:

I'm very hesitant to encourage this as the application might want to override display or context creation which cannot be done without GstContext and thus bus and pipeline.  Not to mention this is EGL, GLESv2 only.
Comment 15 Julien Isorce 2015-07-07 20:00:06 UTC
I have no other suggestions. I compared softpipe and llvmpipe on a slow x86_64. softpipe is really not praticable except if input is RGBA + small resolution. Even in this case ximagesink is faster.
llvmpipe is praticable but always a bit slower than videoconvert+ximagesink no matter the input size and colorspace.
So I suspect the result would be even more in favor of ximagesink on arm.

I was wondering, there is this Driver section in xorg.conf or elsewhere depending on the distro. But maybe not generic and a bit overkill too.
Comment 16 Julien Isorce 2015-07-21 06:40:29 UTC
Review of attachment 306438 [details] [review]:

"so let ximagesink come above."
we should at least only downrank on X11. If gst_gl_window_get_default_type return GST_GL_TYPE_WINDOW_X11 (this function can be extracted by splitting gst_gl_window_new, so without really creating a window).
Because the downgrade only serve case where the video sink is selected automatically. So we are fine if one wants to provides its own gl display and context it will force using glimagesink anyway. Or even use another gl sink.
Comment 17 Alexander Kanavin 2015-09-24 14:10:27 UTC
We've hit the same problem in the Yocto project:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=7787

so we'll be applying the rank downgrade patch until it (or something better) shows up in upstream.
Comment 18 Sebastian Dröge (slomo) 2016-11-01 18:56:59 UTC
Comment on attachment 306438 [details] [review]
glimagesink: Downrank to marginal

What's with this now?
Comment 19 Tim-Philipp Müller 2016-11-01 19:36:02 UTC
Not sure, are we waiting for opinions on whether to downgrade unconditionally or to downgrade only in an X environment?
Comment 20 Matthew Waters (ystreet00) 2016-11-10 15:23:49 UTC
There's another case of this kind of thing occurring in bug 768586 where on the RPi glimagesink is used but doesn't support the X11 window handle.  There are also other permutations of problems with the rank system that exist where one wants glimagesink over ximagesink (like on RPi for zerocopy decoding with playbin).  A simple downgrade everywhere is not the best idea for this reason.

If it's provable that xvimagesink is always available (and usable) where glimagesink is hardware accelerated, then the rank downgrade only on X systems is ok by me.
Comment 21 Matthew Waters (ystreet00) 2018-05-06 11:04:34 UTC
This probably isn't going to change.  If you need a specific video sink, you need to select it rather than using autovideosink.