GNOME Bugzilla – Bug 729278
gl: avoid using an extra FBO pass in some basic cases
Last modified: 2014-05-29 10:16:29 UTC
1- Currently "videotestsrc ! video/x-raw, format=RGBA/RGB/RGBx ! glimagesink" use a shader whereas there is no need. (Talking about this we should compare perf using BGRA instead as RGBA for our texture format, or maybe it should be configurable or bases on what the windowing system prefers. And choose a matching config on egl also) 2- Currently "videotestsrc ! video/x-raw, format=I420 ! glimagesink" uses a shader to do the color convertion, it's fine but we use a FBO for that which can be avoided. Indeed we could draw directly to the window surface. 3- Currently "videotestsrc ! glcolorscale ! "GL" ! fakesink" uses 2 FBO, one for the upload and one that just does the identity. The later should be avoided. See here in_info and out_info are the same http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglupload.c#n593 whereas everything is there to avoid this extra FBO. Same with download "gleffects ! glcolorscale ! ximagesink" http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstgldownload.c#n341, in and out infos are the same. The second FBO pass can be avoided. And same with "videotestsrc ! glcolorscale ! ximagesink".
Any ideas ? I think it can be seen as a follow-up to the lazy gl upload stuffs :)
All of this requires changes to colorconvert to code up these fast paths.
4- I also see between each gl elements, lets say gleffects1 ! gleffect2 ! gleffects3 that gstglcolorconvert is used whereas it's RGBA between each filter. I mean an additional FBO pass (which does the identity, no scale, no colorconvert) is used by gstglupload in the input of each glfilter. i.e. http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst-libs/gst/gl/gstglupload.c#n265 we could avoid an FBO pass in some cases.
This fixes the gl element to gl element case commit a4f0fbe4c3cd1eaad53f922ac33749c517896852 Author: Matthew Waters <ystreet00@gmail.com> Date: Mon May 12 21:51:38 2014 +1000 gl/upload: avoid performing color conversion when there is no need One such example is when the buffer contains GstGLMemory in the RGBA format https://bugzilla.gnome.org/show_bug.cgi?id=729278
Ok, all of these cases are as a result of glcolorconvert/glupload/gldownload implementations/interoperability. Also, case 1 and 3 are functionally the same (avoiding the shader conversion). The current process of say a glfilter is like so: input buffer -> upload -> colorconvert -> filter transform -> colorconvert -> download -> output buffer. This is a list of changes I would do: 1. separate the colorconvert from upload and download objects 2. separate the different upload/download cases into different subobjects. 3. have upload/download/colorconvert allocate their own output buffers (from a glbufferpool). 4. use GstBuffers everywhere I believe these changes would solve the three cases. Each object gets the choice to reuse existing or allocate new buffers that are cached in a (possibly shared) pool and thus can optimise the identity transformations away. It would also be easier to understand and maintain. Doing it like this essentially makes the upload/colorconvert/download essentially mini-GstElements and it might be prudent to upgrade them (or provide wrappers) to normal GstElements and plug them inside a glfilter/glmixer like bin as needed. Thoughts?
For sure suggestions 3 and 4 would be great! Why not just having GstGLColorconvert work in passthrough if no colorconversion is needed ? i.e., by directly setting the input gstbuffer as the output gstbuffer.
Sure, that will solve this bug. However the move to GstBuffer everywhere would require the upload/colorconvert/download to use a bufferpool in order to avoid allocating a destroying buffers and thus memorys/textures. That requires separating the upload from glbufferpool which requires separating at least the upload_meta case from all of the other upload paths.
commit f7f92e6d7a25a57907833bd66915f49fdc871fef Author: Matthew Waters <ystreet00@gmail.com> Date: Thu May 29 15:50:56 2014 +1000 gl/colorconvert: allocate output buffers Allows the nop optimisation by simply reffing the input buffer. commit a181460755cbd0124f8bb6bf447cb6a050e24fad Author: Matthew Waters <ystreet00@gmail.com> Date: Thu May 29 16:11:20 2014 +1000 gl/colorconvert: optimise the same format case simply return the input buffer unchanged
Is anything left to be done here?
There's still case 2 (conversion on framebuffer id 0) which can probably move to its own bug, and case 3 (glcolorscale identity).
Great job! To sum-up case 1 (#1) and 4 (#3) have been fixed in this bug. Also: For case 2, I opened https://bugzilla.gnome.org/show_bug.cgi?id=730927 For case 3, I opened https://bugzilla.gnome.org/show_bug.cgi?id=730928