GNOME Bugzilla – Bug 721925
video-sink: add overlay composition support
Last modified: 2014-09-04 09:15:46 UTC
This patch add overlay composition rendering support to the GLTextureUploadRenderer. It is used for example for subtitles rendering with an hardware decoder capable of using the GLTextureUploadMeta API. In order to do such, the GLTextureUploadRenderer uses an offscreen framebuffer. For each rectangle that should be composited a new overlay transparent texture is created, then the rectangle region is applied on it, then the overlay texture is composited onto the input texture using a shader. This procedure is repeated for each rectangle using the last result of the framebuffer as input. There might be room for improvement here since, a copy of the frambuffer texture is done, I didn't find a way to swap the texture of the framebuffer using the cogl api. Development branch (including the deinterlace feature) can be found here: http://cgit.collabora.com/git/user/mateo/clutter-gst.git/log/?h=clutter-gst-2.0/deinterlace-overlay-gltextureuploadrenderer
Created attachment 265906 [details] [review] [PATCH 1/3] video-sink: correctly reference the upload texture in the GLTextureUploadRenderer
Created attachment 265907 [details] [review] [PATCH 2/3] video-sink: fix gl_target type in the GLTextureUploadRenderer
Created attachment 265908 [details] [review] [PATCH 3/3] video-sink: add overlay composition rendering support to the GLTextureUploadRenderer
I'm not a big fad of doing shader composition. Unless I'm mistaken, it means that subtitle buffers are still negotiated base on the video size rather then the display size. Might be ok as a first step, but it's not really the right direction in order to achieve high quality subtitle and close captioning.
Created attachment 266487 [details] [review] [PATCH 3/3] video-sink: add overlay composition rendering support to the GLTextureUploadRenderer Fixes a material leak.
*** Bug 674547 has been marked as a duplicate of this bug. ***
Created attachment 270929 [details] [review] video-sink: add overlay composition support Patch updated. Overlay composition is now implemented using a proper clutter offscreen effect and is available for all formats supported by cluttersink. The patch applies on-top of the colorbalance patch: https://bugzilla.gnome.org/show_bug.cgi?id=663204 Repo: http://cgit.collabora.com/git/user/mateo/clutter-gst.git/log/?h=colorbalance/overlay
Created attachment 271025 [details] [review] video-sink: add overlay composition support Patch updated, the overlay texture was not unref after being used by the material in ClutterGstOverlayEffect.paint_target.
Created attachment 271026 [details] [review] video-sink: add overlay composition support Patch updated, adding missing cogl object unref on the overlay texture in the video-sink. Sorry the noise.
Created attachment 271030 [details] [review] video-sink: add overlay composition support Correct version of the patch attached, unref the cogl material related to the overlay in ClutterGstOverlayEffect.paint_target instead of unref the overlay cogl texture.
I just push 2 commits to master to implement overlay composition support in a slightly different way : https://git.gnome.org/browse/clutter-gst/commit/?id=541e1078cbbe61c1a5da61ef53c848b2dc3f5fe3 https://git.gnome.org/browse/clutter-gst/commit/?id=3fe0658a4555abd324cff1bd0ea805a5f75bda63 The idea is to create separate nodes in the render graph to avoid writing onto each video frame. That way the video frames are never accessed by the CPU if you have decoding done in hardware. Thanks a lot for submitting the patch though.
This is only clutter-gst 3.0 material, right? Could you add some examples of using it?
There is nothing special about using this feature. It just works by default.