GNOME Bugzilla – Bug 751157
basetextoverlay: Pass down GstVideoOverlayComposition buffers from upstream, receive window size event
Last modified: 2015-07-22 21:12:04 UTC
These patches are part of implementing the GstVideoOverlayComposition in the glimagesink. https://bugzilla.gnome.org/show_bug.cgi?id=745107
Created attachment 305559 [details] [review] Pass down overlay composition buffers from upstream This patch allows chaining multiple textoverlay elements without losing the OverlayComposition meta data. It makes pipelines like this possible: gst-launch-1.0 videotestsrc ! \ textoverlay text="text1" valignment="top" font-desc="sans bold 30" ! \ textoverlay text="text2" halignment="right" font-desc="sans bold 30" ! \ textoverlay text="text3" halignment="left" font-desc="sans bold 30" ! \ glimagesink
Created attachment 305561 [details] [review] adjust render size from downstream window event This patch provides a possibility to render text for the correct screen size of the sink. The sink needs to send an upstream window size event. xvimagesink http://i.imgur.com/zjEDnGI.png glimagesink http://i.imgur.com/2uO1l3z.png
Created attachment 305562 [details] [review] Log if overlay meta nego was succesfull Log if overlay meta nego was succesfull
Created attachment 306504 [details] [review] basetestoverlay: Pass down meta buffers from upstream that supports GstVideoOverlayComposition
Created attachment 306505 [details] [review] basetextoverlay: Receive window size event and adjust rendering
Created attachment 306506 [details] [review] basetextoverlay: Log GstVideoOverlayComposition negotiation
Created attachment 306507 [details] [review] basetextoverlay: Send caps event to receive current allocation query To make the textoverlay receive a current window resolution from the sink, a send caps event needs to be sent, to not receive a cached allocation query.
That last patch is outdated, it relied on a bug in fact. The code now is skipping the allocation query when upstream or configured caps have the overlay feature. Obviously this breaks the window sizing stuff. I have a patch of my own that make it do an allocation query all the time, I'll share later. Now I can't merge this, because it broke the normal case, the case where we render in video size (when the sink does not report the window size). I need to look into this.
Attachment 306504 [details] pushed as f128666 - basetestoverlay: Pass down meta buffers from upstream that supports GstVideoOverlayComposition Attachment 306505 [details] pushed as 91a615f - basetextoverlay: Receive window size event and adjust rendering Attachment 306506 [details] pushed as d1808a5 - basetextoverlay: Log GstVideoOverlayComposition negotiation
Also added few more patches to fix the remaining issues I saw. a2e4ccc38bd108de9bba8b366acaab7e218e3a1b basetextoverlay: Fix upstream composition handling d10959fd3613b1f08d975a948fd9f9dfef5def66 basetextoverlay: Clear reconfigure flags before negotation 2308014963b3d019a51fe80c2e539e3792c283f8 basetestoverlay: Always query window dimension
Created attachment 307801 [details] [review] basetextoverlay: Send caps event to receive current allocation query Sorry, I forgot to propose this patch! Without it the negotation lags behind. To make the textoverlay receive a current window resolution from the sink, a send caps event needs to be sent, to not receive a cached allocation query.
Review of attachment 307801 [details] [review]: I implemented my own.
commit a64a343077851bc2bf6ae427ed8fbc4ad861c944 Author: Nicolas Dufresne <nicolas.dufresne@collabora.co.uk> Date: Mon Jul 20 15:11:06 2015 -0400 basetextoverlay: Send caps before doing allocation query This is currently a limitation of BaseTransform base class. Which means pretty much every filters out there. http://bugzilla.gnome.org/show_bug.cgi?id=751157
And this one to, for the cross-reference. If we don't always do the allocation query, then we endup without outdate window dimension. commit 2308014963b3d019a51fe80c2e539e3792c283f8 Author: Nicolas Dufresne <nicolas.dufresne@collabora.co.uk> Date: Mon Jul 20 15:55:07 2015 -0400 basetestoverlay: Always query window dimension Remove the optimization to skip allocation query so we can always have the latest window size information. Also, correctly deal with the case where there is no window size information. http://bugzilla.gnome.org/show_bug.cgi?id=751157
I lately notice that VideoCompositionMeta coordinate are relative to the window "sometimes" now. They should always be relative to the stream.
Created attachment 307870 [details] [review] basetextoverlay: Ensure meta coordinate are in stream scale The GstVideoOverlayComposition meta coordinates should always be in stream scale, regardless of the window size downstream. This way the sink can always scale the composition if the window size have changed after a buffer (with his meta) was rendered before.
Created attachment 307871 [details] [review] basetextoverlay: Reorder and cleanup class attribute Also add a minimum amount of comment so we can understand what is doing what.
Created attachment 307882 [details] [review] basetextoverlay: Ensure meta coordinate are in stream scale The GstVideoOverlayComposition meta coordinates should always be in stream scale, regardless of the window size downstream. This way the sink can always scale the composition if the window size have changed after a buffer (with his meta) was rendered before.
Created attachment 307883 [details] [review] basetextoverlay: Improve debug trace
In the previews patch, I forgot to scale the rendering. Also added the debug trace I have used to debug this.
Leaving open for now as I'll be doing more testing. Attachment 307871 [details] pushed as db81a73 - basetextoverlay: Reorder and cleanup class attribute Attachment 307882 [details] pushed as 48f877e - basetextoverlay: Ensure meta coordinate are in stream scale
One more iteration, works quite well now (and that reduce the number of spurious negotiation). commit f9e6d38bf926332698c3f7f763135193452d035f Author: Nicolas Dufresne <nicolas.dufresne@collabora.com> Date: Wed Jul 22 16:19:48 2015 -0400 basetextoverlay: Improve further the negotiation function * Only send the caps event once if the query had support for the overlay composition meta. * Only do the allocation query if it is supported through caps. * Send overlay_caps before doing allocation query rather then normal caps https://bugzilla.gnome.org/show_bug.cgi?id=751157