After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 751157 - basetextoverlay: Pass down GstVideoOverlayComposition buffers from upstream, receive window size event
basetextoverlay: Pass down GstVideoOverlayComposition buffers from upstream, ...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gst-plugins-base
git master
Other Linux
: Normal enhancement
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Blocks:
 
 
Reported: 2015-06-18 12:54 UTC by Lubosz Sarnecki
Modified: 2015-07-22 21:12 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Pass down overlay composition buffers from upstream (3.46 KB, patch)
2015-06-18 12:55 UTC, Lubosz Sarnecki
none Details | Review
adjust render size from downstream window event (6.15 KB, patch)
2015-06-18 12:57 UTC, Lubosz Sarnecki
none Details | Review
Log if overlay meta nego was succesfull (970 bytes, patch)
2015-06-18 12:57 UTC, Lubosz Sarnecki
none Details | Review
basetestoverlay: Pass down meta buffers from upstream that supports GstVideoOverlayComposition (3.46 KB, patch)
2015-07-01 13:53 UTC, Lubosz Sarnecki
committed Details | Review
basetextoverlay: Receive window size event and adjust rendering (6.88 KB, patch)
2015-07-01 13:53 UTC, Lubosz Sarnecki
committed Details | Review
basetextoverlay: Log GstVideoOverlayComposition negotiation (970 bytes, patch)
2015-07-01 13:53 UTC, Lubosz Sarnecki
committed Details | Review
basetextoverlay: Send caps event to receive current allocation query (1.69 KB, patch)
2015-07-01 13:54 UTC, Lubosz Sarnecki
none Details | Review
basetextoverlay: Send caps event to receive current allocation query (1.69 KB, patch)
2015-07-21 08:13 UTC, Lubosz Sarnecki
rejected Details | Review
basetextoverlay: Ensure meta coordinate are in stream scale (7.00 KB, patch)
2015-07-21 22:45 UTC, Nicolas Dufresne (ndufresne)
none Details | Review
basetextoverlay: Reorder and cleanup class attribute (5.21 KB, patch)
2015-07-21 22:45 UTC, Nicolas Dufresne (ndufresne)
none Details | Review
basetextoverlay: Ensure meta coordinate are in stream scale (7.64 KB, patch)
2015-07-22 04:46 UTC, Nicolas Dufresne (ndufresne)
none Details | Review
basetextoverlay: Improve debug trace (3.16 KB, patch)
2015-07-22 04:47 UTC, Nicolas Dufresne (ndufresne)
none Details | Review

Description Lubosz Sarnecki 2015-06-18 12:54:10 UTC
These patches are part of implementing the GstVideoOverlayComposition in the glimagesink.

https://bugzilla.gnome.org/show_bug.cgi?id=745107
Comment 1 Lubosz Sarnecki 2015-06-18 12:55:56 UTC
Created attachment 305559 [details] [review]
Pass down overlay composition buffers from upstream

This patch allows chaining multiple textoverlay elements without losing the OverlayComposition meta data.

It makes pipelines like this possible:

    gst-launch-1.0 videotestsrc ! \
      textoverlay text="text1" valignment="top" font-desc="sans bold 30" ! \
      textoverlay text="text2" halignment="right" font-desc="sans bold 30" ! \
      textoverlay text="text3" halignment="left" font-desc="sans bold 30" ! \
      glimagesink
Comment 2 Lubosz Sarnecki 2015-06-18 12:57:27 UTC
Created attachment 305561 [details] [review]
adjust render size from downstream window event

This patch provides a possibility to render text for the correct screen size of the sink. The sink needs to send an upstream window size event.

xvimagesink

http://i.imgur.com/zjEDnGI.png

glimagesink

http://i.imgur.com/2uO1l3z.png
Comment 3 Lubosz Sarnecki 2015-06-18 12:57:53 UTC
Created attachment 305562 [details] [review]
Log if overlay meta nego was succesfull

Log if overlay meta nego was succesfull
Comment 4 Lubosz Sarnecki 2015-07-01 13:53:01 UTC
Created attachment 306504 [details] [review]
basetestoverlay: Pass down meta buffers from upstream that supports GstVideoOverlayComposition
Comment 5 Lubosz Sarnecki 2015-07-01 13:53:29 UTC
Created attachment 306505 [details] [review]
basetextoverlay: Receive window size event and adjust rendering
Comment 6 Lubosz Sarnecki 2015-07-01 13:53:50 UTC
Created attachment 306506 [details] [review]
basetextoverlay: Log GstVideoOverlayComposition negotiation
Comment 7 Lubosz Sarnecki 2015-07-01 13:54:23 UTC
Created attachment 306507 [details] [review]
basetextoverlay: Send caps event to receive current allocation query

To make the textoverlay receive a current window resolution from the sink,
a send caps event needs to be sent, to not receive a cached allocation query.
Comment 8 Nicolas Dufresne (ndufresne) 2015-07-20 19:30:58 UTC
That last patch is outdated, it relied on a bug in fact. The code now is skipping the allocation query when upstream or configured caps have the overlay feature. Obviously this breaks the window sizing stuff. I have a patch of my own that make it do an allocation query all the time, I'll share later.

Now I can't merge this, because it broke the normal case, the case where we render in video size (when the sink does not report the window size). I need to look into this.
Comment 9 Nicolas Dufresne (ndufresne) 2015-07-20 20:27:54 UTC
Attachment 306504 [details] pushed as f128666 - basetestoverlay: Pass down meta buffers from upstream that supports GstVideoOverlayComposition
Attachment 306505 [details] pushed as 91a615f - basetextoverlay: Receive window size event and adjust rendering
Attachment 306506 [details] pushed as d1808a5 - basetextoverlay: Log GstVideoOverlayComposition negotiation
Comment 10 Nicolas Dufresne (ndufresne) 2015-07-20 20:33:44 UTC
Also added few more patches to fix the remaining issues I saw.

a2e4ccc38bd108de9bba8b366acaab7e218e3a1b basetextoverlay: Fix upstream composition handling
d10959fd3613b1f08d975a948fd9f9dfef5def66 basetextoverlay: Clear reconfigure flags before negotation
2308014963b3d019a51fe80c2e539e3792c283f8 basetestoverlay: Always query window dimension
Comment 11 Lubosz Sarnecki 2015-07-21 08:13:06 UTC
Created attachment 307801 [details] [review]
basetextoverlay: Send caps event to receive current allocation query

Sorry, I forgot to propose this patch! Without it the negotation lags behind.

To make the textoverlay receive a current window resolution from the sink,
a send caps event needs to be sent, to not receive a cached allocation query.
Comment 12 Nicolas Dufresne (ndufresne) 2015-07-21 13:25:32 UTC
Review of attachment 307801 [details] [review]:

I implemented my own.
Comment 13 Nicolas Dufresne (ndufresne) 2015-07-21 13:26:14 UTC
commit a64a343077851bc2bf6ae427ed8fbc4ad861c944
Author: Nicolas Dufresne <nicolas.dufresne@collabora.co.uk>
Date:   Mon Jul 20 15:11:06 2015 -0400

    basetextoverlay: Send caps before doing allocation query
    
    This is currently a limitation of BaseTransform base class. Which means
    pretty much every filters out there.
    
    http://bugzilla.gnome.org/show_bug.cgi?id=751157
Comment 14 Nicolas Dufresne (ndufresne) 2015-07-21 13:27:13 UTC
And this one to, for the cross-reference. If we don't always do the allocation query, then we endup without outdate window dimension.

commit 2308014963b3d019a51fe80c2e539e3792c283f8
Author: Nicolas Dufresne <nicolas.dufresne@collabora.co.uk>
Date:   Mon Jul 20 15:55:07 2015 -0400

    basetestoverlay: Always query window dimension
    
    Remove the optimization to skip allocation query so we can
    always have the latest window size information. Also, correctly
    deal with the case where there is no window size information.
    
    http://bugzilla.gnome.org/show_bug.cgi?id=751157
Comment 15 Nicolas Dufresne (ndufresne) 2015-07-21 22:44:39 UTC
I lately notice that VideoCompositionMeta coordinate are relative to the window "sometimes" now. They should always be relative to the stream.
Comment 16 Nicolas Dufresne (ndufresne) 2015-07-21 22:45:00 UTC
Created attachment 307870 [details] [review]
basetextoverlay: Ensure meta coordinate are in stream scale

The GstVideoOverlayComposition meta coordinates should always be
in stream scale, regardless of the window size downstream. This
way the sink can always scale the composition if the window size
have changed after a buffer (with his meta) was rendered before.
Comment 17 Nicolas Dufresne (ndufresne) 2015-07-21 22:45:33 UTC
Created attachment 307871 [details] [review]
basetextoverlay: Reorder and cleanup class attribute

Also add a minimum amount of comment so we can understand what
is doing what.
Comment 18 Nicolas Dufresne (ndufresne) 2015-07-22 04:46:58 UTC
Created attachment 307882 [details] [review]
basetextoverlay: Ensure meta coordinate are in stream scale

The GstVideoOverlayComposition meta coordinates should always be
in stream scale, regardless of the window size downstream. This
way the sink can always scale the composition if the window size
have changed after a buffer (with his meta) was rendered before.
Comment 19 Nicolas Dufresne (ndufresne) 2015-07-22 04:47:02 UTC
Created attachment 307883 [details] [review]
basetextoverlay: Improve debug trace
Comment 20 Nicolas Dufresne (ndufresne) 2015-07-22 04:48:53 UTC
In the previews patch, I forgot to scale the rendering. Also added the debug trace I have used to debug this.
Comment 21 Nicolas Dufresne (ndufresne) 2015-07-22 17:25:11 UTC
Leaving open for now as I'll be doing more testing.

Attachment 307871 [details] pushed as db81a73 - basetextoverlay: Reorder and cleanup class attribute
Attachment 307882 [details] pushed as 48f877e - basetextoverlay: Ensure meta coordinate are in stream scale
Comment 22 Nicolas Dufresne (ndufresne) 2015-07-22 21:12:04 UTC
One more iteration, works quite well now (and that reduce the number of spurious negotiation).

commit f9e6d38bf926332698c3f7f763135193452d035f
Author: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Date:   Wed Jul 22 16:19:48 2015 -0400

    basetextoverlay: Improve further the negotiation function
    
    * Only send the caps event once if the query had support for the
      overlay composition meta.
    * Only do the allocation query if it is supported through caps.
    * Send overlay_caps before doing allocation query rather then normal
      caps
    
    https://bugzilla.gnome.org/show_bug.cgi?id=751157