GNOME Bugzilla – Bug 753824
assrender: render overlay using the video output window size
Last modified: 2018-05-04 11:25:13 UTC
Here is a series of patches to render the overlay to at the resolution of the video output window, like it was done recently in basetextoverlay. The output looks much better when the video is scaled up, and the performance is better when the video is scaled down.
Created attachment 309581 [details] [review] handle reconfigure event on source pad
Created attachment 309582 [details] [review] render the subtitles with the proper aspect
Created attachment 309583 [details] [review] use the window size from downstream
Review of attachment 309583 [details] [review]: ::: ext/assrender/gstassrender.c @@ +792,3 @@ + guint frame_height = render->window_height; + + if (video_aspect >= window_aspect) Why? Add a comment at least :)
You're right, I should have tested more before posting the patches. I've also found cases where negotiation would fail, so I've copied the negotation function from basetextoverlay, which was improved a lot recently. Unfortunately I cannot seem to be able to use the gl elements, so I cannot test against them. For some reason they use mesa drivers instead of my nvidia drivers. I've tested against a custom sink though, that does handle overlay composition.
Created attachment 309613 [details] [review] render the subtitles with the proper aspect
Created attachment 309614 [details] [review] improve negotiation
Created attachment 309616 [details] [review] use the window size from downstream
Did some testing here, there seems to be negotiation issues, could not find why yes. Then I managed to force it upload for now, so I could test the rendering. It seems that the render layout (and wrapping) changes depending on the display size. This should never ever happen. Short story, this patchset need serious testing.
As discussed on IRC, the negotiation issue is a bug in the gl elements which also affects textoverlay (Bug #753850). I disabled hinting in libass and I get better results at small sizes. However, I don't believe it's possible to get pixel perfect rendering with the current overlay API, because of rounding errors. This is because the overlay rectangle render geometry is converted to video coordinates. Then the sink will convert the rectangle render geometry back to window coordinates, resulting in a loss of precision. This wouldn't matter if the overlay rectangle had a constant size, however assrender tries to minimize the composition rectangle size, by computing the bounding box of the rendered image in the video. Since the bounding box can change between each frame, this can lead in slight shifts between frames. Two solutions: - add an api to set floating point render size on the overlay rectangles - just allocate overlay rectangles of the full frame size, but it wastes a lot of memory and slows down rendering.
(In reply to Arnaud Vrac from comment #10) > Two solutions: > - add an api to set floating point render size on the overlay rectangles I think this sounds like the best solution, and is consistent with e.g. the navigation events.
Any idea how I can name those functions ? ;)
Created attachment 309812 [details] [review] video: store overlay composition rectangle render geometry as floating point I haven't added a gst_video_overlay_rectangle_new_raw version that uses floating point coordinates.
Created attachment 312295 [details] [review] assrender: render the subtitles with the proper aspect
Created attachment 312297 [details] [review] assrender: improve overlay composition meta negotiation
Created attachment 312298 [details] [review] assrender: use the window size from downstream
I've rebased the patches against master (1.7). Please ignore the gst-video patch for now, it's only useful when an element wants to transform the overlay composition before the sink.
Comment on attachment 312298 [details] [review] assrender: use the window size from downstream commit ecdc7bb37d896e9bd2da2593157b460bca0d42e0 Author: Arnaud Vrac <avrac@freebox.fr> Date: Wed Aug 19 17:18:31 2015 +0200 assrender: Use the window size from downstream This allows rendering the overlay at a resolution matching the video resolution. https://bugzilla.gnome.org/show_bug.cgi?id=753824 commit f021d32beffc74026cbae891c234dd105c1fcf65 Author: Arnaud Vrac <avrac@freebox.fr> Date: Thu Sep 4 18:16:00 2014 +0200 assrender: Improve negotiation This is mostly a copy/paste of the negotiation function in basetextoverlay, which was improved recently to handle many more cases. This will allow us to negotiate a window size with downstream. https://bugzilla.gnome.org/show_bug.cgi?id=753824 commit 31c69023738b13f2809a292dba04ddf852d474b5 Author: Arnaud Vrac <avrac@freebox.fr> Date: Wed Aug 19 13:44:35 2015 +0200 assrender: render the subtitles with the proper aspect This bumps the libass version requirement to 0.10.2. https://bugzilla.gnome.org/show_bug.cgi?id=753824
Can you move the rectangle patch into a new bug? Thanks :)
*** Bug 604138 has been marked as a duplicate of this bug. ***