GNOME Bugzilla – Bug 692012
assrender: fix, re-enable and add overlay support
Last modified: 2013-01-22 00:31:49 UTC
Here are some patches for assrender to make it work better in 1.0.
Created attachment 233755 [details] [review] Proxy allocations on video pad
Created attachment 233756 [details] [review] Refactor to avoid code duplication
Created attachment 233757 [details] [review] Fix some subtitles not being shown Some subtitles were not being shown anymore after 2213a44214763f
Created attachment 233758 [details] [review] Add support for overlay composition An intermediate ARGB buffer is allocated to composite the ASS images, which is attached to an overlay composition. libass can detect when the subtitles change so we only render the overlay when needed, not for every frame. The patch could be extended to use overlay composition and blend it on the video frames. Rendering might be faster because the ASS images are only composited once on the intermediate buffer. This would also make the plugin support all 8-bit pixel formats.
Created attachment 233759 [details] [review] Up rank to primary again now that the unit tests pass
Created attachment 233760 [details] [review] Up rank to primary again now that the unit tests pass
Created attachment 233762 [details] [review] Add support for overlay composition A gst_query_unref was missing in the previous patch
Created attachment 233765 [details] [review] Remove format specific blit functions and use overlay composition instead
It seems there can be a deadlock after a seek, assrender will wait for a textbuffer forever when used in a multiqueue. I will try to fix it and post the full patch set to a git repo.
The deadlock was due to the missing wait-text property, which I added. I have pushed my patches here: https://github.com/rawoul/gst-plugins-bad/commits/master. Please ignore the last patch which is for high quality overlay rendering but is not usable with gstreamer master.
Thanks a lot for the patches! commit 8d837cbac41b121760207e784ba7b54f1ef84f32 Author: Arnaud Vrac <avrac@freebox.fr> Date: Thu Jan 17 19:02:48 2013 +0100 assrender: reenable unit tests and up rank to primary again commit 143c0cbd45008c208fe8f2201429adb54c101ad2 Author: Arnaud Vrac <avrac@freebox.fr> Date: Fri Jan 18 16:14:03 2013 +0100 assrender: use overlay composition to blit subtitles on video frames Re-uses existing blitting code and also means we can support a lot more video formats out of the box, hence avoid unnecessary pixel format or colour space conversions. https://bugzilla.gnome.org/show_bug.cgi?id=692012 commit 5f913665537e40758540eb8fa9858adace3650bd Author: Arnaud Vrac <avrac@freebox.fr> Date: Thu Jan 17 18:48:12 2013 +0100 assrender: render overlay composition if supported downstream This allows rendering ASS subtitles on top of video when using hardware-accelerated video decoders based on e.g. VA-API or VDPAU. https://bugzilla.gnome.org/show_bug.cgi?id=678389 https://bugzilla.gnome.org/show_bug.cgi?id=692012 commit 30222d608044bcd29537218309882f43f3c35963 Author: Arnaud Vrac <avrac@freebox.fr> Date: Mon Jan 21 10:35:49 2013 +0100 assrender: add wait-text property This is the same property as in the pango element, to allow waiting for text on the text pad before rendering the video. https://bugzilla.gnome.org/show_bug.cgi?id=692012 commit c78714ced3c46f4c0aa66dfdcc909baee2783fa4 Author: Arnaud Vrac <avrac@freebox.fr> Date: Fri Jan 18 15:25:48 2013 +0100 assrender: fix some subtitles being ignored https://bugzilla.gnome.org/show_bug.cgi?id=692012 commit c16d41d7091f808d5c3b06b79160dbe187594ead Author: Arnaud Vrac <avrac@freebox.fr> Date: Thu Jan 17 18:33:11 2013 +0100 assrender: refactor code to avoid duplication https://bugzilla.gnome.org/show_bug.cgi?id=692012 commit 61976b7aa71ab55f0028a1d2fea888a072e945a0 Author: Arnaud Vrac <avrac@freebox.fr> Date: Thu Jan 17 18:26:08 2013 +0100 assrender: proxy allocations on video pad Forward allocation queries on video sink pad instead of discarding them. This allows the upstream elements to use the allocation parameters or pools from the downstream elements like videosink. https://bugzilla.gnome.org/show_bug.cgi?id=692012