GNOME Bugzilla – Bug 754820
Using multiple instances of vaapisink in one application cause problems with rotation (maybe more?)
Last modified: 2017-07-26 13:39:19 UTC
I have made an application for a E3800 where I define 4 different pipelines. I want one of these video's to be rotated by the rotate-parameter of the vaapisink. But when I do so, all of the video's get rotated for some reason... However, when I start two different pipelines from the commandline (so 2 different processes), the rotation of one of the two works perfectly. So somewhere something is shared between these multiple pipelines in one application...
Please provide something to reproduce the issue :)
(In reply to sreerenj from comment #1) > Please provide something to reproduce the issue :) Okelidokeli, the following pipeline shows the issue already. Where you should expect only one output rotated, they both get rotated (but the window itself stays normal): gst-launch-1.0 videotestsrc ! tee name=t t. ! queue ! vaapisink rotation=90 t. ! queue ! vaapisink
Created attachment 311047 [details] Screenshot of issue when running given pipeline gst-launch-1.0 videotestsrc ! tee name=t t. ! queue ! vaapisink rotation=90 t. ! queue ! vaapisink
(In reply to w.vanhauwaert from comment #3) > Created attachment 311047 [details] > Screenshot of issue when running given pipeline > > gst-launch-1.0 videotestsrc ! tee name=t t. ! queue ! vaapisink rotation=90 > t. ! queue ! vaapisink Seems to be a bug!! Thanks for reporting.. I guess, it could be due to VADisplay sharing... Elements sharing contexts and both vaapisink getting same context...
Vitcor, please have a look before the next bug-fix release if possible...
As I suspected, the issue is deeper than just the context sharing. I disabled the context sharing and the problem remains. I printe the VADisplay when setting a VADisplayAttribType: setting attrib 2 value 0 to display 0x8ea900 setting attrib 3 value 50 to display 0x8ea900 setting attrib 0 value 0 to display 0x8ea900 setting attrib 1 value 50 to display 0x8ea900 setting attrib 2 value 0 to display 0x8ea900 setting attrib 3 value 50 to display 0x8ea900 setting attrib 0 value 0 to display 0x8ea900 setting attrib 1 value 50 to display 0x8ea900 setting attrib 6 value 1 to display 0x8ea900 That display pointer is the VADisplay when an attribute is set. It is the same, even if the sink creates its own GstVaapiDisplay instance. So, if we set an attribute to the VADisplay, it is also applied to any other vaapisink in the same pipeline. All this is because we manage the VADisplay creation through GstVaapiDisplayCache, which creates only one VADisplay per process. In my opinion this bug might be a duplicate of bug 747946
(In reply to Víctor Manuel Jáquez Leal from comment #6) > As I suspected, the issue is deeper than just the context sharing. > > I disabled the context sharing and the problem remains. I printe the > VADisplay when setting a VADisplayAttribType: > > setting attrib 2 value 0 to display 0x8ea900 > setting attrib 3 value 50 to display 0x8ea900 > setting attrib 0 value 0 to display 0x8ea900 > setting attrib 1 value 50 to display 0x8ea900 > setting attrib 2 value 0 to display 0x8ea900 > setting attrib 3 value 50 to display 0x8ea900 > setting attrib 0 value 0 to display 0x8ea900 > setting attrib 1 value 50 to display 0x8ea900 > setting attrib 6 value 1 to display 0x8ea900 > > That display pointer is the VADisplay when an attribute is set. It is the > same, even if the sink creates its own GstVaapiDisplay instance. > > So, if we set an attribute to the VADisplay, it is also applied to any other > vaapisink in the same pipeline. > > All this is because we manage the VADisplay creation through > GstVaapiDisplayCache, which creates only one VADisplay per process. > > In my opinion this bug might be a duplicate of bug 747946 Aha, okay,, then may be after 0.6.1 :(
(In reply to Víctor Manuel Jáquez Leal from comment #6) > > So, if we set an attribute to the VADisplay, it is also applied to any other > vaapisink in the same pipeline. > Just as a small remark, but in my real application, I use different pipelines (the gst-launch command was just a quick way to demo the bug), but the behaviour is still the same as explained...
Just a heads-up about this: In order to avoid the display caching mechanism, this simple patch does the job: diff --git a/gst-libs/gst/vaapi/gstvaapidisplay.c b/gst-libs/gst/vaapi/gstvaapidisplay.c index 44b6f00..99805b3 100644 --- a/gst-libs/gst/vaapi/gstvaapidisplay.c +++ b/gst-libs/gst/vaapi/gstvaapidisplay.c @@ -940,11 +940,6 @@ gst_vaapi_display_create_unlocked (GstVaapiDisplay * display, GST_DEBUG ("VA-API version %d.%d", major_version, minor_version); } - if (!cached_info) { - if (!gst_vaapi_display_cache_add (priv->cache, &info)) - return FALSE; - } - g_free (priv->display_name); priv->display_name = g_strdup (info.display_name); return TRUE; But it arises several issues: 1\ GST_GL_API=opengl GST_GL_PLATFORM=glx jh gst-play-1.0 "/home/vjaquez/Downloads/L'écume des jours (2013)/L.ecume.des.jours.2013.BDRip.x264.AAC-MED.mp4" --videosink=glimagesink WORKS OK 2\ GST_GL_API=opengl GST_GL_PLATFORM=glx jh gst-play-1.0 "/home/vjaquez/Downloads/L'écume des jours (2013)/L.ecume.des.jours.2013.BDRip.x264.AAC-MED.mp4" --videosink=vaapsink DOES NOT WORK OK Analyzing this, an element (vaapipostproc) already receives the context, but it queries for it again because it doesn't check that it already has it. We need to fix that. 3\ gst-launch-1.0 videotestsrc ! tee name=t ! queue ! vaapisink rotation=90 t. ! queue ! vaapisink WORKS OK 4\ gst-launch-1.0 filesrc location=~/patterns/big_buck_bunny_1080p_h264.mov ! qtdemux ! vaapidecode ! vaapisink WORKS OK but This pipeline works, but vaapidecode and vaapisink creates their own display, and ignore each other contexts. 5\ gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=1800 ! video/x-raw,format=I420,width=640,height=480 ! vaapipostproc ! queue ! vaapiencode_h264 ! qtmux ! filesink location=test2.mp4 WORKS OK Interestingly enough, in this pipeline the context is shared correctly among the pipeline.
Experimenting a bit, in order to remove the display cache (or perhaps only its usage), we need to fix the gtreamer context sharing mechanism. Right now we are creating the VADisplay *before* the bus is attached to the elements, so there's no way to share the gstreamer context, which shares the VADisplay among the pipeline. For example this commit: commit 5e5d62cac79754ba60057fc2516135aad8d7de35 Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com> Date: Fri Mar 7 17:40:34 2014 +0100 vaapisink: fix initialization with "drm" display type. Force early initializatin of the GstVaapiDisplay so that to make sure that the sink element display object is presented first to upstream elements, as it will be correctly featuring the requested display type by the user. Otherwise, we might end up in situations where a VA/X11 display is initialized in vaapidecode, then we try VA/DRM display in vaapisink (as requested by the "display" property), but this would cause a failure because we cannot acquire a DRM display that was previously acquired through another backend (e.g. VA/X11). Creates a VADisplay in the vaapisink before it has attached a bus. So it never gets shared among the pipeline. But the display cache have hidden the problem. Also, in the case of the decoder, the caps query might happen before having a bus, so creating the VADisplay when generating the allowed caps leads to break the context sharing. So, we need to delay, as much as possible, the VADisplay creation, at least until the bus is attached to the element. Though, this could lead to issues, such as that one commit 5e5d62 related to DRM.
Finally we have push the patches of bug 757598. Now the context sharing is in better shape than before. Nonetheless, removing the cache would not fix this use case, since the same VADisplay will be shared among the pipeline and we will see the same output. This blog post explains how to pass a user-provided GL context to an element via the sync bus: http://ystreet00.blogspot.com.au/2015_09_01_archive.html My current idea is to provide a similar machinery to gstreamer-vaapi, so the user could set a custom GstVaapiDisplay to a specific element (in this case, a second vaapisink). But this strategy implies to expose to the users the gstvaapi library, which currently is only consumed internally by the elements. What do you think?
This issue seems to be requires fixes which goes more deeper into design. Unfortunately it is risky to do this for immediate 0.7 release so I am postponing it for 0.8, sorry.. But we must fix this for 0.8 !
Now it is possible to do this, but you need to create VADisplay in your app. At level of gst-launch is not possible. Look the test: https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/tests/elements/test-vaapicontext.c I'm closing this issued then.