After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 754820 - Using multiple instances of vaapisink in one application cause problems with rotation (maybe more?)
Using multiple instances of vaapisink in one application cause problems with ...
Status: RESOLVED FIXED
Product: GStreamer
Classification: Platform
Component: gstreamer-vaapi
git master
Other Linux
: High major
: git master
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on: 747946 757376 757598 766704
Blocks: 758397
 
 
Reported: 2015-09-10 09:23 UTC by w.vanhauwaert
Modified: 2017-07-26 13:39 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Screenshot of issue when running given pipeline (432.32 KB, image/png)
2015-09-10 09:47 UTC, w.vanhauwaert
Details

Description w.vanhauwaert 2015-09-10 09:23:20 UTC
I have made an application for a E3800 where I define 4 different pipelines. I want one of these video's to be rotated by the rotate-parameter of the vaapisink. But when I do so, all of the video's get rotated for some reason... 
However, when I start two different pipelines from the commandline (so 2 different processes), the rotation of one of the two works perfectly. So somewhere something is shared between these multiple pipelines in one application...
Comment 1 sreerenj 2015-09-10 09:29:51 UTC
Please provide something to reproduce the issue :)
Comment 2 w.vanhauwaert 2015-09-10 09:38:40 UTC
(In reply to sreerenj from comment #1)
> Please provide something to reproduce the issue :)

Okelidokeli, the following pipeline shows the issue already. Where you should expect only one output rotated, they both get rotated (but the window itself stays normal):

gst-launch-1.0 videotestsrc ! tee name=t t. ! queue ! vaapisink rotation=90 t. ! queue ! vaapisink
Comment 3 w.vanhauwaert 2015-09-10 09:47:06 UTC
Created attachment 311047 [details]
Screenshot of issue when running given pipeline

gst-launch-1.0 videotestsrc ! tee name=t t. ! queue ! vaapisink rotation=90 t. ! queue ! vaapisink
Comment 4 sreerenj 2015-09-10 10:11:13 UTC
(In reply to w.vanhauwaert from comment #3)
> Created attachment 311047 [details]
> Screenshot of issue when running given pipeline
> 
> gst-launch-1.0 videotestsrc ! tee name=t t. ! queue ! vaapisink rotation=90
> t. ! queue ! vaapisink

Seems to be a bug!! Thanks for reporting..
I guess, it could be due to VADisplay sharing... Elements sharing contexts and both vaapisink getting same context...
Comment 5 sreerenj 2015-09-10 10:12:17 UTC
Vitcor, please have a look before the next bug-fix release if possible...
Comment 6 Víctor Manuel Jáquez Leal 2015-09-10 11:15:32 UTC
As I suspected, the issue is deeper than just the context sharing.

I disabled the context sharing and the problem remains. I printe the VADisplay when setting a VADisplayAttribType:

setting attrib 2 value 0 to display 0x8ea900
setting attrib 3 value 50 to display 0x8ea900
setting attrib 0 value 0 to display 0x8ea900
setting attrib 1 value 50 to display 0x8ea900
setting attrib 2 value 0 to display 0x8ea900
setting attrib 3 value 50 to display 0x8ea900
setting attrib 0 value 0 to display 0x8ea900
setting attrib 1 value 50 to display 0x8ea900
setting attrib 6 value 1 to display 0x8ea900

That display pointer is the VADisplay when an attribute is set. It is the same, even if the sink creates its own GstVaapiDisplay instance.

So, if we set an attribute to the VADisplay, it is also applied to any other vaapisink in the same pipeline.

All this is because we manage the VADisplay creation through GstVaapiDisplayCache, which creates only one VADisplay per process.

In my opinion this bug might be a duplicate of bug 747946
Comment 7 sreerenj 2015-09-10 11:19:40 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #6)
> As I suspected, the issue is deeper than just the context sharing.
> 
> I disabled the context sharing and the problem remains. I printe the
> VADisplay when setting a VADisplayAttribType:
> 
> setting attrib 2 value 0 to display 0x8ea900
> setting attrib 3 value 50 to display 0x8ea900
> setting attrib 0 value 0 to display 0x8ea900
> setting attrib 1 value 50 to display 0x8ea900
> setting attrib 2 value 0 to display 0x8ea900
> setting attrib 3 value 50 to display 0x8ea900
> setting attrib 0 value 0 to display 0x8ea900
> setting attrib 1 value 50 to display 0x8ea900
> setting attrib 6 value 1 to display 0x8ea900
> 
> That display pointer is the VADisplay when an attribute is set. It is the
> same, even if the sink creates its own GstVaapiDisplay instance.
> 
> So, if we set an attribute to the VADisplay, it is also applied to any other
> vaapisink in the same pipeline.
> 
> All this is because we manage the VADisplay creation through
> GstVaapiDisplayCache, which creates only one VADisplay per process.
> 
> In my opinion this bug might be a duplicate of bug 747946

Aha, okay,, then may be after 0.6.1 :(
Comment 8 w.vanhauwaert 2015-09-10 11:23:33 UTC
(In reply to Víctor Manuel Jáquez Leal from comment #6)
> 
> So, if we set an attribute to the VADisplay, it is also applied to any other
> vaapisink in the same pipeline.
> 

Just as a small remark, but in my real application, I use different pipelines (the gst-launch command was just a quick way to demo the bug), but the behaviour is still the same as explained...
Comment 9 Víctor Manuel Jáquez Leal 2015-10-22 11:13:50 UTC
Just a heads-up about this:

In order to avoid the display caching mechanism, this simple patch does the job:

diff --git a/gst-libs/gst/vaapi/gstvaapidisplay.c b/gst-libs/gst/vaapi/gstvaapidisplay.c
index 44b6f00..99805b3 100644
--- a/gst-libs/gst/vaapi/gstvaapidisplay.c
+++ b/gst-libs/gst/vaapi/gstvaapidisplay.c
@@ -940,11 +940,6 @@ gst_vaapi_display_create_unlocked (GstVaapiDisplay * display,
     GST_DEBUG ("VA-API version %d.%d", major_version, minor_version);
   }
 
-  if (!cached_info) {
-    if (!gst_vaapi_display_cache_add (priv->cache, &info))
-      return FALSE;
-  }
-
   g_free (priv->display_name);
   priv->display_name = g_strdup (info.display_name);
   return TRUE;

But it arises several issues:

1\ GST_GL_API=opengl GST_GL_PLATFORM=glx jh gst-play-1.0 "/home/vjaquez/Downloads/L'écume des jours (2013)/L.ecume.des.jours.2013.BDRip.x264.AAC-MED.mp4" --videosink=glimagesink 
WORKS OK


2\ GST_GL_API=opengl GST_GL_PLATFORM=glx jh gst-play-1.0 "/home/vjaquez/Downloads/L'écume des jours (2013)/L.ecume.des.jours.2013.BDRip.x264.AAC-MED.mp4" --videosink=vaapsink 
DOES NOT WORK OK

Analyzing this, an element (vaapipostproc) already receives the context, but it queries for it again because it doesn't check that it already has it. We need to fix that.

3\ gst-launch-1.0 videotestsrc ! tee name=t ! queue ! vaapisink rotation=90 t. ! queue ! vaapisink 
WORKS OK

4\ gst-launch-1.0 filesrc location=~/patterns/big_buck_bunny_1080p_h264.mov ! qtdemux ! vaapidecode ! vaapisink 
WORKS OK but

This pipeline works, but vaapidecode and vaapisink creates their own display, and ignore each other contexts.

5\ gst-launch-1.0 -e v4l2src device=/dev/video0 num-buffers=1800 ! video/x-raw,format=I420,width=640,height=480 ! vaapipostproc ! queue ! vaapiencode_h264 ! qtmux ! filesink location=test2.mp4 
WORKS OK

Interestingly enough, in this pipeline the context is shared correctly among the pipeline.
Comment 10 Víctor Manuel Jáquez Leal 2015-10-27 10:10:08 UTC
Experimenting a bit, in order to remove the display cache (or perhaps only its usage), we need to fix the gtreamer context sharing mechanism.

Right now we are creating the VADisplay *before* the bus is attached to the elements, so there's no way to share the gstreamer context, which shares the VADisplay among the pipeline.

For example this commit:

commit 5e5d62cac79754ba60057fc2516135aad8d7de35
Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Date:   Fri Mar 7 17:40:34 2014 +0100

    vaapisink: fix initialization with "drm" display type.
    
    Force early initializatin of the GstVaapiDisplay so that to make sure
    that the sink element display object is presented first to upstream
    elements, as it will be correctly featuring the requested display type
    by the user.
    
    Otherwise, we might end up in situations where a VA/X11 display is
    initialized in vaapidecode, then we try VA/DRM display in vaapisink
    (as requested by the "display" property), but this would cause a failure
    because we cannot acquire a DRM display that was previously acquired
    through another backend (e.g. VA/X11).

Creates a VADisplay in the vaapisink before it has attached a bus. So it never gets shared among the pipeline. But the display cache have hidden the problem.

Also, in the case of the decoder, the caps query might happen before having a bus, so creating the VADisplay when generating the allowed caps leads to break the context sharing.

So, we need to delay, as much as possible, the VADisplay creation, at least until the bus is attached to the element. Though, this could lead to issues, such as that one commit 5e5d62 related to DRM.
Comment 11 Víctor Manuel Jáquez Leal 2015-11-09 17:23:40 UTC
Finally we have push the patches of bug 757598. Now the context sharing is in better shape than before.

Nonetheless, removing the cache would not fix this use case, since the same VADisplay will be shared among the pipeline and we will see the same output.

This blog post explains how to pass a user-provided GL context to an element via the sync bus: http://ystreet00.blogspot.com.au/2015_09_01_archive.html

My current idea is to provide a similar machinery to gstreamer-vaapi, so the user could set a custom GstVaapiDisplay to a specific element (in this case, a second vaapisink).

But this strategy implies to expose to the users the gstvaapi library, which currently is only consumed internally by the elements.

What do you think?
Comment 12 sreerenj 2015-11-20 14:37:05 UTC
This issue seems to be requires fixes which goes more deeper into design. Unfortunately it is risky to do this for immediate 0.7 release so I am postponing it for 0.8, sorry..

But we must fix this for 0.8 !
Comment 13 Víctor Manuel Jáquez Leal 2017-07-26 13:39:19 UTC
Now it is possible to do this, but you need to create VADisplay in your app. At level of gst-launch is not possible.

Look the test: https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/tree/tests/elements/test-vaapicontext.c

I'm closing this issued then.