GNOME Bugzilla – Bug 705579
Playbin prevents plugins requesting a GstContext to work properly
Last modified: 2015-10-30 16:57:05 UTC
When using Gstreamer-vaapi, the vaapidecode plugin instances usually need informations about the display (is it wayland, X11 or even direct drm access?). This works well if you're building a custom pipeline by hand, because all the elements are linked one to another, down to the video sink (which is the element most likely to provide display informations). Unfortunately when using playbin/decodebin, the pipeline is built incrementally and even though a video sink has been provided to playbin, the video sink element is not attached to the GstBus of the pipeline when vaapidecode request the display informations to allocate its buffers. Would it be possible for the playbin/decodebin plugins to forward the GST_QUERY_CONTEXT message to the video sink or to connect the video sink to GstBus earlier in the process or creating the pipeline dynamically?
The alternative currently proposed in the Gstreamer documentation is for each application using a playbin/decodebin pipeline, to set a sync handler on the pipeline to respond to display information request. This might be a short term solution but most of the Gtk+/Clutter applications tend to not really know whether they're running under wayland or X11 these days.
After some discussions with graphic people we came up with the idea of enforcing vaapi plugins to use the drm backend and use a Mesa extension to create appropriate display surfaces for these drm buffers. But someone pointed that would prevent plugins from nvidia (for example) to work.
playbin already forwards the context queries and messages and everything to the video sink. An application would need to implement handling of the context messages though, like gst-launch does. It doesn't work for you? What happens? Also see bug #701996
Almost all application using ClutterGst (like Totem/Cheese/etc...) do not use the ClutterGstPlayer object, but instead create their own pipeline using ClutterSink. The problem is, none of these applications actually know what display they're running on (X or Wayland) and therefore can't set the appropriate display information when vaapidecode requests it. So the symptom is that if you have vaapi plugins installed and playbin tries to use these plugins, you will get a blank screen because vaapidecode can't allocate appropriate buffers for the display. It seems that the video sink could be able to answer all video context requests from vaapidecode. But last time I tried (on 1.1.2), when vaapidecode fires a GST_QUERY_CONTEXT message in a playbin pipeline, the ClutterSink isn't connected to respond.
That falls in the should work with playbin, I'll have a look.
Nicolas?
Right so I currently have a couple of cases where the context machinery does not work inside playbin/playsink as it stands currently. Basically it seems to boil down to the propagation of the GstContext from the sink in NULL->READY will never reach playsink/playbin as playsink tries to set the element to ready before adding it to the bin. This also highlights a problem with GstContext and dynamic pipelines. Adding an element that has already propagated a GstContext to a bin will fail to re-propagate the GstContext through the pipeline.
(In reply to Matthew Waters from comment #7) > Right so I currently have a couple of cases where the context machinery does > not work inside playbin/playsink as it stands currently. > > Basically it seems to boil down to the propagation of the GstContext from > the sink in NULL->READY will never reach playsink/playbin as playsink tries > to set the element to ready before adding it to the bin. This also > highlights a problem with GstContext and dynamic pipelines. Adding an > element that has already propagated a GstContext to a bin will fail to > re-propagate the GstContext through the pipeline. This probably means that gst_bin_add_func() should call gst_element_get_context(), and propagate whatever context comes back?
Wait, there is no get_context(), that was only in experiments before all this was merged. Maybe we indeed need it for cases like this :)
Created attachment 312019 [details] [review] bin: implement context propagation when adding elements Fixes the sharing of contexts with the attempted patch in https://bugzilla.gnome.org/show_bug.cgi?id=754786
Review of attachment 312019 [details] [review]: ::: gst/gstelement.c @@ +3073,3 @@ + GST_OBJECT_LOCK (element); + + context_type = gst_context_get_context_type (context); I think this code should all be in the default implementation of set_context(). That would then allow subclasses to actually override this new behaviour. @@ +3106,3 @@ + * MT safe. + * + * Returns: (element-type Gst.Context) (transfer full): List of #GstContext Sprinkle some "Since: 1.8" around here
Created attachment 312040 [details] [review] bin: implement context propagation when adding elements Address previous comments. This means that any element implementing GstElement::set_context should now chain up to the parent class.
(In reply to Matthew Waters from comment #12) > > This means that any element implementing GstElement::set_context should now > chain up to the parent class. Are they? If not, moar patches please :)
commit d5ded1588920c4471eefe055d09095d9e5e989b5 Author: Matthew Waters <matthew@centricular.com> Date: Thu Sep 24 00:04:48 2015 +1000 bin: implement context propagation when adding elements When adding an element to a bin we need to propagate the GstContext's to/from the element. This moves the GstContext list from GstBin to GstElement and adds convenience functions to get the currently set list of GstContext's. This does not deal with the collection of GstContext's propagated using GST_CONTEXT_QUERY. Element subclasses are advised to call gst_element_set_context if they need to propagate GstContext's received from the context query. https://bugzilla.gnome.org/show_bug.cgi?id=705579
The decoder wanting to performing a context query that always fails is due to https://bugzilla.gnome.org/show_bug.cgi?id=742924 Closing.