GNOME Bugzilla – Bug 757933
When a pipeline is in GST_STATE_READY, "sync-message::need-context" is not emitted for GstGL.
Last modified: 2018-11-03 11:43:40 UTC
This issue is found while testing yoon's experimental webkit branch implementing GstGL support along with threaded compositor in webkitgtk+. https://github.com/ryumiel/webkit-experimental/tree/threaded-compositor-wip It works fine in overall, but there is some issues related with gstreamer. How to reproduce. 1. Play a video and watch it completely. 2. Replay it in 60 secs. When replaying a video, an unknown xwindow shows up of which name is "OpenGL renderer". I have investigated it for a while, I found it was created by gstreamer if no gl context is passed from the outside to the gstreamer at beginning of the gst pipeline. Of course webkit create a GstGL context and passes it to gst when listening the "sync-message::need-context" signal from gstreamer. The problem is that the singal is only emitted when the pipeline is in GST_STATE_NULL, but not in GST_STATE_READY. I don't know why. WebKit changes the pipeline state to GST_STATE_READY when receiving GST_MESSAGE_EOS, and then changes it again to GST_STATE_NULL after 60 secs. As I heard, this is due to better performance for replaying video. Anyway if replaying video is triggered in the 60secs, the need-context signal is not emitted so webkit can not create and pass a GstGL context for the new gst pipeline and then it eventually leads gstreamer to create its own xwindow along with a gl context. Replaying *after* 60secs is no problem. So I think the best fix for this issue is that gstreamer emits "sync-message::need-context" even in GST_STATE_READY so that it gives an application a chance to create a gstgl context. Any idea? I'trying to make a test case for you though, I'm not a gst expert. So don't believe me. :P (Of course, I'll provide you one if it is ready.)
The need-context message is not sent because glimagesink doesn't need the GL context when only going back to READY. It keeps the application provided GL context until the state drops to NULL. Currently the other requirement [0] for the window to never appear is that you must have a connection to the "client-draw" signal on glimagesink (which won't disappear on state changes). If webkit changes the GL context it needs to set on the gstreamer pipeline, it needs to 1. call gst_element_set_context explicitly or 2. Drop down to NULL. [0] - http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/ext/gl/gstglimagesink.c?h=1.6#n1964
However he says that glimagesink is creating a *new* window (and context?) when going from READY to PAUSED/PLAYING again. That seems weird. Apart from that, what you (Matthew) describe is how it should behave and how WebKit should handle this.
(In reply to Matthew Waters from comment #1) > The need-context message is not sent because glimagesink doesn't need the GL > context when only going back to READY. oh.. there is a miscommunication. I am not talking that the message does not happen when PLAYING -> READY. When replaying i,e READY -> PLAYING, the message is expected to happen if gst does not have a gl context. > It keeps the application provided GL context until the state drops to NULL. O.K. I think this is a root cause. You mean the given gl context should be alive in READY state unless it goes back to null state, right? As my investigation, when webkit received the GST_MESSAGE_EOS and the state changed from PLAYING to READY, the glimagesink already lost the provided gl context. so when replaying(READY -> PLAYING), the sink had null for gl context so it created its own one. Let me look into this, who destroys the gl context even in READY and actually when it happens. > Currently the other requirement [0] for the window to never appear is that you must have a connection to the "client-draw" signal on glimagesink (which won't disappear on state changes). Yes, the connection is established here. https://github.com/ryumiel/webkit-experimental/blob/threaded-compositor-wip/Source/WebCore/platform/graphics/gstreamer/MediaPlayerPrivateGStreamerBase.cpp#L701 > > If webkit changes the GL context it needs to set on the gstreamer pipeline, > it needs to 1. call gst_element_set_context explicitly or 2. Drop down to > NULL. Good to know. However, I don't think webkit changes the gl context for video when replaying. Thanks.
I looked into this in detail for a while. In a nutshell, gstreamer changes READY to NULL and destory the provided other_context while removing gst pads and elements, even though webkit set the pipeline GST_STATE_READY. sequence flow is like that. webkit receives GST_MESSAGE_EOS -> webkit sets GST_STATE_READY to the pipeline -> gst set READY state successfully but eventually set the statue NULL while processing -> webkit's state is still READY though, gst's state is already NULL -> webkit set NULL to the pipeline after 60secs. I dumped a backtrace when gstglimage destroyed the given other context and highlighted important functions there. [1] Also, I leave GST_DEBUG logs as well. [2] In my humble opinion, this is much related with unlinking pads and destroying them when setting the READY state. [1] http://pastebin.com/BXkaYCuy [2] http://pastebin.com/gHSSd2QD
Yes, setting playsink to READY apparently changes the state of all sinks inside it to NULL. Not sure that makes sense.
(In reply to Sebastian Dröge (slomo) from comment #5) > Yes, setting playsink to READY apparently changes the state of all sinks > inside it to NULL. Not sure that makes sense. Hrm.. So what would be the best fix for this issue? It sounds apparently odd that setting READY state causes setting NULL state? Is emitting a 'sync-message::need-context' signal a much bad idea when changing the state from READY to PLAYING and having no gl context(other_context in terms of gstreamer)?
We would have to do some archaeology to understand why it does that, it seems weird to me but it was probably one of my changes a long time ago that added this
What should be done with this? Should this be retitled to a playbin/playsink bug?
*** Bug 793224 has been marked as a duplicate of this bug. ***
-- GitLab Migration Automatic Message -- This bug has been migrated to freedesktop.org's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/242.