After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 759869 - glimagesink: navigation messages are not posted on the bus
glimagesink: navigation messages are not posted on the bus
Product: GStreamer
Classification: Platform
Component: gst-plugins-bad
git master
Other Linux
: Normal normal
: 1.7.2
Assigned To: GStreamer Maintainers
GStreamer Maintainers
Depends on:
Reported: 2015-12-25 22:15 UTC by Florent Thiéry
Modified: 2015-12-28 08:06 UTC
See Also:
GNOME target: ---
GNOME version: ---

glimagesink: Post unhandled navigation events on the bus (2.10 KB, patch)
2015-12-26 21:42 UTC, Florent Thiéry
committed Details | Review

Description Florent Thiéry 2015-12-25 22:15:43 UTC
gst-launch-1.0 videotestsrc ! glimagesink -m
<click/press keys while sink in focus> -- nothing

gst-launch-1.0 videotestsrc ! xvimagesink -m
<click/press keys while sink in focus>
Got message #32 from element "xvimagesink0" (element): GstNavigationMessage, type=(string)event, event=(GstEvent)NULL;

It seems to be quite recent as described in #747245 ; what i do not understand is that glimagesink code is supposed to push the key events in 

static void
gst_glimage_sink_key_event_cb (GstGLWindow * window, char *event_name, char
    *key_string, GstGLImageSink * gl_sink)
  GST_DEBUG_OBJECT (gl_sink, "glimagesink event %s key %s pressed", event_name,
  gst_navigation_send_key_event (GST_NAVIGATION (gl_sink),
      event_name, key_string);

Debugging confirms that the cb is called:
LANG=C GST_DEBUG=glimagesink:5 gst-launch-1.0 videotestsrc ! glimagesink -m
0:00:00.923722580 15729      0x19fe770 DEBUG            glimagesink gstglimagesink.c:721:gst_glimage_sink_mouse_event_cb:<sink> glimagesink event mouse-move at 217, 183

Am i missing something ?
Comment 1 Sebastian Dröge (slomo) 2015-12-26 10:57:30 UTC
xvimagesink additionally post a message (gst_navigation_message_new_event()) while glimagesink only pushes the event upstream.

Do you want to provide a patch? (Also note that navigation support is currently only implemented in the X11 backend of libgstgl)
Comment 2 Florent Thiéry 2015-12-26 20:24:20 UTC
Okay, i will try to provide a patch.

Out of curiosity, why does glimagesink push the event upstream without posting it (for which purpose) ?
Comment 3 Tim-Philipp Müller 2015-12-26 20:44:27 UTC
Perhaps you're missing some context here. Historically, navigation events were *only* pushed upstream into the pipeline (e.g. for DVD menus or such). Only very recently (1.6 I think?) did we add API so that sinks could post navigation events also as messages on the bus in case they were not handled by elements inside the pipeline. So what's happened here is simply that glimagesink hasn't been fixed up to do this yet, but x*imagesink have.
Comment 4 Florent Thiéry 2015-12-26 21:33:48 UTC
Thanks tim, it helps :)

Is it normal that messages are of type GST_MESSAGE_ELEMENT and not GST_MESSAGE_NAVIGATION ?
Comment 5 Florent Thiéry 2015-12-26 21:42:09 UTC
Created attachment 317908 [details] [review]
glimagesink: Post unhandled navigation events on the bus
Comment 6 Florent Thiéry 2015-12-26 22:47:00 UTC
(In reply to Florent Thiery from comment #4)
> Is it normal that messages are of type GST_MESSAGE_ELEMENT and not

Okay, so after some fiddling i found how to get the data from the element message; it's a bit sad that we can't just bus.connect('message::navigation', self._on_navigation_evt), and it's pretty convoluted to get the actual event itself, and to distinguish key presses from pointer activity, but i do now understand that the navigation event is simply posted to the bus as is, as a regular element message.

This means that to extract the actual content (key press for instance), it is required to do the following (python); please let me know if i got it right:

    def _on_message(self, bus, message):
        if message.type == Gst.MessageType.ELEMENT:
            m_struct = message.get_structure()
            m_name = struct.get_name()
            if m_name == 'GstNavigationMessage':
                event = struct.get_value('event')
                e_struct = event.get_structure()
                if e_struct.has_field('key'):
                    key = e_struct.get_value('key')
                elif e_struct.has_field('pointer_x'):
                    x = e_struct.get_value('pointer_x')
                    y = e_struct.get_value('pointer_y')

I'm interested in better ways to separate cursor events from key presses, because the event field name takes quite a lot of different values; for instance, for cursor events, it can have "mouse-move", "mouse-button-press", "mouse-button-release". Having an additional field could help (e.g. "event-type" = "mouse" or "key").
Comment 7 Tim-Philipp Müller 2015-12-26 23:10:29 UTC
The corresponding code in gst-play is this fwiw (not entirely perfect, but gives you another approach):

It would not have been possible to add message::navigation because we ran out of bits/flags, so it would have had to be an 'extended' message type which isn't that much better than an ELEMENT message. C'est la vie, can be fixed in 2.0. The other reason is that this is how to extend stuff from -base that the core module doesn't know about.
Comment 8 Sebastian Dröge (slomo) 2015-12-28 08:05:39 UTC
Attachment 317908 [details] pushed as c68e842 - glimagesink: Post unhandled navigation events on the bus