After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 619155 - [Patch] allow embedded Cocoa NSViews within a GtkWindow to receive events
[Patch] allow embedded Cocoa NSViews within a GtkWindow to receive events
Status: RESOLVED OBSOLETE
Product: gtk+
Classification: Platform
Component: Backend: Quartz
unspecified
Other Mac OS
: Normal enhancement
: ---
Assigned To: gtk-quartz maintainers
gtk-bugs
csw
Depends on:
Blocks:
 
 
Reported: 2010-05-20 02:29 UTC by Paul Davis
Modified: 2018-04-15 00:03 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
proof of concept patch to allow event dispatch to embedded NSViews (1.34 KB, patch)
2010-05-20 02:29 UTC, Paul Davis
none Details | Review
Updated patch that applies cleanly against gtk-2-24 (1.29 KB, patch)
2011-11-29 13:11 UTC, Michael Natterer
none Details | Review
GtkNSView, a widget that embeds an NSView (16.87 KB, patch)
2011-11-29 15:17 UTC, Michael Natterer
none Details | Review
Updated GtkNSView patch (18.29 KB, patch)
2011-12-06 13:52 UTC, Michael Natterer
none Details | Review
Updated GtkNSView patch (18.85 KB, patch)
2012-03-20 12:39 UTC, Michael Natterer
none Details | Review
Experimental patch for proper focus handling (3.52 KB, patch)
2012-03-20 12:42 UTC, Michael Natterer
none Details | Review
1/2: return events on embedded NSViews back to cocoa (2.89 KB, patch)
2013-01-25 10:16 UTC, Michael Natterer
none Details | Review
2/2: add GtkNSView which does the actual embedding, and a testcase (24.28 KB, patch)
2013-01-25 10:18 UTC, Michael Natterer
none Details | Review

Description Paul Davis 2010-05-20 02:29:17 UTC
Created attachment 161512 [details] [review]
proof of concept patch to allow event dispatch to embedded NSViews

Since the switch to CSW, the model in the Quartz backend has been 1 NSWindow == 1 NSView. This prevents applications from playing games and embedding extra NSViews (based on native Cocoa code) within a GtkWindow *if* they want events delivered to the extra NSView(s). The attached proof-of-concept patch provides a mechanism for this to work for the following quartz events:

    case NSLeftMouseDown:
    case NSRightMouseDown:
    case NSOtherMouseDown:
    case NSLeftMouseUp:
    case NSRightMouseUp:
    case NSOtherMouseUp:
    case NSMouseMoved:
    case NSScrollWheel:
    case NSLeftMouseDragged:
    case NSRightMouseDragged:
    case NSOtherMouseDragged:

MouseEntered, MouseExited are already handled correctly. Key events are a different story entirely, and would not be handled by this code.

the basic idea of the patch is (1) if there is more than 1 NSView in the NSWindow, which indicates some non-GTK packing going on then (2) see if the event occured within 1 of the subviews (known to be non-GTK by definition) and return NULL if it did, causing the event to passed "back" to the normal Cocoa event handling system rather than being processed by GDK.

I remain agnostic on whether or not GTK+ ought to be providing the ability to embed Cocoa views into a GtkWindow, but its critical for the Ardour project and I imagine might prove useful for other things too. Comments invited.
Comment 1 Kristian Rietveld 2010-06-26 19:40:36 UTC
(In reply to comment #0)
> the basic idea of the patch is (1) if there is more than 1 NSView in the
> NSWindow, which indicates some non-GTK packing going on then (2) see if the
> event occured within 1 of the subviews (known to be non-GTK by definition) and

As far as I know, there can only be a single toplevel NSView for an NSWindow.  So you only check for condition (2) in the patch, only for subviews of the toplevel view (which is known to belong to GTK).

With this patch, I guess you use gdk_quartz_window_get_nswindow()/gdk_quartz_window_get_nsview() to get your own NSViews packed.

> I remain agnostic on whether or not GTK+ ought to be providing the ability to
> embed Cocoa views into a GtkWindow, but its critical for the Ardour project and
> I imagine might prove useful for other things too. Comments invited.

I think it's an interesting idea, from reading the bug subject I actually thought your patch would allow embedding non-GTK NSViews *anywhere* in the widget hierarchy.  Though I am not even sure if that would be easily possible with the current backend. (I doubt it).

I would say that if we would add a patch like this, we also want to provide the proper GdkQuartz specific API to get the non-GTK views embedded.  Poking inside the NSView hierarchy using a pointer retrieved from gdk_quartz_window_get_nsview() is not something that I think we should encourage ;)  The patch would then turn into something that's more general and abstract.

Related idea is to make it possible to have select GtkWidgets/GdkWindows being backed by an NSView (basically by a "native window" as is explicitly possible in the X backend) or by a CALayer for doing fancy stuff.  I would have to read up on CoreAnimation first then.  Also, this might not be easily possible with the current backend either ...


Final thought: isn't this actually related to "foreign" windows?  If so, then this API will have to be correctly implemented.
Comment 2 Paul Davis 2010-06-27 00:01:37 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > the basic idea of the patch is (1) if there is more than 1 NSView in the
> > NSWindow, which indicates some non-GTK packing going on then (2) see if the
> > event occured within 1 of the subviews (known to be non-GTK by definition) and
> 
> As far as I know, there can only be a single toplevel NSView for an NSWindow. 
> So you only check for condition (2) in the patch, only for subviews of the
> toplevel view (which is known to belong to GTK).

No. I'm counting how many subviews there are of the the View that GTK has set as the "content view" for the window. If its a pure GTK window, there aren't any - GTK only creates no subviews of the toplevel. If we've added one somehow, there we see if the event is within the subviews, since by definition, they are all non-GTK.

> I think it's an interesting idea, from reading the bug subject I actually
> thought your patch would allow embedding non-GTK NSViews *anywhere* in the
> widget hierarchy.  Though I am not even sure if that would be easily possible
> with the current backend. (I doubt it).

this strikes me as a nonsensical goal simply because of the observation we've both made above: there is 1 GTK NSView per window; you can only pack NSViews inside other NSViews, so there's only 1 NSView available to pack anything into. Not very flexible, but powerful for embedding Cocoa views into GTK parent windows.
 
> I would say that if we would add a patch like this, we also want to provide the
> proper GdkQuartz specific API to get the non-GTK views embedded.  Poking inside
> the NSView hierarchy using a pointer retrieved from
> gdk_quartz_window_get_nsview() is not something that I think we should
> encourage ;)  The patch would then turn into something that's more general and
> abstract.

there's no reason to provide any such API. the added view didn't appear from outer space, and it wasn't added by some alien being: the application had the NSView before it packed it into the window, and thus it doesn't need GTK to provide a handle on it. i don't think that trying to make this more general and abstract is really a wise goal. 
 
> Related idea is to make it possible to have select GtkWidgets/GdkWindows being
> backed by an NSView (basically by a "native window" as is explicitly possible
> in the X backend) or by a CALayer for doing fancy stuff.

i can't see any sensible use for this.

> Final thought: isn't this actually related to "foreign" windows?  If so, then
> this API will have to be correctly implemented.

i don't think so. an NSView is not an NSWindow, and its not even equivalent to a GdkWindow either.
Comment 3 Kristian Rietveld 2010-06-27 08:02:26 UTC
(In reply to comment #2)
> > As far as I know, there can only be a single toplevel NSView for an NSWindow. 
> > So you only check for condition (2) in the patch, only for subviews of the
> > toplevel view (which is known to belong to GTK).
> 
> No. I'm counting how many subviews there are of the the View that GTK has set
> as the "content view" for the window. If its a pure GTK window, there aren't
> any - GTK only creates no subviews of the toplevel. If we've added one somehow,
> there we see if the event is within the subviews, since by definition, they are
> all non-GTK.

That's exactly what I said above, I should have written "can only be a single content NSView for an NSWindow".

> > I think it's an interesting idea, from reading the bug subject I actually
> > thought your patch would allow embedding non-GTK NSViews *anywhere* in the
> > widget hierarchy.  Though I am not even sure if that would be easily possible
> > with the current backend. (I doubt it).
> 
> this strikes me as a nonsensical goal simply because of the observation we've
> both made above: there is 1 GTK NSView per window; you can only pack NSViews
> inside other NSViews, so there's only 1 NSView available to pack anything into.

Getting this done would likely involve changing the current semantics of how GTK+ deals with NSViews of course.

> Not very flexible, but powerful for embedding Cocoa views into GTK parent
> windows.
> 
> > I would say that if we would add a patch like this, we also want to provide the
> > proper GdkQuartz specific API to get the non-GTK views embedded.  Poking inside
> > the NSView hierarchy using a pointer retrieved from
> > gdk_quartz_window_get_nsview() is not something that I think we should
> > encourage ;)  The patch would then turn into something that's more general and
> > abstract.
> 
> there's no reason to provide any such API. the added view didn't appear from
> outer space, and it wasn't added by some alien being: the application had the
> NSView before it packed it into the window, and thus it doesn't need GTK to
> provide a handle on it. i don't think that trying to make this more general and
> abstract is really a wise goal. 

Why doesn't the application need to provide a handle on that window?  In the current setup you would be packing the view into the window without GTK+ knowing about it.  This usually counts as poking inside internals which is generally discouraged.

> > Related idea is to make it possible to have select GtkWidgets/GdkWindows being
> > backed by an NSView (basically by a "native window" as is explicitly possible
> > in the X backend) or by a CALayer for doing fancy stuff.
> 
> i can't see any sensible use for this.

More of a research idea and I am not sure if this is easily possible either.  But the former would give you back a hierarchy of NSViews for select widgets, which could help the case if one does want to pack a Cocoa NSView in a non-toplevel NSView.  The later is for doing animations based on the native library (people have done similar things with Clutter on Linux for example).

So this is really a fuzzy research idea: I am absolutely not planning to look into this right now of course, there are much more pressing issues to be resolved first.


> > Final thought: isn't this actually related to "foreign" windows?  If so, then
> > this API will have to be correctly implemented.
> 
> i don't think so. an NSView is not an NSWindow, and its not even equivalent to
> a GdkWindow either.

Pre-CSW all GdkWindows were backed by an NSView, so you could say that NSViews and GdkWindows were roughly equivalent.  Foreign windows have been defined in the GDK API as foreign X windows.  X windows are mostly equivalent to GdkWindows.  In the Cocoa setting it would really be "foreign" NSViews I guess.  An issue here is that GdkWindow kind of acts as both the NSView and NSWindow in case of toplevels windows, which feels a bit unnatural.

The main problem here could be that the foreign windows API is too specific to X11.
Comment 4 Paul Davis 2010-06-27 15:10:47 UTC
(In reply to comment #3)
> (In reply to comment #2)

> > there's no reason to provide any such API. the added view didn't appear from
> > outer space, and it wasn't added by some alien being: the application had the
> > NSView before it packed it into the window, and thus it doesn't need GTK to
> > provide a handle on it. i don't think that trying to make this more general and
> > abstract is really a wise goal. 
> 
> Why doesn't the application need to provide a handle on that window?  In the
> current setup you would be packing the view into the window without GTK+
> knowing about it.  This usually counts as poking inside internals which is
> generally discouraged.

my point was that the application has a handle on the NSView already. it doesn't need a way to get it back from GTK at a later time. in addition, as you noted, gtk/quartz already provides a way to get the content view for a given top level window, and that's about all that is needed.

otherwise you're talking about a GTK API to permit the manipulation of NSView packing, which is totally not part a part of GTK. 

the purpose of this patch was to admit that an application (e.g. Ardour) might do this sort of thing, and that if it did, events should still get delivered to the view(s) that were added. i.e. GTK should not try to facilitate this, but it should not get in the way of it either.
Comment 5 Michael Natterer 2011-11-29 13:11:08 UTC
Created attachment 202368 [details] [review]
Updated patch that applies cleanly against gtk-2-24
Comment 6 Michael Natterer 2011-11-29 15:17:35 UTC
Created attachment 202379 [details] [review]
GtkNSView, a widget that embeds an NSView

Apply this on top of the event handling patch:

Known issues:

- the widget takes focus by keynav, but not by button press
- key repeat is weirdly broken, I likely use the wrong API
  to forward key events
- while the widget has focus, cursor keys are only good to
  navigate the focus out of the widget again, but not to
  e.g. scroll the WebView in the test case (cursor keys
  in text entries do work though)
- it's completely untested for the case where widget->window
  is not the toplevel window (easy to fix, just not done)
Comment 7 Michael Natterer 2011-12-06 13:52:34 UTC
Created attachment 202911 [details] [review]
Updated GtkNSView patch

- The widget now properly works in a sub-Gdkwindow, added a test
  for that to tests/testnsview
- can-focus is now set to [view acceptsFirstResponder]

The key repeat issue mentioned above doesn't exist,
Lion disables key repeat by default. The remaining keynav
issues still exist.
Comment 8 Michael Natterer 2012-03-20 12:39:47 UTC
Created attachment 210173 [details] [review]
Updated GtkNSView patch

Rebases patch, with a slightly enhanced test case.
Comment 9 Michael Natterer 2012-03-20 12:42:35 UTC
Created attachment 210174 [details] [review]
Experimental patch for proper focus handling

This patch is on top of the GtkNSView patch, and documents my experiments
to get focus handling done properly.
Comment 10 John Ralls 2012-03-20 13:46:48 UTC
(In reply to comment #9)
> Created an attachment (id=210174) [details] [review]
> Experimental patch for proper focus handling
> 
> This patch is on top of the GtkNSView patch, and documents my experiments
> to get focus handling done properly.

Is this to fix the problem of having to move the mouse pointer out of a top level parent in order to get buttons to work?
Comment 11 Michael Natterer 2012-03-22 13:37:11 UTC
No this experimental patch is only for keyboard focus, and the patch
doesn't work properly.
Comment 12 Michael Natterer 2013-01-25 10:16:49 UTC
Created attachment 234377 [details] [review]
1/2: return events on embedded NSViews back to cocoa

This new patch add a new GdkWindow signal "native-child-event" which
gets emitted on events on the embedded NSView, so the GtkWidget which
handles the embedding can do its part.
Comment 13 Michael Natterer 2013-01-25 10:18:48 UTC
Created attachment 234378 [details] [review]
2/2: add GtkNSView which does the actual embedding, and a testcase

This GtkNSView patch adds proper focus handling, which now works almost
perfectly, unless the embedded view has its own idea how things work.
The only known issue is now that Shift-Tab on an embedded WebView doesn't
get the focus order right.
Comment 14 xiexingood 2014-05-19 03:11:35 UTC
With this patch, the NSViews within a GtkWindow can receive events, but the GtkWidget at the same position can't receive event, I use Gtk# widget, so I want to handle the event with C# code, do we have any ways to solve this problem?
Comment 15 Matthias Clasen 2018-02-10 05:14:43 UTC
We're moving to gitlab! As part of this move, we are moving bugs to NEEDINFO if they haven't seen activity in more than a year. If this issue is still important to you and still relevant with GTK+ 3.22 or master, please reopen it and we will migrate it to gitlab.
Comment 16 Matthias Clasen 2018-04-15 00:03:36 UTC
As announced a while ago, we are migrating to gitlab, and bugs that haven't seen activity in the last year or so will be not be migrated, but closed out in bugzilla.

If this bug is still relevant to you, you can open a new issue describing the symptoms and how to reproduce it with gtk 3.22.x or master in gitlab:

https://gitlab.gnome.org/GNOME/gtk/issues/new