GNOME Bugzilla – Bug 119189
Add OpenGL support to GTK+
Last modified: 2014-10-21 21:43:18 UTC
Having support for OpenGL distributed with GTK+ might be something that would be useful in the future. It would likely need to be a separate small library so that every GTK+ app didn't need to link I'd would imagine something like the old GtkGLArea widget (gtkglarea in GNOME CVS) More currently maintained is GtkGLExt: http://gtkglext.sourceforge.net/ Nut it's sort of an idiosynchratic scheme that I don't think fits into the GTK+ model very well. You can attach OpenGL capabilities to arbitrary widgets without cooperation from the widget implementation.
*** Bug 118982 has been marked as a duplicate of this bug. ***
Hi Owen, I'm a maintainer of GtkGLExt. About the "idiosynchratic scheme" (adding extra gtk_widget_*() functions), this is why I decided to use such approach... 1. GtkDrawingArea can be used for OpenGL rendering, and no additional widget is needed. 2. For glade and libglade, adding new custom widget cannot be used until glade supports it. 3. I'm lazy ;-) GtkGLExt package is conposed with two libraries, GdkGLExt and GtkGLExt. I think that GdkGLExt is more important than GtkGLExt. GtkGLExt is just a simple support library which enables developers to use OpenGL with GtkDrawingArea. If GdkGLExt library is integrated into GDK, GtkDrawingArea will be able to support OpenGL, and GtkGLExt library will go away. void gtk_drawing_area_enable_gl (GtkDrawingArea *drawingarea, GdkGLConfig *glconfig); GdkGLDrawable * gtk_drawing_area_get_gl_drawable (GtkDrawingArea *drawingarea); GdkGLContext * gtk_drawing_area_create_gl_context (GtkDrawingArea *drawingarea); GdkGLContext * gtk_drawing_area_get_gl_context (GtkDrawingArea *drawingarea); Most important thing is: GtkDrawingArea can be used for OpenGL rendering. If GdkGLExt is integrated into GDK, GtkDrawingArea can support both X11 and OpenGL rendering naturally. Regards, --Naofumi
I wrote that GtkDrawingArea can be used for OpenGL rendering. But I don't mind adding new GtkGLArea widget or something. One year ago, I suggested to GtkGLArea people that GtkGLArea-2.0 could be impremented upon GtkGLExt. But I received no answer... I'm just working for new GtkGLExt-2.0 API, so I'd like to ask your advice. Thanks, --Naofumi
Created attachment 22045 [details] OpenGL integration RFC
Any news here? Can't GtkGlExt be blessed peacefully in the mean time?
I don't particularly understand why a separate GL widget is needed, but for what it's worth, I wrote a hybrid GTK/GL program a while back: http://www.jwz.org/lightlab/ All there was to it, really, was getting the proper X Visual from glXChooseVisual() and then ensuring that gtk_widget_set_visual() and gtk_widget_set_colormap() were called on the DrawingArea before it was realized; and then glXCreateContext() called with the GtkWidget's underlying Window. After that, I just ran vanilla OpenGL rendering code from an idle timer. Works good. I guess some of this could use a patina of abstraction, to in service of the notion that someone might someday run GTK on top of Windows, but it doesn't sound like that's more than a dozen lines of code. Heck, it may just be macros.
Owen, This is my initial implementation of GtkGLDrawingArea widget. http://gtkglext.sourceforge.net/test/gtkgldrawingarea-0.0.1.tar.gz To build it, GtkGLExt is needed, because it is built upon GtkGLExt's libgdkglext library. This GtkGLDrawingArea widget basically models after the traditional SGI's GLwDrawingArea widget. I'd like to ask your advice.
This is the new version of GtkGLDrawingArea widget implementation. http://gtkglext.sourceforge.net/test/gtkgldrawingarea-0.0.2.tar.gz What do you think of it? Please tell me your opinion. If it looks well, I'm going to provide it with GtkGLExt-2.0, instead of gtk_widget_* add-on APIs.
I am not fond of this API abstraction. I'm *especially* troubled by the functions which attempt to "set" characteristics but actually destroy and rebuild a framebuffer. Such actions normally require trips through the kernel, big memory clears, context resets, etc. There are a lot of side effects here that will not be immediately obvious; these things are not just a couple of variables in memory. If a framebuffer needs to get destroyed to change a characteristic, the programmer needs to handle it. I do agree that framebuffer configuration needs an abstraction; however, an abstraction of the fbconfig protocol (either the SGIX-enumerate everything and let the user pick or the X11-send a set of flags and get a list of valid modes) is probably the way to go even for WGL.
With the recent announcement, this is part of Project Ridley. Looking at the gtkglext code, I think it's a bit overcomplicated for what we need. Like the cairo integration, we really need a single function (gdk_gl_create?) that returns an OpenGL visual. It's not that easy though, since OpenGL assumes a global current context, so that needs a few more functions to push a GL context to global and pop it... Honestly I do not understand why is all the code in gtlglext needed. But anyway, anybody like to work on this?
Once again, (I got this confirmed by Owen a few months back), we don't want an OpenGL widget. The gdk+cairo integration should give a good idea about how this should look like...
*** Bug 337499 has been marked as a duplicate of this bug. ***
By gdk+cairo API what I meant is that, the entire cairo integration of gtk+gdk api comes down to a single call gdk_cairo_create, that returns a new cairo context for a gdk drawable. You can use cairo to draw to any widget this way. The rest of gdkcairo.h is four other calls, to use GdkColor, GdkPixbuf, GdkRectangle, and GdkRegion directly on cairo contexts. We need similar API for OpenGL too, but it's easy stuff, except for the pixbuf one. texture_from_pixbuf extension comes to my mind. I had a quick look at gtkglext. It almost gets it all right and just needs to be integrated into gdk/gtk. The calls like gtk_widget_get_gl_context look specially good to me. We need to automate some of the configuration, like double-buffering and visual selection..., but definitely don't want to create any limitations. The only controversial part of gtkglext, is the huge generated wrappers for all the OpenGL API, as it dlopen's GLX. I'm not quite sure what pros/cons are. It can better handle different versions of OpenGL this way, but it also means a rather huge library. I've heard Qt dlopens too. If the OpenGL support in gtk is going to be put in a separate library, dlopen makes a lot of sense, otherwise I'm not sure. Matthias, what do you think?
*** Bug 341726 has been marked as a duplicate of this bug. ***
In gnome-games, we are considering to add glChess, which uses GtkGLExt. The dependency on GtkGLExt would be optional, given the current situation. However, it would better if GtkGlExt would be fully supported. I would like to say that from the gnome-games maintainers' perspective, it would be very practical if GtkGLExt would be supported in GNOME 2.17.x. Proper OpenGL support in GNOME is a win-win situation for everyone. So I fully support OpenGL support in GNOME and hope that it will happen before 2.18 :)
As a "heads up", I am the new maintainer of GtkGLExt, so I'm extremely interested in this thread. Since I am primarily an application developer, I'd love to see GtkGLExt or similar functionality moved into GTK proper. In particular, I'm very open to slimming-down the library so that its sole focus is on GTK/OpenGL integration. Is there anyone out there who is actively working this issue? Regards, Tim Shead tshead@k-3d.com
Hi Tim. I don't think anyone's actively working on this. Lets get it rolling though. Can you enlighten me why the wrappers in GtkGlExt are there in the first place? Why not call into GLX directly?
I assume you're talking about the contents of gdk/glext. The wrappers you see there are actually for a wide variety of extensions to OpenGL/GLX/WGL, most of which are vendor-specific. Each extension usually includes custom compile-time constants and function prototypes, which would normally have to be obtained from the vendor's header files. GtkGLExt is providing that information all in one place as a convenience. It's useful stuff to have, but it isn't Gtk-specific, and it doesn't belong in a Gtk glue library like GtkGLExt (there are other libraries that do this better already). Cheers, Tim
I have a strong interest in seeing gtkglext get incorporated into gtk (although I'm mainly only interested in X/GLX integration). Since there doesn't seem to be a lot of action in this direction, I'm wondering if there is anything I can do to help. Has there been any progress in deciding what form the integration would take? -bryan
Hello Bryan, I don't think we can get it integrated if there's not interest in getting it working on all platforms. We have a lot of users on non-X/GLX systems that we don't want to marginalize. What features of gtkglext are you interested in seeing? You want buttons that sparkle, shine and rotate? I'd like to see OpenGL find its way into gtk+ proper, but I'm not convinced that we need to have the entire widget set support GL rendering. I wouldn't be opposed to seeing gtkglext finding its way in to the core, should we be able to do it and support all platforms while we're at it.
(In reply to comment #3) > I wrote that GtkDrawingArea can be used for OpenGL rendering. But I don't mind > adding new GtkGLArea widget or something. > > One year ago, I suggested to GtkGLArea people that GtkGLArea-2.0 could be > impremented upon GtkGLExt. But I received no answer... > > I'm just working for new GtkGLExt-2.0 API, so I'd like to ask your advice. > > Thanks, > --Naofumi > Heya Naofumi, The GtkGLArea folks have not been around for a while, so I've taken over the project. Let me know how you would like to see gtkglarea interface with gtkglext. Cheers, C.J.
C.J: I am the maintainer of GtkGLExt, having taken over from Naofumi Yasufuku last year. Over time GtkGLExt has accumulated a moderate amount of general-purpose OpenGL functionality that isn't particular to the GTK/OpenGL integration task. I consider that a mistake (and I know that it has been off-putting to others), so I am starting to outline the minimal subset of GtkGLExt that is actually necessary to write OpenGL applications on GTK. The results will be posted at the GtkGLExt wiki at: http://www.k-3d.org/gtkglext GtkGLExt has backends for GLX and Win32, so platform compatibility isn't an issue - the code is widely deployed and reliable. I think the main obstacle to getting something suitable for inclusion in GTK is my lack of experience working with the Gnome/GTK community. If there is experience here on the preferred way to submit GtkGLExt for inclusion with GTK, please let me know. Regards, Tim
(In reply to comment #22) > my lack of experience working with the Gnome/GTK community. If there is > experience here on the preferred way to submit GtkGLExt for inclusion with GTK, > please let me know. This bugzilla entry is the right place to submit stuff, although at some point we may want to propose any design choices etc on gtk-devel-list or something. > Regards, > Tim >
I'll make this happen. I'm currently adjusting gdkgl.c and gtkglarea.c in such a way that they can be merged into the gtk+ package. I need to add some gtk-doc documentation to the source, make sure that all of the core GDK_WINDOWING_* platforms are supported. It will take time, though. It's taken me a year to get maintainer status of gtkglarea and to make it compile on win32. Don't expect miracles. I could use some support.
So what are your plans? A new widget?
(In reply to comment #25) > So what are your plans? A new widget? > Well, I still need to do some research. As stated above, I'd like to work with the GtkGLExt folks to see how they're interacting with wgl, agl, and glx. I need to find out if there's an interface to the opengl boards that works well on fb and directfb. GtkGLArea seems pretty solid to me. I'm going to make a few changes to it. I expect to add the functionality referred to in the RFC that does not already exist in the widget (very little) and then integrate these changes into the core of gtk+. After initial integration, I expect the gtkglarea/ source tree to be the staging ground for new features. We can try out new ideas this way without altering the core. I need some folks to register as developers for this project who have experience with GtkGLArea and GtkGLExt, WGL, GLX, AGL, XGL, AIGL, etc. Even if you don't have time to write code, we could use your ideas and experience. Feel free to contact me offline if you prefer to be anonymous.
glChess in gnome-games uses GtkGLExt: http://live.gnome.org/glChess It's a great library for developing OpenGl applications and games because it integrates well with GNOME already.
(In reply to comment #27) > glChess in gnome-games uses GtkGLExt: http://live.gnome.org/glChess > It's a great library for developing OpenGl applications and games because it > integrates well with GNOME already. > Thank you Andraes. If you have familiarity with using any of the common GL APIs, please sign up as a developer and let me know where your strengths lay. Take ownership of any bugs that you think you can address, or offer to support the current assignee if they have already been accepted.
Hey folks, I've been idling for a few months. I got a working win32 gtkglarea into head, but haven't made a formal release yet. Should I reduce the two files (X11 case, win32 case) into a single file with #ifdef macros, or handle platform specification using configure.ac and Makefile.am? After gtkglarea proves stable on win32 and X11, I'll look in to how it can be merged in as a core widget. Cheers, C.J.
Again, I don't think we need a new widget here... That's just my opinion though.
Some have been ringing this bell again and again, what is the use/point of a new widget here ? I agree with Behdad that we dont need a widget, and that it would be nice to see cairo/gtkglext as two separate drawing subsystems that could be used on widgets. On that note, it would be nice to take into consideration a possible future GtkCanvas, will the user have an easy option to use GtkGLExt on their canvas ? will it be easy-as-pie to use cairo to draw stuff onto offscreen texture buffers ?
(In reply to comment #31) > I agree with Behdad that we dont need a widget, and that > it would be nice to see cairo/gtkglext as two separate > drawing subsystems that could be used on widgets. Exactly. Except that instead of "cairo/gtkglext" I'd say "cairo/OpenGL". > On that note, it would be nice to take into consideration > a possible future GtkCanvas, will the user have an easy > option to use GtkGLExt on their canvas ? will it be > easy-as-pie to use cairo to draw stuff onto offscreen > texture buffers ? To mix cairo and OpenGL there are basically two routes: 1) Using the pixmap-to-texture extension present in newer drivers (the one that Compiz uses), and 2) Use cairo's glitz backend. The glitz backend is pretty experimental at this point. But in the future, it may be possible to have Gtk+ probe and decide which of the two to use.
Great to see interest in the domain of proper GL-integration into gtk+/GNOME happening. I'm available for any testing and sugguestions towards this new API and set of functionalities. I'll guess C.J. Adams-Collier and Timothy M. Shead are the people to contact for up-to-date information on this effort? My normal email-address is currently not working (but will in the near future). If you want to contact me right now you can do so at: macslow at gmail dot com
What is the recent progress? Is there any review or roadmap?
I'm going to be working on this tomorrow. It would help me if I could get some testers. I'm in the Seattle area, so I'm on the Pacific timezone. I'll be on the #gtk+ channel while I'm hacking away. I did some changes to the code that bring it pretty close to a 2.0 alpha release. I'll push a tarball to my home machine for testing: http://karma-colliertech.dyndns.org/~cjcollier/tmp/gtkglarea-2.0.0a.tar.gz I'll be up in a few hours to answer questions :) Cheers, C.J.
I've committed this code to the win32-fixes branch. It basically relegates x11 to one of the supported gdk targets rather than having it be the default. The current gdk target is discovered using pkg-config. x11 and win32 seem to be working fine to me. The others have not been tested but are expected to fail. The keyboard's broken on my powerbook, and I haven't tried out gtk+ on quartz yet anyhow, and since I've only got a little time this weekend, I've pushed that back. I'm testing the lib with gtkglarea-sharp on linux right now and may try it on windows in the next few days. I'd be glad of any win32 testers with mingw32 experience. Behdad, I'm curious about your thoughts concerning a stack of gl contexts. Could you share a bit more of your thoughts on the subject? Cheers, C.J.
I've been re-organizing for a 1.999 release of gtkglarea, and it looks like it might be better to push first some of the opengl context manipulation methods into the core before we consider anything further. I'm going to get this dist wrapped up and then I'll come write some more thoughts here.
I'm splitting up the contents of gtkglarea's gtkgl/ directory into something that more resembles gtk+'s structure: http://svn.gnome.org/svn/gtkglarea/trunk/gtkgl/ gtkgl/gdkgl-win32.c -> gdk/win32/gdkgl-win32.c -> gdk/win32/gdkglpixmap-win32.c gtkgl/gdkgl.c -> gdk/x11/gdkgl-x11.c -> gdk/x11/gdkglpixmap-x11.c gtkgl/gdkgl.h -> gdk/gdkgl.h -> gdk/gdkglpixmap.h gtkgl/gtkgl.def -> gdk/gdkgl.symbols gtkgl/gtkglarea.c -> gtk/gtkglarea.c gtkgl/gtkglarea.h -> gtk/gtkglarea.h ---- I am also going to document in gtk-doc format the public API from gdk/gdkgl.h: gdk_gl_query gdk_gl_get_info gdk_gl_choose_visual gdk_gl_get_config gdk_gl_context_get gdk_gl_context_new gdk_gl_context_share_new gdk_gl_context_attrlist_share_new gdk_gl_make_current gdk_gl_swap_buffers gdk_gl_wait_gdk gdk_gl_wait_gl (TODO: move the gdk_gl_pixmap class into its own .c/.h files) gdk_gl_pixmap_class_init gdk_gl_pixmap_get_type gdk_gl_pixmap_finalize gdk_gl_pixmap_new gdk_gl_pixmap_make_current Likewise gtk/gtkglarea.h: gtk_gl_area_get_type gtk_gl_area_new gtk_gl_area_share_new gtk_gl_area_new_vargs gtk_gl_area_make_current gtk_gl_area_swap_buffers
I don't want to discourage you, but I'd like to point out that there has been no endorsement given that any of this will land in GDK, nor can there before the design has been discussed on gtk-devel-list and generally agreed to.
also split gtk_gl_context_* into its own .h and .c file: http://svn.gnome.org/svn/gtkglarea/branches/win32-fixes/gdk/
Thanks Matthias, I'm not at all discouraged. I'm going to get things working in the gtkglarea module before I propose anything to the list. Also, my mail server was compromised, so the address that ccollier@cvs.gnome.org points at is bouncing right now anyhow. I guess I should probably subscribe with my temporary address, but I've only got so many minutes per day to work on this :) Cheers, C.J.
Humm, are the clutter and other OpenGL toolkit guys interested in tackling this?
Just a comment with my company's head of R&D hat on. We sell professional GTK#/OpenGL-based software that makes use of gtkglarea (and gtkglarea-sharp). We chose it because it is simple (we don't need much more than a GL context and an event loop), portable (our #1 platform is Win32) and stable (I have been using gtkglarea in production for years). Other projects featuring offscreen rendering, arbitrary widget embedding, scene graphs etc. are appealing, but really too complex for our needs. We would like to invest company time and resources in helping integrate a simple solution for embedding OpenGL in GTK+.
Thanks Sam. That's encouraging. We don't want to have a radical solution with scene graphs, etc in Gtk+. However, we don't want an OpenGL widget either. OpenGL should be an alternative way (to cairo) to draw to a widget. That's the essential different between gtkglext and gtkglarea approaches.
I'm developing Zeobuilder. (http://molmod.ugent.be/code/wiki/Zeobuilder) We are currently using the python binding to gtkglext. I just want to say that a gtkglarea (as part of the gtk+ distribution), would be sufficient four our needs. Could anyone mention me an existing application where gtkglext offers real life advantages over gtkglarea?
just to add some actual API discussion: this is how I'd like to see the integration happen - * add a GdkGLContext, wrapping the per-platform GL context * add a single call: GdkGLContext *gdk_gl_create_context (GdkDrawable *drawable, guint *gl_attributes); this function would: 1. create the GdkGLContext, if not already present 1a. [GLX implementation] call glXChooseVisual() on the X11 display and screen of the GdkDrawable, and the passed gl_attributes; 1b. [GLX implementation] call glXCreateContext() to create the GLX context; 1c. [GLX implementation] call glXMakeCurrent() to bind the GL context on the current drawable; 1d. cache the GdkGLContext and return it with refcount++; 2. return the cached GdkGLContext, with refcount++, unless the gl_attributes are different from the cached ones - in which case destroy the cached GdkGLContext and return to 1. * the function above would be called within an handler of the ::expose-event signal * at the end of the handler, g_object_unref (gl_context) should be called to: 1. wait for vblank, if the drivers/platform support it, to avoid tearing; 2. [GLX implementation] call glXSwapBuffers() to get the drawing on the window. ref: http://mail.gnome.org/archives/gtk-app-devel-list/2008-March/msg00025.html and http://mail.gnome.org/archives/gtk-app-devel-list/2008-March/msg00036.html the only problem is: GTK+ allows multiple drawables, while GL contexts are per process, so the locking and/or other limitations must be taken into account. (In reply to comment #45) > I'm developing Zeobuilder. (http://molmod.ugent.be/code/wiki/Zeobuilder) We are > currently using the python binding to gtkglext. I just want to say that a > gtkglarea (as part of the gtk+ distribution), would be sufficient four our > needs. Could anyone mention me an existing application where gtkglext offers > real life advantages over gtkglarea? GtkGLArea is a full widget; with a cairo-like/gtkglext-like implementation you could add GL support to every widget and develop custom widgets without subclassing GtkGLArea.
(In reply to comment #46) > just to add some actual API discussion: this is how I'd like to see the > integration happen - > > * add a GdkGLContext, wrapping the per-platform GL context > * add a single call: > > GdkGLContext *gdk_gl_create_context (GdkDrawable *drawable, > guint *gl_attributes); > > this function would: > > 1. create the GdkGLContext, if not already present > 1a. [GLX implementation] call glXChooseVisual() on the X11 display > and screen of the GdkDrawable, and the passed gl_attributes; > 1b. [GLX implementation] call glXCreateContext() to create the GLX > context; > 1c. [GLX implementation] call glXMakeCurrent() to bind the GL context > on the current drawable; > 1d. cache the GdkGLContext and return it with refcount++; > 2. return the cached GdkGLContext, with refcount++, unless the > gl_attributes are different from the cached ones - in which case > destroy the cached GdkGLContext and return to 1. > > * the function above would be called within an handler of the > ::expose-event signal > * at the end of the handler, g_object_unref (gl_context) should be > called to: > > 1. wait for vblank, if the drivers/platform support it, to avoid > tearing; > 2. [GLX implementation] call glXSwapBuffers() to get the drawing > on the window. I prefer an explicit begin()/end() pair for making current and swapping. How do you make unref(gl_context) trigger the swapping if the object has more than one references anyway? > ref: http://mail.gnome.org/archives/gtk-app-devel-list/2008-March/msg00025.html > and http://mail.gnome.org/archives/gtk-app-devel-list/2008-March/msg00036.html > > the only problem is: GTK+ allows multiple drawables, while GL contexts are per > process, so the locking and/or other limitations must be taken into account. > > (In reply to comment #45) > > I'm developing Zeobuilder. (http://molmod.ugent.be/code/wiki/Zeobuilder) We are > > currently using the python binding to gtkglext. I just want to say that a > > gtkglarea (as part of the gtk+ distribution), would be sufficient four our > > needs. Could anyone mention me an existing application where gtkglext offers > > real life advantages over gtkglarea? > > GtkGLArea is a full widget; with a cairo-like/gtkglext-like implementation you > could add GL support to every widget and develop custom widgets without > subclassing GtkGLArea. >
I saw that GtkGLExt was proposed for inclusion in Gnome 2.22, How'd that transition go or did it stall? Was any resolve found on whether an openGL context should be provided by a widget or whether OpenGL should be an option provided in a similar manner to the cairo API?
Any news on this bug? Is someone working on it? GtkGLExt seems to be rather obsolete by now since last release was on the 5th of February 2006.
Hey there George, I haven't really done anything with the bug in a while. I've been preoccupied with work that pays the bills for a while now. Sam and I have (but mostly Sam) been making patches to gtkglarea and its c# wrapper. We need to ensure better support for win32 before we propose integrating the widget into core. There is a lot of opposition to it, so we'll also need to prove that there is interest in seeing an OpenGL widget as part of the core. The examples and documentation could use a little love, as well. If anyone's interested in contributing some of these fixes, I think I'll have time to coordinate. Cheers, C.J.
(In reply to comment #49) > Any news on this bug? Is someone working on it? GtkGLExt seems to be rather > obsolete by now since last release was on the 5th of February 2006. GtkGLExt isn't obsolete. To the contrary, it is actively maintained. There only hasn't been a new release for 2 reasons * There had not been a technical reason for doing so. * The GTKGLExt project is trying to move to gnome.org, but progress on this has shown to be a snail.
I'm interested in this for two reasons. The programs I work with already have a GL context, and it would be good to move parts of those programs into c#, so what is left behind is a DLL for which the C# program now has control over. The other reason is to be more portable between linux and win32. I've tried Cairo, Rsvg, and other alpha-enabled techniques and found them a hassle to be portable. Cairo sounds like it would carry the context well, but it is not exposed by default since .net wouldn't know about it. Rsvg and Pixbuf blitters don't expose enough to enable dynamic content. Static images work for composites, but I need them dynamically controlled and resize-able. See bug 566389 for example. Besides those two, there is an active 3D view (not just 2D composites), and it would be nice to manipulate the 3D view from C# even if the main 3D engine is left in C++. Just being able to specify/handle the outer most GL context in Gtk#/Gdk# would probably be enough for the 3D view. It has hooks for a Cairo context, so I figure Gtk/Gdk could do a subset likewise for GL. 2D composites don't have to be done in GL, as others felt likewise in above messages where Cairo is suggested. It may be worthwhile to rewrite parts of Rsvg into C# so that dynamics can be improved and used more easily, like being able to use System.Xml instead of SAX, animation callbacks, and UI controls, all with the fancy composite images without worry if the system has a composite manager or not.
Dzonatas: Sam and I have been building a c# wrapper around gtkglarea for a while. Take a look and let us know if it's what you're looking for: http://www.mono-project.com/GtkGLSharp http://packages.debian.org/source/lenny/gtkglarea-sharp http://anonsvn.mono-project.com/viewvc/trunk/gtkglarea-sharp/ Cheers, C.J.
After discussing this issue with the list last weekend, it sounds like distributing a .so of gdkgl along with gtk might be an interesting solution. gdkgl.[hc] define classes for managing OpenGL contexts and pixmaps. http://svn.gnome.org/svn/gtkglarea/trunk/gtkgl/gdkgl.c http://svn.gnome.org/svn/gtkglarea/trunk/gtkgl/gdkgl.h Thoughts?
> http://packages.debian.org/source/lenny/gtkglarea-sharp It appears the related packages in lenny are still stuck in unstable, which probably prevents further consideration for a binary package rather than just a source package. I found why: http://wiki.debian.org/Teams/DebianMonoGroup/CSCNameClash However, I was able to stay with lenny (and avoid an unstable upgrade), and resolve all depends, currently, with stable mono packages 1.9dfsg. After the build depends are apt-get installed: $ apt-get source gtkglarea-sharp $ cd gtkglarea-sharp-0.0.17 $ make -f debian/rules $ ./configure --prefix=/usr $ make Main difference is to (re-)configure with csc, which the debian/rules sets. I think this will work, source wise, until gdkgl can be apt-get with gtk.
"Main difference is to (re-)configure with csc, which the debian/rules sets." My bad, I meant "without csc"
I've used this for awhile now with GtkGLArea#, as you can see from this screenshot: http://mono.dzonux.net/file/MonoVida-20090619a.png Being able to reparent() a GdkWindow ontop of a GtkGLArea works well and was the main feature needed to composite windows/layers together. There was no need to resort to a GL texture or compositing manager to create the effect. GtkWindows can be left detached or their GdkWindow can be reparent()'d as an overlay on the GtkGLArea.
*** Bug 624759 has been marked as a duplicate of this bug. ***
Created attachment 166867 [details] [review] A modified gdk/gdktypes.h that includes third coordinates (for 3D geometry) This is a modified gdk/gdktypes.h file that includes geometric 3D support. Look near the end of the file and you'll find the _GdkPoint, _GdkRectangle (now _GdkPrism), _GdkSegment, and _GdkSpan structs and, above, their matching typedefs modified to define widget geometry in 3D. This is what the bug that has been (falsely) marked a duplicate of this one is about.
(In reply to comment #59) > Created an attachment (id=166867) [details] [review] > A modified gdk/gdktypes.h that includes third coordinates (for 3D geometry) > > This is a modified gdk/gdktypes.h file that includes geometric 3D support. Look > near the end of the file and you'll find the _GdkPoint, _GdkRectangle (now > _GdkPrism), _GdkSegment, and _GdkSpan structs and, above, their matching > typedefs modified to define widget geometry in 3D. This is what the bug that > has been (falsely) marked a duplicate of this one is about. The 2D drawing API is being removed from GDK in favor of cairo, there's no way that that new 3D drawing API will be added. What this bug was always about was providing a shim to integrate GTK+ with OpenGL. [Not clear that that will happen, especially now that Clutter is in the mix, but it conceptually makes sense.]
Yes, Owen, but that 3D drawing API is what Bug #624759 was about. Apparently, Mr. Bassi didn't know the difference.
(In reply to comment #61) > Yes, Owen, but that 3D drawing API is what Bug #624759 was about. yes, and Owen is quite right: no 3D API will be added to GTK+, given the future removal of the 2D API. > Apparently, Mr. Bassi didn't know the difference. I know full well the difference, thank you very much; and you can drop your condescending tone: it doesn't belong to Bugzilla (nor anywhere else in the real world). your bug was non-descriptive and about adding a "3D widget drawing tool" - redirecting you here was perfectly legitimate, since the only way to support a 3D drawing API is to depend on another 3D drawing API. Cairo is not going to get 3D support, and since GTK+ won't have drawing primitives exposed as public API, the only way to get 3D drawing is by using OpenGL - or use Clutter by embedding it into a GTK+ application. finally, adding a set of data types without an actual implementation has no point (no pun intended); who is going to use them, or how? how do I implement a 3D widget with the current drawing API available in GTK+ (Cairo) which doesn't support 3D? why would I use GTK+ data types when 3D drawing APIs already provide me with those - and might even be optimized? in short: even if GTK+ gets the ability to handle GL natively, in terms of creating a GL contents on a GdkWindow, you still will need to use OpenGL yourself.
Sorry for being so vague and I really shouldn't have assumed that you didn't know the difference. I just signed the Code of Conduct and I know you, Emmanuele Bassi, are really just doing your job. However, yes, supposedly GtkGLExt is also being integrated into the core of GTK+ starting with either 3.0 or 3.2. Maybe that's what we will be using.
(In reply to comment #63) > However, yes, supposedly GtkGLExt is also being integrated into the core of > GTK+ starting with either 3.0 or 3.2. Maybe that's what we will be using. no, at the moment there are no plans of merging GtkGLExt in gtk+ - not for 3.0 for sure.
(In reply to comment #64) > no, at the moment there are no plans of merging GtkGLExt in gtk+ - not for 3.0 > for sure. http://live.gnome.org/ProjectRidley (From that page) >Target Libraries > > * libgnome (4/7) > * libgnomeui (4/7) > * libgnomeprint22 (2/2) > * libgnomeprintui22 (2/2) > * libglade (1/1) > * libgnomecanvas (0/1) > * libegg (4/9) > * libeel (0/1) > '''* gtkglext (0/1)''' > * libsexy (2/5) See the pattern?
Please stay on-topic and technical here. I could also post a list of kitchen sink modules to underline the urgent need of a kitchen sink included in gtk+.
Sorry to wake up an old bug, but are there any news about OpenGL integration?
Hi, in case anyone is interested: I was taking a look to the gtkglarea project and made some patches to the build system and also to upgrade the code as much as I can before a possible transition to GTK+3: https://git.gnome.org/browse/gtkglarea/ I also pushed a branch with a GTK+3 port: https://git.gnome.org/browse/gtkglarea/log/?h=jjardon/gtk3 everything seems to work but the zkor example, not sure yet why. Any help welcomed!
Hey Javier, I really appreciate all of the recent work you've put in to the project. There are a couple of concerns I've got, though. 1) I'd like to make sure the windows use case is exercised. Do you have a windows VM you can test this in? Perhaps I can get cygwin/mingw up and running on one of my systems and build+test. Can you tell me which git repo and branch you've pushed your changes to? 2) Since the zkor example does not work, can you give us a stack trace? cjac@foxtrot:/usr/src/deb/gtkgl2-2.0.1/examples$ strace .libs/lt-zktor Cheers, C.J.
(In reply to comment #69) > 2) Since the zkor example does not work, can you give us a stack trace? > > cjac@foxtrot:/usr/src/deb/gtkgl2-2.0.1/examples$ strace .libs/lt-zktor That's not a stack trace, that's a syscall trace. A stack trace you'd get with gdb.
Yes, thank you for the clarification Bastein. My apologies.
(In reply to comment #69) > 1) I'd like to make sure the windows use case is exercised. Do you have a > windows VM you can test this in? Perhaps I can get cygwin/mingw up and running > on one of my systems and build+test. Can you tell me which git repo and branch > you've pushed your changes to? I do not have a windows machines, sorry As commented before, the repo with the GTK+3 port is in a branch in the main gtkglarea repo: https://git.gnome.org/browse/gtkglarea/log/?h=jjardon/gtk3 > 2) Since the zkor example does not work, can you give us a stack trace? The example doesn't crash; the problem is that it does not run correctly (the elements move in a erratic way)
Created attachment 283077 [details] [review] build: Require libepoxy If we want to use OpenGL in GDK then we have two choices; either: - find the GL headers on each platform - do extension discovery - implement all the crazy dlopen()/dlsym() dispatch tables *or* use libepoxy, which shields us from all this madness and provides a decent layer for GL clients to use, without creating its own namespace. Epoxy is also used by other projects, like Xorg and piglit, and it's portable to all the platforms GDK cares about.
Created attachment 283078 [details] [review] Implement GdkGLContext and GdkGLPixelFormat GdkGLPixelFormat is an ancillary class to specify pixel formats, buffer types, and buffer sizes for GL context. GdkGLContext is the wrapper around platform-specific GL context API. This commit adds the base classes, and an implementation on X11.
Created attachment 283079 [details] [review] gtk: Add a GL drawing widget
Created attachment 283080 [details] [review] tests: Add simple GtkGLArea test
so: after only 6 years, here are some patches. the API is pretty self-explanatory: * GdkGLPixelFormat allows configuration and discovery of available support for GL-backed visuals * GdkGLContext is the wrapper class for platform-specific GL contexts the API is heavily modeled on the equivalent API on Quartz and Windows. I added a GtkGLArea widget which tries really hard to do everything as automatically as possible, dropping you into a "draw-with-context" signal that lets you just deal with GL calls. it's not finished, and not wholly documented, but I think it's a good starting point for discussions on the implementation. it's also unimplemented and untested on anything that is not X11; I can probably implement the MacOS backend, and test the Wayland one; for Windows, we can probably ask LRN or nacho to have a look.
(In reply to comment #74) > Created an attachment (id=283078) [details] [review] > Implement GdkGLContext and GdkGLPixelFormat > So here's some API review, as usual I don't look at any actual implementations first. - Is GdkGLPixelFormat immutable? Is it always valid? It seems to be but it's not immediately visible from the API. - What's the relationship between GdkDisplay and GdkPixelFormat? The constructor doesn't seem to take a display (or can it?), so it seems unnecessary to have one. It seems better to me to have an explicit display argument in the constructor to make the relationship clear. - Should there be getter APIs for GdkPixelFormat properties? This object reminds me of GtkTextTag and GtkSettings and I dislike those property bag object without real APIs a lot. Nobody knows what they do and they accumulate more and more cruft as time goes on... - Are pixel formats per-backend (or would it make sense to make them)? Just a generic question as they seem to be per-display and you might want to call specific getter API on them? - Do I keep around pixel formats? Are those objects considered expensive and I should cache them or do I just create one when I need it? I guess that goes into how expensive validation is?
(In reply to comment #75) > Created an attachment (id=283079) [details] [review] > gtk: Add a GL drawing widget - This widget has lifecycle issues I think. GdkGLFormat and GdkGLContext are bound to a GdkDisplay while the display used by a GtkWidget can change (also via a screen-changed signal emission). Binding a widget to a display only happens on ::realize afaics. Last but not least the code doesn't do anything to ensure that its display and the GL context's/format's display match. - What happens on error? This GL API has a lot of error situtations. Either they are so rare that we can ignore them in our public API (like we do with Cairo or X errors) or we need some visual feedback to the end user (like drawing the error message if rendering GL failed. - small nitpick: draw-with-context name I think I prefer draw-gl as this is all about GL not about context. Or we could take one of the many names for drawing - like "render" - and let that refer to "drawing with GL" in the GTK world.
I addressed parts of comment #78 and comment #79 - instead of re-attaching patches, I have a branch on github: https://github.com/ebassi/gtk/tree/wip/gdk-gl/ now, off to the other comments: (In reply to comment #78) > - Is GdkGLPixelFormat immutable? Is it always valid? > It seems to be but it's not immediately visible from the API. once a GdkGLPixelFormat is created, it's immutable from an application developer's perspective, but once it's validated, its fields are updated to contain the details (as much as we can) of the pixel format that was selected by the system; I recorded this in the API documentation. > - What's the relationship between GdkDisplay and GdkPixelFormat? > The constructor doesn't seem to take a display (or can it?), so it seems > unnecessary to have one. > It seems better to me to have an explicit display argument in the constructor > to make the relationship clear. a "pixel format" is similar to a GdkVisual. you need to create a GdkGLContext using a display connection and a pixel format. we cannot reuse GdkVisual because most of this stuff is platform-specific and GL-specific — plus, pixel formats are pretty much what people using OpenGL on Quartz and Windows expect them to be called: * https://developer.apple.com/library/mac/documentation/cocoa/reference/applicationkit/classes/NSOpenGLPixelFormat_Class/Reference/Reference.html * http://www.opengl.org/wiki/Creating_an_OpenGL_Context_%28WGL%29 > - Should there be getter APIs for GdkPixelFormat properties? > This object reminds me of GtkTextTag and GtkSettings and I dislike those > property bag object without real APIs a lot. Nobody knows what they do and they > accumulate more and more cruft as time goes on... the properties are described by the various platform-specific specifications, so we cannot really make them up as we go; the reason why I went with properties is because using C arrays of integers is pretty terrible from every language. we can definitely add getters for the properties, tho, since we do expect people to be able to query the validated GLPixelFormats. > - Are pixel formats per-backend (or would it make sense to make them)? > Just a generic question as they seem to be per-display and you might want to > call specific getter API on them? as I said above, the property name, ranges, and defaults are defined in the platform-specific specifications for GL implementations, but they are vaguely matching on all platforms. I don't think this warrants a platform-specific API or implementation, and if push come to shove we can have platform-specific properties that are just ignored on different platforms. > - Do I keep around pixel formats? > Are those objects considered expensive and I should cache them or do I just > create one when I need it? > I guess that goes into how expensive validation is? validation should not be expensive; you're supposed to keep a pixel format around if you plan to create a new GL context using the same configuration post-validation, otherwise you can just discard it. (In reply to comment #79) > - What happens on error? > This GL API has a lot of error situtations. Either they are so rare that we can > ignore them in our public API (like we do with Cairo or X errors) or we need > some visual feedback to the end user (like drawing the error message if > rendering GL failed. every GL call may fail, and you're supposed to call glGetError() to get the error condition after each call. obviously, this is *insanely* expensive on some platform, so it cannot be done in production without paying a hefty performance penalty. GL developers know this, so they already have code in place that handles the issue in their utility libraries. the only errors I can see are the platform-specific ones coming from creating contexts, setting the context current, and swapping buffers. for the first one, we have a GError that we bubble up. the other two we can handle ourselves exactly like we trap X11 errors.
Ok I looked at this and it seems like a good start to me, but it needs some work. First of all, I have two issues with on the intel driver on F21 here. With DRI3 it draws nothing at all, I have to run with LIBGL_DRI3_DISABLE=1. Secondly, there is some weird issue with resizes. Whenever i resize the toplevel and the GtkGlArea at the same time i get random parts of the glarea draw at the top left corner of the toplevel. Both these are likely driver issues, so lets ignore them now. However, it is weird that other GL users like clutter don't hit these issues. The boolean properties of GdkGLPixelFormat (double-buffer, stereo, multi-sample) need to change to a tri-state type so that you can properly express what format you want. I.e. must-be-double-buffered, must-be-single-buffered, or don't-care. It will probably be pretty common for a single toplevel to contain multiple GtkGLAreas (for instance if you have a 3d editor + a preview). The most efficient way to do this is to share the gl context and just point it at different windows. We should make this simple with the API. One possibility is to have gtk_gl_area_new_with_context() as a helper function to not have to write a create-context signal handler. gtk_gl_area_get_context() Will fail for unrealized widgets. Should we realize it automatically, or document this for the method? The viewport setup code doesn't handle scaled windows. gdk_gl_context_get_current() uses a per-display global cache, but the current context is really a per-thread property. I know threads and gl are a pain, and with gdk its even worse, but lets at least do the best we can here. The way we handle draw for gl windows is a bit weird. We'll be creating a cairo_t etc for the window which we will not use. Also, do we possibly want to handle partial region redraw for render? I guess one could extract the clip from the cairo_t, but thats a bit weird. Also, I think we could do much better on integration with the gdk drawing model. For instance, rather than each GtkGLArea doing a swapbuffer with a possible blocking in the middle off rendering all the other widgetry i think we should submit all gl drawing and do all cairo rendering fully, and then do the swapbuffers at the end, so gl and normal rendering is better synchronized. Also, it would be good if possible if we could actually use the normal frame clock for gl rendering too, rather than just for the cairo rendering. Ideally in the gnome-shell case we should just immediately swapbuffer to the window back buffer without waiting for a new frame, and then have the shell do the waiting for the frame to draw the result, rather than both the app and the compositor both trying to sync to the vblank.
Specifically commenting on using the normal frame clock for GL rendering. With DRI2 or DRI3+present, glXSwapBuffers() with swap interval disabled is basically just X rendering - when the X server receives the X request, it immediately a) sends a blit from the back buffer to the front buffer to the kernel b) sends damage events. In this model, there's nothing extra that has to be done to make frame sync work with GL rendering other than disabling the swap interval. The problem is that nothing defines things as working this way, and the NVIDIA drivers don't work this way - what happens is that the client does the copy from back to front buffer completely independently from X server rendering, and then at some subsequent point damage events are emitted. As far as I understand it, the point where the damage events are emitted is completely outside of the control of the application. So going ahead and using mutter-style frame synchronization with glXSwapBuffers and the NVIDIA drivers results in the following: - Mutter sees a frame counter increment with no damage, does fallback synchronization - Mutter sees damage events outside of the frame scope EXT_x11_sync_object doesn't help this situation - it only defines a way for GL rendering to block on X rendering, not vice versa ("This extension stops short of allowing the GL to change the state of imported/reusable sync objects, but does not add any language that would prohibit such functionality from being added in a subsequent extension.") I think it *should* work if you manually redirect the GLX window, glXWaitGL() after drawing and then use X to copy from the subwindow to the parent window. The disadvantages I see: * To avoid a double copy, you would ideally do front buffer drawing to the GLX window, but I think that might trigger badness with DRI. * When the compositor unredirects, we really would like to go back to glXSwapBuffers and let the driver have free run to do magic. But double-buffered or not is part of the fbconfig. But the approach could be worth experimenting with. The idea GLX extension would probably be a glXSwapBuffers variant that emitted damage events immediately and triggered an imported sync fence object so it could be waited upon by X code (or imported back into another GL client to use with glWaitSync.)
owen: Can't you just draw with gl on a pixmap, rather than a redirected glx window? And the paint that.
(In reply to comment #83) > owen: Can't you just draw with gl on a pixmap, rather than a redirected glx > window? And the paint that. in theory, yes; but there's no guarantee that the system will have a visual matching a pixmap, or that the drawing on the pixmap is going to be hardware accelerated (and most likely it isn't).
also: you really want to draw on a GLX pixmap on the GPU and then blend it over in system memory with cairo, and then pass it to the X server so that it passes it over to the GPU again? that would be the most inefficient pipeline ever.
Why would it blend in system memory? Would not a blend via cairo be an XRender operation that blends from GPU to GPU? (Obviously assuming the default xlib cairo backend here.)
(In reply to comment #86) > Why would it blend in system memory? Would not a blend via cairo be an XRender > operation that blends from GPU to GPU? (Obviously assuming the default xlib > cairo backend here.) because XRender is not implemented in terms of GL commands, and the contents of the Pixmap will still need to end up copied on the GPU anyway. it *may* work to work if and when Glamor will land, but I wouldn't hold my breath on either of those conditions. frankly, I think you're overcomplicating what it's actually pretty straightforward. yes, there's no full synchronization for child windows; the drawing still happens when the tool kit says we ought to be drawing, but the contents are presented when the GL machinery thinks they should be presented — and the compositor adds another layer on top of that. I'd be much more worried about ensuring that Wayland surfaces can be used to draw subregions with GL; if that is not possible, it means we need to create an EGL context on the top-level, and then multiplex it so that subregions of a Wayland surface gets drawn with the correct, user-selected context. otherwise, we need to go to the FBO route, and always draw on an off screen frame buffer, and blend the result on the top-level — which is actually my plan anyway, because that's needed for GSK.
What worries me is that we now have an excellent animation framework including a frame clock and synchronization with the compositor and detailed frame time reporting. But with the opengl rendering path, which in some sense is the future, we completely bypass this and add an extra frame of latency and source of jitter. > because XRender is not implemented in terms of GL commands, and the contents of > the Pixmap will still need to end up copied on the GPU anyway. I'm not sure exactly how this differs from the case with the current code: We have a toplevel XWindow, which is redirected to a XPixmap. Whenever we update the Pixmap the compositor recieves DAMAGE events and uses texture_from_pixmap to draw the new Pixmap data on the screen at the next vblank (and tells the client when this happened). The toplevel has a child window, which has a glXWindow wrapper. This essentially means that it allocates a separate pixmap for the back-buffer, which all gl drawing happens in. After drawing the gl commands we do glXSwapBuffer, which essentially queues a XCopyArea from the back buffer pixmap to the redirected toplevel pixmap. The problem with this is that the gl rendering will happen at the same time as the non-gl rendering for the rest of the toplevel. So, at the time the frame clock deems it is time to produce a new frame we will allocate a double buffer pixmap, paint to it using XRender, then in the middle do some GL rendering to the back buffer and a SwapBuffer . Then XCopyArea the double buffer pixmap to the toplevel, causing damage events to be sent to the xserver which triggers it to schedule a repaint the next frame. However, at some later time (maybe after the compositor repainted for the regular windows) it will get a new damage event from the swap-buffer copy and cause a new update with the gl changes only. Thus we will see out-of-sync subwindow updates, added latency for the gl rendering and possible jitter due to rendering twice per frame. I guess if we just disable the swap interval then we're fine on any non-nvidia driver (as only nvidia delays the swapbuffer damage events as per owen). Maybe that is good enough. (Of course, we have to detect the no-compsitor or unredirected case and use swap interval then.) For wayland I don't think there is a huge problem. Here is how I imagine rendering working using wayland+gl: The toplevel buffer we hand over to the wayland server is an EGL buffer, which we also map as a cairo image surface. First we draw the widgets, where regular widgets draw to the image surface, and gl widgets render to a per-gl-widget FBObject. Then we sync the sofware drawing with the gl machinery as needed and draw (using opengl) each of the fbobjs onto the toplevel buffer. Then we hand the finished buffer over to wayland (with whatever fences needed for wayland to not draw until all the fbobj copies are done). This means we get an extra offscreen buffer per gl widget. But that is not really any different from what we do on X, where there is a backbuffer that we draw into that gets copied onto the front buffer. Another option is to draw the regular widgets to a separate cairo image surface and draw that using some gl call rather than rendering directly into the gl buffer. I don't know what performs best there... I don't think using subsurfaces for wayland GL is really much better than a FBO. You still need two buffers (wasted space in the toplevel + the subsurface buffer), and there are all sorts of complexity to handle with the subsurface handling (for instance, scrolling and clipping). At best subsurfaces you may save a copy, if the compositor is smart enough to avoid copying the region of the normal surface below the subsurface. And with subsurfaces there is no chance of having a more complex integration with the parent like an RGBA blend.
> I guess if we just disable the swap interval then we're fine on any non-nvidia > driver (as only nvidia delays the swapbuffer damage events as per owen). I guess this means that we should avoid exposing swap-interval as a public API as the backend code actually sometimes want to use it (when unredirected) and sometimes not (composited). We just always make sure in the gdk backend that drawing is always double buffered and synced to some form of frame clock or vsync.
I implemented "don't use swap-interval on composted WMs: https://github.com/ebassi/gtk/pull/1 (It also has an unrelated minor fix) Also, i fixed a bug in gdk that made us redraw the GtkGLArea to often: https://git.gnome.org/browse/gtk+/commit/?id=6243c7122cf1a315c9b3b9ab71fad4d93f2d5275 ebassi, can you rebase your branch on latest to get that fix?
Do we have something like gdkglxgears? Or rather: Do we have a way to collect meaningful numbers from the GL stuff to compare with other rendering mechanisms (like native apps)?
As you wished for, i added gdkgears to the pull request above: https://github.com/alexlarsson/gtk/commit/a4f9d20ae1fc1040ba1481f02560697c38856ede It features the standard glxgears stuff, plus integration with Gtk+ side animations (a spinner), and the possibility of using multiple GLAreas in the same window. However, the multiple GLAreas doesn't seem to work for me, once i add a new gears (the "Moar gears!" button) the old ones stop updating. I don't know if this is a driver bug here or if we're doing something wrong.
Also, note how the GtkGears code need to hook into realize to initialize the gl context, and size_allocate to reshape the view matrix. Perhaps we want an API that makes this easier?
Is it possible to stack "normal" GtkWidget on top of the GTK GL widget? I'm thinking GtkRevealer or even GtkRevealer with non opaque widgets.
Right now we're using a plain native window for the GL rendering, which means overlapping opaque widgets should work, but non-opaque will not work. Doing non-opaque overlapping widgets is tricky as this means combining gl drawing with cairo drawing. Basically we'd need the reverse of texture-from-pixmap, a "XCopyArea-from-FBO" kind of operation, for this to work efficiently.
I already talked about it on IRC: I wouldn't mind if we used a completely different drawing model for toplevels where OpenGL is enabled, ie one where we do all GtkWidget::draw onto an image surface, upload that to a texture and render it on top of GL later. To me the two paramount concerns are: (1) features Bastien's question in comment 94 is an example of that. Not allowing transparent overlays is a pretty huge limitation in my book for example. Using native subwindows inherently has loads of limitations, so I'm not a fan of those at all. (2) performance of the GL drawing In a GL enabled app, the GL part should ideally draw as fast as a native GL app. I'm not much concerned about any non-GL rendering, because I'm pretty sure that software rendering + upload to texture as a worst case is good enough, because that's what Wayland does.
Benjamin: Unfortunately doing ::draw to an image backend and then blasting it and gl FBOs to the toplevel using openGL doesn't really solve bastiens problem. If we want to be able to combine the GL content with other content we need to: a) be able to composite the gl output on top of the previously drawn cairo surface b) be able to draw on top the output from a) using cairo. This is not really possible even if we use opengl only to draw to the toplevel (and cairo image surfaces), as we can't import the gl buffer into the cairo buffer. I guess we can do things in a layered fashion, where we render everything below the gl window to a cairo surface, then render the gl widget to a gl FBO, then draw anything above the gl widget to a separate cairo surface, and then combine them in the end using opengl. That can only achieve OVER compositing though, although I don't think we ever do anything else currently.
(In reply to comment #97) > Benjamin: Unfortunately doing ::draw to an image backend and then blasting it > and gl FBOs to the toplevel using openGL doesn't really solve bastiens problem. > > If we want to be able to combine the GL content with other content we need to: > a) be able to composite the gl output on top of the previously drawn cairo > surface > b) be able to draw on top the output from a) using cairo. that's what I'm working on separately, because GSK will use the same model: * draw on image surfaces using cairo * blend each image surface as a GL texture on a GL frame buffer and if we have a GL layer, we draw on an FBO and then blend the backing texture like the above. > This is not really possible even if we use opengl only to draw to the toplevel > (and cairo image surfaces), as we can't import the gl buffer into the cairo > buffer. we *could* read back the GL buffer, but it's slow and I'd really avoid that like the plague. > I guess we can do things in a layered fashion, where we render everything below > the gl window to a cairo surface, then render the gl widget to a gl FBO, then > draw anything above the gl widget to a separate cairo surface, and then combine > them in the end using opengl. That can only achieve OVER compositing though, > although I don't think we ever do anything else currently. we can change the glBlendFunc and parameters to change the blending function, and if we really want to get fancy we could nab some of the GLSL shaders from cairo-gl to implement other operators.
ebassi: I don't even think it has to be image surfaces. Can't we use X surfaces that are drawn on the window with texture-from-pixmap? > we can change the glBlendFunc and parameters to change the blending function, > and if we really want to get fancy we could nab some of the GLSL shaders from > cairo-gl to implement other operators. Sure we can extend this a bit, but we're fundamentally limited in what is essentially a "combine layers" operation. Like, current cairo implementation of ::draw() does a full recursive draw using cairo. It could do a push/pop around the child widget rendering and rotate the result. Or it could call it twice with some transformation applied to create e.g. mirroring effects. Or it could call the child ::draw with a completely different cairo_t, redirecting to an offscreen which is used to draw to the screen later. This last part is used for the pixel cache, which is going to run into issues if the child is doing GL rendering. This is something we have to look into and solve properly, because we do want efficient scrolling. Otoh, the layer approach we're talking about above is very similar to pixelcache itself... Another aspect of opengl integration is how we expect it to work if you call gtk_widget_draw() on a non-standard cairo_t, to e.g. get offscreen rendering or pdf output.
(In reply to comment #99) > ebassi: I don't even think it has to be image surfaces. Can't we use X surfaces > that are drawn on the window with texture-from-pixmap? we can, but then we need to defer a lot of this stuff to a per-platform implementation, because texture-from-pixmap only works on GLX. EGL has some extension for conversion between EGLImage and Wayland surface, if I remember correctly, which could be used as a way to approximate texture_from_pixmap, but I fully expect an images surface to be used on every platform that is not X11, so I'd ensure that we had a default implementation with it.
(In reply to comment #99) > Sure we can extend this a bit, but we're fundamentally limited in what is > essentially a "combine layers" operation. Like, current cairo implementation of > ::draw() does a full recursive draw using cairo. It could do a push/pop around > the child widget rendering and rotate the result. Or it could call it twice > with some transformation applied to create e.g. mirroring effects. > This goes back to my two points from comment 96 I think: It would be great if it's possible but I don't mind if it's slow. I don't even mind if it's below 10fps. And I think we can get the "possible" with glGetPixels().
(In reply to comment #101) > (In reply to comment #99) > > Sure we can extend this a bit, but we're fundamentally limited in what is > > essentially a "combine layers" operation. Like, current cairo implementation of > > ::draw() does a full recursive draw using cairo. It could do a push/pop around > > the child widget rendering and rotate the result. Or it could call it twice > > with some transformation applied to create e.g. mirroring effects. > > > This goes back to my two points from comment 96 I think: It would be great if > it's possible but I don't mind if it's slow. I don't even mind if it's below > 10fps. > > And I think we can get the "possible" with glGetPixels(). I'd keep the glReadPixels() approach only for *very* special cases, like trying to draw a GtkGLArea with a cairo_t coming from a PDF surface; glReadPixels() will *never* be fast, and having stuff rendering at 10 FPS in a common case is never acceptable.
> And I think we can get the "possible" with glGetPixels(). Yes, it is always possible to get it done slowly. The hard part is detecting when we can use the "fast" implementation. We can do similart tracking to what we did for XCopyArea optimizations, i.e. keep track of when things overlap, and when things have a solid background (although detecting solid background relies on the gl code respecting css bg properties which is a bit weird, maybe we need an RGB vs RGBA option for GtkGLArea). But even that only detects when we need to do layering tricks, it cannot detect when the parent ::draw code is doing something complicated like manually applying an overall alpha or redirecting output to another surface. Needs more thought.
Why is it hard to detect if somebody does something stupid? We set up the cairo_t ourselves and can check that it's the one we care about. Or said differently: We have total control over the arguments used in drawing code and we can attach the information we need to it. And it should be noted that your examples pretty much do not matter at all. There is not a single GTK GL application in existence that does anything close to that (because there is no GTK GL application). And even when looking at the non-GL application, nobody does anything like that apart from demos in the GTK source code and some applications that use ugly hacks because they don't know what they're doing. In short: I'm not worried.
At the time you render the glarea, how do you know the parent called cairo_push_group() and will pop after you're done to do something weird?
An example here is the push/pop group in the crossfade of GtkStack. This particular code is in Gtk+, so we can special case it, but the same could be in a third party widget.
I rebased ebassis branch to gtk 3.14 and made some further experimental changes here: https://github.com/alexlarsson/gtk/tree/wip/gdk-gl Apart from adding a gdkgears test the changes are related to how things are drawn. When we draw a window we start by creating a gl context for it (the paint context) and clear the back buffer in the dirty region. We then emit the draw as usual painting to the double buffer cairo surface (typically an xlib surface), although we ensure that it always is a color+alpha surface. Then at the end we blend the cairo surface to the back buffer using opengl, and blit the dirty regions from the back buffer to the front buffers. The blend uses texture-from-pixbuf for the xlib case and falls back to a texture from cairo image surface otherwise. Additionally, in gtkglarea we *always* allocate a render buffer and draw to that. Then we use the new gdk_cairo_draw_gl_render_buffer() to draw (part of) the render buffer to the cairo_t. By default this will read the pixels from the render buffer and blend them on the cairo surface. However, in the normal case where we're drawing to the real GdkWindow with no fancy cairo effects we hit a fast path that uses GL to draw the render buffer directly to the back buffer and clear the corresponding area of the double buffer surface to transparent. This means in the normal case everything is very fast. Note that this also requires the gl context for GtkGLArea to share with the paint context for the native window for the widget, which requires us to expose this publically. There are some optimizations that could be done. For instance, in the case where we clear the double buffer and don't paint any more on top of that area we could avoid blending that area. This requires us to track what areas are drawn on the double buffer though, which is a bit of work. Also, there is a fix for gdk_x11_display_make_gl_context_current() where it bailed early if "drawable == context_x11->current_drawable". This is wrong since it could easily be the same drawable as before, only we switch which context is the global current. I just neutered the check, but in general i think this shows that the separating of "make current" and "make draw in this window" into separate calls may be a poor API. There is some bug that causes a lack of repainting when resizing windows. I need to look into that.
the approach looks good to me; I'm going to have a better look at the API, and fix the use or texture-rectangle to be a fall back path for older GPUs (newer GPUs should always use non-power-of-two textures). since the approach is to have at least one context for top-level and separate contexts for GL areas, we'll probably need to check and use the GLX_ARB_flush_control extension: http://lists.freedesktop.org/archives/mesa-dev/2014-October/068914.html this improves the performance when using more than one GLX context.
I updated the branch with new commits that optimize a bunch of cases to avoid doing unnecessary blends and clears.
New version, which includes full support for wayland.
I recreated a new nicer series of this based on master at: https://git.gnome.org/browse/gtk+/log/?h=wip/gdk-gl2 I think this is in a decent state for merging, if people agree with the general principles. My current list of outstanding core TODOs is: * Use TEXTURE_2D instead of GL_TEXTURE_RECTANGLE_ARB if non-power-of-2 textures are supported. * Make sure that the GL frame sync code used for non-composited WMs work. I think there is some issue with it, but i have not really tested it. * Detect the case where we're fullscreen-unredirected in a composited WM and switch to using frame-sync via GL. * Currently clients get a GL context that can be used to draw to the window back buffer, but are not allowed to. Maybe we can instead give them a gl context for some dummy window with the same visual to enforce this. * Think about how our current tracking of the current gl context should work, as the current code is not quite right. The gl context is really per-thread, but we treat it as display-global. There are also some GtkGLArea API issues: * Its impossible atm to add extra aux buffers, or to negotiate the format of the framebuffer and depth buffer in more details. We should probably have some vfuncs for this. * Right now we allocate a new framebuffer on each draw. We may want to allow apps to keep it around between redraws if redrawing is costly. * We may want to pass clipping information to render, so that apps can optionally only render the part that needs repainting in the window.
(In reply to comment #111) > I recreated a new nicer series of this based on master at: > https://git.gnome.org/browse/gtk+/log/?h=wip/gdk-gl2 > > I think this is in a decent state for merging, if people agree with the general > principles. first of all, thanks: it looks really awesome, and it makes my job for integrating GSK about a thousand times easier. kudos. > My current list of outstanding core TODOs is: > > * Use TEXTURE_2D instead of GL_TEXTURE_RECTANGLE_ARB if non-power-of-2 textures > are supported. rectangle textures are mostly fine if we're just drawing cairo surfaces coming from damaged regions of windows: we're not going to use user-provided shader samplers on them, and we don't care about mipmapping. things change a bit if we make the "turn this cairo surface into a GL texture" a semi-public API (like I intend to for GSK), since cairo surfaces will be rendered with custom modelview matrices, which may also include scaling and custom minification/magnification filters, or fragment shading snippets. for those we definitely need TEXTURE_2D. > * Make sure that the GL frame sync code used for non-composited WMs work. I > think there is some issue with it, but i have not really tested it. the code is an adaptation of what Clutter does, but it's also true that we don't really exercise that code path much, except probably in the shell itself. > * Detect the case where we're fullscreen-unredirected in a composited WM and > switch to using frame-sync via GL. it's interesting to note that on MacOS you're supposed to use a NSOpenGLPixelFormat with a "give me a GL context for a full screen rendering" hint for precisely that case. game developers (the intended target audience) are supposed to create a pixel format/GL context for the full screen case, and switch to it if they are full screen. > * Think about how our current tracking of the current gl context should work, > as the current code is not quite right. The gl context is really per-thread, > but we treat it as display-global. the main issue is that we don't really allow people to use different threads with GTK+ at all, so I'm a bit wary about having a "per-thread" guarantee when the totality of our exposed API is definitely not "per-thread" but "single threaded with guaranteed Exploding in Your Face™ effect if you use it from different threads". > There are also some GtkGLArea API issues: > > * Its impossible atm to add extra aux buffers, or to negotiate the format > of the framebuffer and depth buffer in more details. We should probably > have some vfuncs for this. yup. also, we create a GL context with the default profile, but I can definitely foresee developers using a core 3.2 profile in the future. > * Right now we allocate a new framebuffer on each draw. We may want to > allow apps to keep it around between redraws if redrawing is costly. we definitely want to recreate the FBO only if the visual format or size of the rendering buffer changes; it nothing changed, we should reuse the same FBO if we want to draw new content. if nothing changed in the content, we really want to keep reusing the FBOs backing texture. > * We may want to pass clipping information to render, so that apps can > optionally only render the part that needs repainting in the window. a way to query the current clipping state would be good indeed; probably similarly to how devs can query the clipped region out of the cairo_t passed to the ::draw vfunc.
(In reply to comment #112) > (In reply to comment #111) > > * Detect the case where we're fullscreen-unredirected in a composited WM and > > switch to using frame-sync via GL. > > it's interesting to note that on MacOS you're supposed to use a > NSOpenGLPixelFormat with a "give me a GL context for a full screen rendering" > hint for precisely that case. game developers (the intended target audience) > are supposed to create a pixel format/GL context for the full screen case, and > switch to it if they are full screen. > Are there any important pros or cons with either approach? (hint: somebody please implement a GtkWindow::fullscreen property) > the main issue is that we don't really allow people to use different threads > with GTK+ at all, so I'm a bit wary about having a "per-thread" guarantee when > the totality of our exposed API is definitely not "per-thread" but "single > threaded with guaranteed Exploding in Your Face™ effect if you use it from > different threads". > Yes, all GDK displays uniquely belong to a single thread. So everything that hangs off them (like GL) would be the same. Though I guess for shared contexts different rules should apply so that we can do out-of-thread rendering later? > > * We may want to pass clipping information to render, so that apps can > > optionally only render the part that needs repainting in the window. > > a way to query the current clipping state would be good indeed; probably > similarly to how devs can query the clipped region out of the cairo_t passed to > the ::draw vfunc. It is kind of important to me that I don't know any widget that uses more sophistication than the clipping rectangle for optimization. So keeping a region (or worse: the spans like cairo_copy_clip_rectangle_list() is counter-productive. Because all the widgets I know that use these for drawing got slower for it (due to expensive per-draw setup costs) and I could speed them all up by using gdk_cairo_get_clip_rectangle().
(In reply to comment #113) > (In reply to comment #112) > > (In reply to comment #111) > > > * Detect the case where we're fullscreen-unredirected in a composited WM and > > > switch to using frame-sync via GL. > > > > it's interesting to note that on MacOS you're supposed to use a > > NSOpenGLPixelFormat with a "give me a GL context for a full screen rendering" > > hint for precisely that case. game developers (the intended target audience) > > are supposed to create a pixel format/GL context for the full screen case, and > > switch to it if they are full screen. > > > Are there any important pros or cons with either approach? not really, but it kind of depends on the API we expose. if we're re-using GdkVisual, then we'd need a visual for full screen windows that we can set on a GtkWidget using GL (e.g. for a game, or a video player). that would work for custom widgets, though, and not GtkGLArea, since we don't allow using custom GL contexts with it. the solution would be for GtkGLArea to detect whether or not it's a) inside a full screen window and b) its size is the size of the screen, in which case it can use a frame-sync rendering and the compositor can unredirect the window and improve performance. > (hint: somebody please implement a GtkWindow::fullscreen property) > > > the main issue is that we don't really allow people to use different threads > > with GTK+ at all, so I'm a bit wary about having a "per-thread" guarantee when > > the totality of our exposed API is definitely not "per-thread" but "single > > threaded with guaranteed Exploding in Your Face™ effect if you use it from > > different threads". > > > Yes, all GDK displays uniquely belong to a single thread. So everything that > hangs off them (like GL) would be the same. > Though I guess for shared contexts different rules should apply so that we can > do out-of-thread rendering later? off-the-main-thread rendering would be interesting, but it would happen only for custom GL rendering — i.e. the GtkGLArea::render, for instance. since that happens inside the GtkGLArea::draw implementation, I don't think we can easily move it off the main thread. another option would be to move compositing off the main thread — i.e. the internal path that GDK uses to blit the textures coming from cairo surface. still, that would probably require some major re-factoring, and new constraints on which parts of the GDK state we can access from different threads.
For the multi threaded case. In general, opengl works best in single threaded mode, as the context switches otherwise just make it slow. However, if you look around the web there is one case where using threads with opengl is important, and that is to use a separate thread to upload texture data asynchronously to opegl, for instance here: http://stackoverflow.com/questions/8116305/opengl-secondary-thread-for-loading-resources It would be nice if this was possible with GdkGL. You would create two shared contexts on the main thread, use one on the main thread to draw, but then make the other one current on another thread and use it to stream textures in. This is not currently possible, because when we make_current on the other thread we change the global "current context" on the display.
> It is kind of important to me that I don't know any widget that uses more > sophistication than the clipping rectangle for optimization. While this is true for any particular widget we do gain overal from it, because we never even expose widgets that don't lie in the current update region. This matters a lot if two small widgets update in different areas of the window.
>> * Detect the case where we're fullscreen-unredirected in a composited WM and >> switch to using frame-sync via GL. > >it's interesting to note that on MacOS you're supposed to use a >NSOpenGLPixelFormat with a "give me a GL context for a full screen rendering" >hint for precisely that case. game developers (the intended target audience) >are supposed to create a pixel format/GL context for the full screen case, and >switch to it if they are full screen. Well, there are two things here. 1) If we're in gnome-shell, but the shell unredirects us because we're fullscreen then the gdk paint loop needs to switch to a frame-sync:ed swapbuffer because it can't rely on the compositor to do the frame-syncing. This case is true even if we're mostly using cairo and only a small gl widget. 2) If we're really a game, and are rendering only opengl to the entire window, then the setup we have with a framebuffer that gets copied to the back buffer is not ideal, and one would need some kind of api that gives you complete back buffer setup and drawing access. However, at that point, do you really want to be using Gtk (at least for that window)? >> * Right now we allocate a new framebuffer on each draw. We may want to >> allow apps to keep it around between redraws if redrawing is costly. > >we definitely want to recreate the FBO only if the visual format or size of the >rendering buffer changes; it nothing changed, we should reuse the same FBO if >we want to draw new content. if nothing changed in the content, we really want >to keep reusing the FBOs backing texture. I'm not sure this is always best. Sure, in the case of an animating GtkGLArea in the currently active window, yeah. But if there are a lot of gl areas here and there that don't repaint all the time (maybe they are not even visible), do we really need all those buffers allocated? It seems to me that we may well want to opportunistically drop them when idle. Also, and related, if we're keeping the buffer around inbetween exposes, then we may also want a separate tracking of "dirtiness". A GtkGLArea could be drawn either because we need the toplevel back buffer to be up to date, or because the *content* of the gl rendering changed and we want to display that. In the second case we need to re-render the gl buffer, but not in the first case. Maybe we should have code in GtkGLArea that makes such invalidation tracking easy?
(In reply to comment #117) > I'm not sure this is always best. Sure, in the case of an animating GtkGLArea > in the currently active window, yeah. But if there are a lot of gl areas here > and there that don't repaint all the time (maybe they are not even visible), do > we really need all those buffers allocated? It seems to me that we may well > want to opportunistically drop them when idle. > So this is a question of creating FBOs - every draw event - on map() - on realize() or are you thinking about even other ways? > Also, and related, if we're keeping the buffer around inbetween exposes, then > we may also want a separate tracking of "dirtiness". A GtkGLArea could be drawn > either because we need the toplevel back buffer to be up to date, or because > the *content* of the gl rendering changed and we want to display that. In the > second case we need to re-render the gl buffer, but not in the first case. > Maybe we should have code in GtkGLArea that makes such invalidation tracking > easy? We're getting pretty close to replacing GdkWindows with FBOs here, aren't we? Or is that just because GtkGLArea is conceptually similar to widgets that embed external functionality (like GtkSocket)?
>So this is a question of creating FBOs >- every draw event >- on map() >- on realize() >or are you thinking about even other ways? currently gtkglarea allocates the framebuffer object on realize(), and allocates the renderbuffer storage each draw(). Another alternative would be to allocate storage in map() and then resize it in size_allocate. The other alternative is to allocate it in draw() the first time, and then keep it around until either unmap, or the widget is not rendered in some amount of time. > We're getting pretty close to replacing GdkWindows with FBOs here, aren't we? > Or is that just because GtkGLArea is conceptually similar to widgets that embed > external functionality (like GtkSocket)? Well, we're more or less using FBOs to do the rendering side of native windows. I.e. have unrelated pieces of code coexist in one common rendering.
the branch has been merged
Created attachment 289082 [details] copy of gtkglarea-2.0.0a.tar.gz as mentioned in the thread