GNOME Bugzilla – Bug 692856
Regain Visual C++ support via resurrecting distutils build system
Last modified: 2017-12-06 04:04:43 UTC
Hi, When we were approaching 3.0, the distutils build system was removed from the PyGObject package as we are now able to build PyGObject on/for Windows using GCC/MinGW using the normal autotools-based build system. Since support for using G-I on Windows (using Visual C++) was officially added to its master branch since 1.35.2, I thought it might be a good idea that we get the distutils-based build system back for PyGObject, especially that there is the claim that MSVC is also supported for building/using PyGObject[1]. This is a series of patches that I have to resurrect the distutils-based build system-please note that this is meant only for Visual C++ builds as Dieter (dieterv) mentioned when he removed the distutils-based build system from PyGObject[2]. [1]: Please see the README file in the srcroot directory. [2]:http://git.gnome.org/browse/pygobject/commit/?id=8d3125c8ce9890c70400dd8a3ac273b590fe6a31, (Remove distutils based build system.)
Created attachment 234797 [details] [review] Remove GCCism from gi/pygobject-external.h First up is the patch to get rid of GCCism that went into gi/pygobject-external.h. We could make use of G_GNUC_UNUSED in place of that, which serves the same purpose on GCC, as the GLib headers should already been included before including that header (due to the existing use of gboolean).
Created attachment 234799 [details] [review] Remove C99ism by defining variables toward the top of their respective blocks This moves variable declarations towards the top of their respective blocks, so that C89 compilers such as MSVC would not complain.
Created attachment 234800 [details] [review] Deal with empty array declarations with no length specified This deals with another C99ism as C89 requires empty arrays to have lengths specified for them during declaration. There might be better ways to deal with this, but this is what I have which worked for me (TM) :)
Created attachment 234801 [details] [review] Add implementation of NAN and INFINITY Again, Visual C++ does not have NAN nor INFINITY defined, which was introduced very recently into the code, so we define them here for its use. I know this is not very elegant :)
Created attachment 234802 [details] [review] The distutils setup.py and associated items The last part of the patches I have at the moment is the setup.py and related items for the distutils build. This will build the gi/_glib, gi/_gobject, gi and _gi_cairo (_gi_cairo if applicable) with the distutils build approach. This will enable PyGObject be built on Windows in both x86 and x64 flavors using Visual C++. This is adapted from the distutils build system from 2.28.6, but since I am not that good in Python, there might be areas where this could be improved upon (although again, this worked for me :) (TM)) Please note that in order to use this on Python 3.x, one needs to run this through 2to3 which is included with a standard Python installation. I will come up with a README.win32 which will attempt to walk people through the process of building PyGOBject-3.x on Windows using Visual C++. Thanks for the time, with blessings!
Comment on attachment 234797 [details] [review] Remove GCCism from gi/pygobject-external.h Thanks for those! It feels like a step backwards, but I committed the first two patches as they are quite harmless.
Comment on attachment 234800 [details] [review] Deal with empty array declarations with no length specified A declaration like static PyMethodDef _PyGIRepository_methods[9]; seems outright dangerous, as it's surely forgotten the next time the array is updated. However, we do not need these forward declarations in the first place, so I slightly modified the patch (and its changelog): http://git.gnome.org/browse/pygobject/commit/?id=9e0c415
Comment on attachment 234801 [details] [review] Add implementation of NAN and INFINITY This is outright ugly -- Visual C must have _some_ less hackish way to represent NAN and INFINITY? These constants aren't exactly new, after all?
Comment on attachment 234802 [details] [review] The distutils setup.py and associated items That was already discussed in bug 680138 and declined. It's really not maintainable to have two different build systems; the non-default one will just keep breaking all the time. If we do this, then we need to do it all the way through and only use distutils. But this would then break compatibility as a GNOME module and jhbuild etc. But if we have a working distutils, this could be rectified with having a small static Makefile which just calls $PYTHON setup.py <whatever>.
(In reply to comment #9) > (From update of attachment 234802 [details] [review]) > It's really not maintainable to have two different build systems; the > non-default one will just keep breaking all the time. Hi Martin, I'd agree on this part, as sources are added/removed from time to time, which is the main reason why these static build files would get into trouble. Would it be better for you if I feed in the sources to compile from the autotools files into the distutils files during 'make dist', such as what is being done for the MSVC project files in the GTK+ stack? Otherwise, I might need to come up with an universal distutils build system as you mentioned, and I would most probably need help for testing them on various platforms that PyGObject supports, as I don't have access to all of them. (In reply to comment #8) > This is outright ugly -- Visual C must have _some_ less hackish way to > represent NAN and INFINITY? These constants aren't exactly new, after all? I have to agree with you that it is very ugly-but that's life on that compiler. I will try to see whether there are nicer ways to represent those constants on that platform. The kdewin project, for example, also tries to emulate these macros through hacks as well-I would want blame Microsoft for this myself. :| Thanks for your time for the reviews, feedbacks; and for hearing my rants and thoughts on this:) With blessings.
Created attachment 234952 [details] [review] 2nd take on resurrecting the distutils-based build files Hi, I was looking through the bug post that Martin posted and it does seem to me that there is quite a bit of reluctance to forgo autotools for building PyGObject. I thought, then, that it might be a good idea to make the distutils make advantage of the autotools files as a result. It seems that by Python we could avoid updating the autotools files (as I mentioned in my previous comment) to make sure the file list of sources to build and the scripts to install are up-to-date. So as a result, I made use of some regex items from setup.py (under build/win32) from the GLib package, which allowed me to read from the autotools files. This means that we can have an up-to-date C-source list to build for all the submodules, versioning and dependency versioning info and list of Python scripts to package/install for all the submodules as long as the autotools files are up-to-date about these items. This should help to greatly reduce the odds of the distutils build files breaking as we move forward, without adding stuff into the autotools files, except of course to dist the distutils build files (setup.py and the utility dsextras_pygobject3.py). New sub-modules to build/install can be added without much difficulty here, which I can voluteer to do if that is needed (as I currently do for the MSVC build files for the Clutter/GTK+ stack). Hope this is better for an interim solution before (and if we) we take the bite to go to a completely disutils-based build approach. With blessings, thank you!
Good to see the return of my friend dsextras ;-) I will try a mingw build with this sometime. We should also CC dieterv on this bug...
(In reply to comment #12) > Good to see the return of my friend dsextras ;-) > > I will try a mingw build with this sometime. We should also CC dieterv on this > bug... Am already here :P MinGW would have no need for these as the necessary bits to build with the standard autotools have been in master for quite a while (back when J5 was still maintaining PyGObject, before Martin took over). I do remember having a couple ideas to further improve our python.m4 situation re proper automatic python interpreter detection on win32 but have lacked the time to play much with it :( Haven't built fresh binaries for a couple of months though so all this might have regressed a bit left and right? The way I see it, distutils support should go back to be msvc only (like how it was when originally written) and thus it might be better to put everything (setup.py, dsextras.py, etc) into a build/win32 directory as is done in glib, gtk+, etc to prevent people getting the wrong ideas? (In reply to comment #9) > That was already discussed in bug 680138 and declined. If I understand this correctly, this is not asking for generic cross platform distutils support (that would be somewhat crazy, autotools gives us so much more than distutils is capable of) so treating this like bug #680138 is a bit unfair. This is about putting new life in supporting msvc which we haven't had since Cedric Gustin stopped maintaining this for PyGTK way back around 2006. Martin: can you please reconsider? Thanks :)
(In reply to comment #13) > (In reply to comment #12) > > Good to see the return of my friend dsextras ;-) > > > > I will try a mingw build with this sometime. We should also CC dieterv on this > > bug... > > Am already here :P > > MinGW would have no need for these as the necessary bits > to build with the standard autotools have been in master > for quite a while (back when J5 was still maintaining > PyGObject, before Martin took over). I do remember having > a couple ideas to further improve our python.m4 situation > re proper automatic python interpreter detection on win32 > but have lacked the time to play much with it :( Yeah, but I always found the autotools builds on windows slow, and preferred using distutils with mingw set as the compiler. Anyway, all this is moot unless g-i and typelibs can be built on windows. Whats the status of that? I gave up waiting and we started moving to QT for some stuff at work. I'd like to return to Gtk if feasible.
(In reply to comment #14) > > Yeah, but I always found the autotools builds on windows slow, and preferred > using distutils with mingw set as the compiler. Unfortunately this patchset is only meant for building on Windows using Visual C++-as Dieter said (and perhaps I wasn't being clear in the initial bug report) that this patchset is initially (and intentionally) meant to resurrect building PyGOBject with Visual C++, instead of trying to replace the autotools builds universally (which Visual C++ does not readily support). I can try to adapt this for MinGW builds if needed, but I will need help testing it on there since I don't have a complete GLib-installed MinGW environment set up. > Anyway, all this is moot unless g-i and typelibs can be built on windows. > > Whats the status of that? I gave up waiting and we started moving to > QT for some stuff at work. I'd like to return to Gtk if feasible. In fact, using the distutils build here would require G-I to be built beforehand:) Well, for the Visual C++ part, building and using G-I is possible since G-I 1.35.2, which includes building the GLib .girs and .typelibs. I am working on building the .gir/.typelibs for the other GOBject-based parts of the GTK+ and Clutter stack by making use of .bat files for the Visual C++ builds. Might need to ask Dieter regarding the status of the Windows/MinGW (autotools) builds regarding Windows though, as I am not too sure about its status there:) With blessings.
Created attachment 235106 [details] Image of gtk-demo (for PyGOBject) running on Windows Hi, In case someone might be interested about the status of PyGObect/G-I running on Windows, I attached an image of the gtk-demo PyGObject-3.x running on my Windows 7 x64 system, under Python 3.2.3/x64. With blessings.
Created attachment 237409 [details] [review] Improve the implementation of NAN and INFINITY when not available on MSVC Hi, I thought perhaps it is time to have a better workaround for the C99 NAN and INFINITY constants on Visual C++, where they aren't available from the compiler. Thank you, with blessings!
Created attachment 237410 [details] [review] Resurrect distutils build system (for Windows only) (take iii) Hi, I thought it might be also a good idea to update the distutils build system that's strictly meant for Windows builds. Like the previous attempt, this reads and processes from the autotools files directly during the build process, so that when sources are added or removed, they are reflected in the distutils builds automatically, so that the likelihood of these build files breaking can be greatly reduced (and so that their maintenance will be much simpler). I have added ability to build PyGObject with distutils for GCC/MinGW on Windows, as John asked for, but I would need John (or someone using MinGW) to test it there, as I don't have such build environments for that-so that is what I have for now. Thank you for your time, with blessings.
For Dieter's comments on putting the distutils items in build/win32, I've yet to figure that paths issue, so I don't have that for now, and perhaps any help or tip on that would be great. With blessings, thank you!
Comment on attachment 237409 [details] [review] Improve the implementation of NAN and INFINITY when not available on MSVC Much better, thank you! I pushed this.
Hi, Out of curiosity, did anyone manage to try the distutils build (the last patch I have here) with MinGW so far? With blessings, thank you!
*** Bug 708542 has been marked as a duplicate of this bug. ***
Review of attachment 237410 [details] [review]: Note that some of this may have been simplified with the bug 712197 being fixed (there is now only a single shared Python extension used by all of PyGObject). Similarly, support for single-threaded builds should not be needed as we required multithreading. For the record, I'm also not a fan of multiple build systems or supporting an m4 parser, but we may be able to compromise on some parts of this. For instance, could we maintain a single file which simply lists source files for building the extension and an ini like file which lists version numbers. Both of these useable by autotools and a simplified setup.py without writing parsers?
I wonder what is the status of this. I'd also like to be able to build PyGobject on Windows using MSVC via distutils. Thanks. Cosimo
There is some work on adding a distutils build system in bug 789211 which might help MSVC (similar to how pycairo handles things right now)
Hi, For records, as we are regaining distutils support for PyGObject, let's focus on bug 789211 instead to track ongoing work there. So, I think we should close this bug and redirect attention there. With blessings, thank you!