After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 692856 - Regain Visual C++ support via resurrecting distutils build system
Regain Visual C++ support via resurrecting distutils build system
Status: RESOLVED OBSOLETE
Product: pygobject
Classification: Bindings
Component: general
3.7.x
Other Windows
: Normal normal
: ---
Assigned To: Nobody's working on this now (help wanted and appreciated)
Python bindings maintainers
Depends on:
Blocks:
 
 
Reported: 2013-01-30 06:38 UTC by Fan, Chun-wei
Modified: 2017-12-06 04:04 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Remove GCCism from gi/pygobject-external.h (951 bytes, patch)
2013-01-30 06:41 UTC, Fan, Chun-wei
committed Details | Review
Remove C99ism by defining variables toward the top of their respective blocks (7.63 KB, patch)
2013-01-30 06:43 UTC, Fan, Chun-wei
committed Details | Review
Deal with empty array declarations with no length specified (3.67 KB, patch)
2013-01-30 06:45 UTC, Fan, Chun-wei
committed Details | Review
Add implementation of NAN and INFINITY (2.21 KB, patch)
2013-01-30 06:47 UTC, Fan, Chun-wei
needs-work Details | Review
The distutils setup.py and associated items (31.26 KB, patch)
2013-01-30 06:55 UTC, Fan, Chun-wei
none Details | Review
2nd take on resurrecting the distutils-based build files (33.68 KB, patch)
2013-02-01 10:04 UTC, Fan, Chun-wei
none Details | Review
Image of gtk-demo (for PyGOBject) running on Windows (580.74 KB, image/jpeg)
2013-02-03 14:45 UTC, Fan, Chun-wei
  Details
Improve the implementation of NAN and INFINITY when not available on MSVC (1.97 KB, patch)
2013-02-26 08:26 UTC, Fan, Chun-wei
committed Details | Review
Resurrect distutils build system (for Windows only) (take iii) (34.60 KB, patch)
2013-02-26 08:33 UTC, Fan, Chun-wei
reviewed Details | Review

Description Fan, Chun-wei 2013-01-30 06:38:53 UTC
Hi,

When we were approaching 3.0, the distutils build system was removed from the PyGObject package as we are now able to build PyGObject on/for Windows using GCC/MinGW using the normal autotools-based build system.

Since support for using G-I on Windows (using Visual C++) was officially added to its master branch since 1.35.2, I thought it might be a good idea that we get the distutils-based build system back for PyGObject, especially that there is the claim that MSVC is also supported for building/using PyGObject[1].

This is a series of patches that I have to resurrect the distutils-based build system-please note that this is meant only for Visual C++ builds as Dieter (dieterv) mentioned when he removed the distutils-based build system from PyGObject[2].

[1]: Please see the README file in the srcroot directory.
[2]:http://git.gnome.org/browse/pygobject/commit/?id=8d3125c8ce9890c70400dd8a3ac273b590fe6a31, (Remove distutils based build system.)
Comment 1 Fan, Chun-wei 2013-01-30 06:41:15 UTC
Created attachment 234797 [details] [review]
Remove GCCism from gi/pygobject-external.h

First up is the patch to get rid of GCCism that went into gi/pygobject-external.h.  We could make use of G_GNUC_UNUSED in place of that, which serves the same purpose on GCC, as the GLib headers should already been included before including that header (due to the existing use of gboolean).
Comment 2 Fan, Chun-wei 2013-01-30 06:43:26 UTC
Created attachment 234799 [details] [review]
Remove C99ism by defining variables toward the top of their respective blocks

This moves variable declarations towards the top of their respective blocks, so that C89 compilers such as MSVC would not complain.
Comment 3 Fan, Chun-wei 2013-01-30 06:45:58 UTC
Created attachment 234800 [details] [review]
Deal with empty array declarations with no length specified

This deals with another C99ism as C89 requires empty arrays to have lengths specified for them during declaration.  There might be better ways to deal with this, but this is what I have which worked for me (TM) :)
Comment 4 Fan, Chun-wei 2013-01-30 06:47:57 UTC
Created attachment 234801 [details] [review]
Add implementation of NAN and INFINITY

Again, Visual C++ does not have NAN nor INFINITY defined, which was introduced very recently into the code, so we define them here for its use.  I know this is not very elegant :)
Comment 5 Fan, Chun-wei 2013-01-30 06:55:19 UTC
Created attachment 234802 [details] [review]
The distutils setup.py and associated items

The last part of the patches I have at the moment is the setup.py and related items for the distutils build.  This will build the gi/_glib, gi/_gobject, gi and _gi_cairo (_gi_cairo if applicable) with the distutils build approach.  This will enable PyGObject be built on Windows in both x86 and x64 flavors using Visual C++.

This is adapted from the distutils build system from 2.28.6, but since I am not that good in Python, there might be areas where this could be improved upon (although again, this worked for me :) (TM))

Please note that in order to use this on Python 3.x, one needs to run this through 2to3 which is included with a standard Python installation.

I will come up with a README.win32 which will attempt to walk people through the process of building PyGOBject-3.x on Windows using Visual C++.

Thanks for the time, with blessings!
Comment 6 Martin Pitt 2013-01-30 14:07:15 UTC
Comment on attachment 234797 [details] [review]
Remove GCCism from gi/pygobject-external.h

Thanks for those! It feels like a step backwards, but I committed the first two patches as they are quite harmless.
Comment 7 Martin Pitt 2013-01-30 14:12:48 UTC
Comment on attachment 234800 [details] [review]
Deal with empty array declarations with no length specified

A declaration like

  static PyMethodDef _PyGIRepository_methods[9];

seems outright dangerous, as it's surely forgotten the next time the array is updated. However, we do not need these forward declarations in the first place, so I slightly modified the patch (and its changelog):

http://git.gnome.org/browse/pygobject/commit/?id=9e0c415
Comment 8 Martin Pitt 2013-01-30 14:15:44 UTC
Comment on attachment 234801 [details] [review]
Add implementation of NAN and INFINITY

This is outright ugly -- Visual C must have _some_ less hackish way to represent NAN and INFINITY? These constants aren't exactly new, after all?
Comment 9 Martin Pitt 2013-01-30 14:22:40 UTC
Comment on attachment 234802 [details] [review]
The distutils setup.py and associated items

That was already discussed in bug 680138 and declined. It's really not maintainable to have two different build systems; the non-default one will just keep breaking all the time.

If we do this, then we need to do it all the way through and only use distutils. But this would then break compatibility as a GNOME module and jhbuild etc. But if we have a working distutils, this could be rectified with having a small static Makefile which just calls $PYTHON setup.py <whatever>.
Comment 10 Fan, Chun-wei 2013-01-30 15:55:25 UTC
(In reply to comment #9)
> (From update of attachment 234802 [details] [review])
> It's really not maintainable to have two different build systems; the 
> non-default one will just keep breaking all the time.

Hi Martin,

I'd agree on this part, as sources are added/removed from time to time, which is the main reason why these static build files would get into trouble.

Would it be better for you if I feed in the sources to compile from the autotools files into the distutils files during 'make dist', such as what is being done for the MSVC project files in the GTK+ stack?

Otherwise, I might need to come up with an universal distutils build system as you mentioned, and I would most probably need help for testing them on various platforms that PyGObject supports, as I don't have access to all of them.

(In reply to comment #8)
> This is outright ugly -- Visual C must have _some_ less hackish way to
> represent NAN and INFINITY? These constants aren't exactly new, after all?

I have to agree with you that it is very ugly-but that's life on that compiler.  I will try to see whether there are nicer ways to represent those constants on that platform.  The kdewin project, for example, also tries to emulate these macros through hacks as well-I would want blame Microsoft for this myself. :|

Thanks for your time for the reviews, feedbacks; and for hearing my rants and thoughts on this:)

With blessings.
Comment 11 Fan, Chun-wei 2013-02-01 10:04:34 UTC
Created attachment 234952 [details] [review]
2nd take on resurrecting the distutils-based build files

Hi,

I was looking through the bug post that Martin posted and it does seem to me that there is quite a bit of reluctance to forgo autotools for building PyGObject.  I thought, then, that it might be a good idea to make the distutils make advantage of the autotools files as a result.

It seems that by Python we could avoid updating the autotools files (as I mentioned in my previous comment) to make sure the file list of sources to build and the scripts to install are up-to-date.  So as a result, I made use of some regex items from setup.py (under build/win32) from the GLib package, which allowed me to read from the autotools files.  This means that we can have an up-to-date C-source list to build for all the submodules, versioning and dependency versioning info and list of Python scripts to package/install for all the submodules as long as the autotools files are up-to-date about these items.  This should help to greatly reduce the odds of the distutils build files breaking as we move forward, without adding stuff into the autotools files, except of course to dist the distutils build files (setup.py and the utility dsextras_pygobject3.py).  New sub-modules to build/install can be added without much difficulty here, which I can voluteer to do if that is needed (as I currently do for the MSVC build files for the Clutter/GTK+ stack).

Hope this is better for an interim solution before (and if we) we take the bite to go to a completely disutils-based build approach.

With blessings, thank you!
Comment 12 John Stowers 2013-02-01 10:33:28 UTC
Good to see the return of my friend dsextras ;-)

I will try a mingw build with this sometime. We should also CC dieterv on this bug...
Comment 13 Dieter Verfaillie 2013-02-01 11:22:45 UTC
(In reply to comment #12)
> Good to see the return of my friend dsextras ;-)
> 
> I will try a mingw build with this sometime. We should also CC dieterv on this
> bug...

Am already here :P

MinGW would have no need for these as the necessary bits
to build with the standard autotools have been in master
for quite a while (back when J5 was still maintaining
PyGObject, before Martin took over). I do remember having
a couple ideas to further improve our python.m4 situation
re proper automatic python interpreter detection on win32
but have lacked the time to play much with it :(

Haven't built fresh binaries for a couple of months though
so all this might have regressed a bit left and right?

The way I see it, distutils support should go back to be
msvc only (like how it was when originally written) and
thus it might be better to put everything (setup.py,
dsextras.py, etc) into a build/win32 directory as is done
in glib, gtk+, etc to prevent people getting the wrong ideas?

(In reply to comment #9)
> That was already discussed in bug 680138 and declined.

If I understand this correctly, this is not asking for
generic cross platform distutils support (that would be
somewhat crazy, autotools gives us so much more than
distutils is capable of) so treating this like bug #680138
is a bit unfair. This is about putting new life in
supporting msvc which we haven't had since Cedric Gustin
stopped maintaining this for PyGTK way back around 2006.

Martin: can you please reconsider? Thanks :)
Comment 14 John Stowers 2013-02-03 10:41:52 UTC
(In reply to comment #13)
> (In reply to comment #12)
> > Good to see the return of my friend dsextras ;-)
> > 
> > I will try a mingw build with this sometime. We should also CC dieterv on this
> > bug...
> 
> Am already here :P
> 
> MinGW would have no need for these as the necessary bits
> to build with the standard autotools have been in master
> for quite a while (back when J5 was still maintaining
> PyGObject, before Martin took over). I do remember having
> a couple ideas to further improve our python.m4 situation
> re proper automatic python interpreter detection on win32
> but have lacked the time to play much with it :(

Yeah, but I always found the autotools builds on windows slow, and preferred using distutils with mingw set as the compiler.

Anyway, all this is moot unless g-i and typelibs can be built on windows.

Whats the status of that? I gave up waiting and we started moving to 
QT for some stuff at work. I'd like to return to Gtk if feasible.
Comment 15 Fan, Chun-wei 2013-02-03 14:24:49 UTC
(In reply to comment #14)
> 
> Yeah, but I always found the autotools builds on windows slow, and preferred
> using distutils with mingw set as the compiler.

Unfortunately this patchset is only meant for building on Windows using Visual C++-as Dieter said (and perhaps I wasn't being clear in the initial bug report) that this patchset is initially (and intentionally) meant to resurrect building PyGOBject with Visual C++, instead of trying to replace the autotools builds universally (which Visual C++ does not readily support).

I can try to adapt this for MinGW builds if needed, but I will need help testing it on there since I don't have a complete GLib-installed MinGW environment set up.

> Anyway, all this is moot unless g-i and typelibs can be built on windows.
> 
> Whats the status of that? I gave up waiting and we started moving to 
> QT for some stuff at work. I'd like to return to Gtk if feasible.

In fact, using the distutils build here would require G-I to be built beforehand:)

Well, for the Visual C++ part, building and using G-I is possible since G-I 1.35.2, which includes building the GLib .girs and .typelibs.  I am working on building the .gir/.typelibs for the other GOBject-based parts of the GTK+ and Clutter stack by making use of .bat files for the Visual C++ builds.

Might need to ask Dieter regarding the status of the Windows/MinGW (autotools) builds regarding Windows though, as I am not too sure about its status there:)

With blessings.
Comment 16 Fan, Chun-wei 2013-02-03 14:45:04 UTC
Created attachment 235106 [details]
Image of gtk-demo (for PyGOBject) running on Windows

Hi,

In case someone might be interested about the status of PyGObect/G-I running on Windows, I attached an image of the gtk-demo PyGObject-3.x running on my Windows 7 x64 system, under Python 3.2.3/x64.

With blessings.
Comment 17 Fan, Chun-wei 2013-02-26 08:26:30 UTC
Created attachment 237409 [details] [review]
Improve the implementation of NAN and INFINITY when not available on MSVC

Hi,

I thought perhaps it is time to have a better workaround for the C99 NAN and INFINITY constants on Visual C++, where they aren't available from the compiler.

Thank you, with blessings!
Comment 18 Fan, Chun-wei 2013-02-26 08:33:02 UTC
Created attachment 237410 [details] [review]
Resurrect distutils build system (for Windows only) (take iii)

Hi,

I thought it might be also a good idea to update the distutils build system that's strictly meant for Windows builds.  Like the previous attempt, this reads and processes from the autotools files directly during the build process, so that when sources are added or removed, they are reflected in the distutils builds automatically, so that the likelihood of these build files breaking can be greatly reduced (and so that their maintenance will be much simpler).

I have added ability to build PyGObject with distutils for GCC/MinGW on Windows, as John asked for, but I would need John (or someone using MinGW) to test it there, as I don't have such build environments for that-so that is what I have for now.

Thank you for your time, with blessings.
Comment 19 Fan, Chun-wei 2013-02-26 08:34:42 UTC
For Dieter's comments on putting the distutils items in build/win32, I've yet to figure that paths issue, so I don't have that for now, and perhaps any help or tip on that would be great.

With blessings, thank you!
Comment 20 Martin Pitt 2013-02-26 15:44:25 UTC
Comment on attachment 237409 [details] [review]
Improve the implementation of NAN and INFINITY when not available on MSVC

Much better, thank you! I pushed this.
Comment 21 Fan, Chun-wei 2013-03-21 08:17:32 UTC
Hi,

Out of curiosity, did anyone manage to try the distutils build (the last patch I have here) with MinGW so far?

With blessings, thank you!
Comment 22 Simon Feltman 2013-09-21 19:31:01 UTC
*** Bug 708542 has been marked as a duplicate of this bug. ***
Comment 23 Simon Feltman 2014-02-01 08:09:31 UTC
Review of attachment 237410 [details] [review]:

Note that some of this may have been simplified with the bug 712197 being fixed (there is now only a single shared Python extension used by all of PyGObject). Similarly, support for single-threaded builds should not be needed as we required multithreading.

For the record, I'm also not a fan of multiple build systems or supporting an m4 parser, but we may be able to compromise on some parts of this. For instance, could we maintain a single file which simply lists source files for building the extension and an ini like file which lists version numbers. Both of these useable by autotools and a simplified setup.py without writing parsers?
Comment 24 Cosimo Lupo 2016-02-17 11:08:39 UTC
I wonder what is the status of this.
I'd also like to be able to build PyGobject on Windows using MSVC via distutils.
Thanks.

Cosimo
Comment 25 Christoph Reiter (lazka) 2017-12-04 15:06:32 UTC
There is some work on adding a distutils build system in bug 789211 which might help MSVC (similar to how pycairo handles things right now)
Comment 26 Fan, Chun-wei 2017-12-06 04:04:43 UTC
Hi,

For records, as we are regaining distutils support for PyGObject, let's focus on bug 789211 instead to track ongoing work there.  So, I think we should close this bug and redirect attention there.

With blessings, thank you!