GNOME Bugzilla – Bug 658126
Fix cross compilation with mingw
Last modified: 2018-02-08 12:12:06 UTC
I managed to cross compile gobject-introspection for windows from a linux machine using an installed win32 version of Python with wine. Here are the steps I followed: * Install Python27 with wine: $ wine msiexec /i python27.msi * Apply attached patches * Set PYTHON env to use windows' python: $ export PYTHON="wine ~/.wine/drive_c/Python27/python.exe" * Configure with: $ am_cv_python_pyexecdir=/home/andoni/.wine/drive_c/Python27/Lib/site-packages gure I haven't been able to run successfully gir-scanner yet, which is the last missing step, but I'll post an update once I have it working.
Created attachment 195575 [details] [review] Fix compilation with Mingw The problem here is that grealpath defines GetFullPathNameA() as windows.h is not imported, but for gitscanner.c, windows.h is imported and the compiler throws an error.
Created attachment 195576 [details] [review] Don't include mman.h MinGW doesn't prove sys/mman.h. I'm not sure if commenting out this include is enough or there should be a port to the win32 API too.
Created attachment 195577 [details] [review] Workarround for PYTHON="wine python.exe" I didn't know how to fix properly this one, so I replaced $(PYTHON) with 'pyhton' to make sed happy
Created attachment 195578 [details] [review] Fix extension of g-ir-compiler On windows g-ir-compiler is g-ir-compiler.exe
Created attachment 195579 [details] [review] Make g-ir-scanner at least execute
Review of attachment 195577 [details] [review]: ::: Makefile-tools.am @@ +7,3 @@ tools/g-ir-doc-tool.in +TOOL_SUBSTITUTIONS = sed -e s,@libdir\@,$(libdir), -e s,@datarootdir\@,$(datarootdir), -e s,@PYTHON\@,python, Just putting the arguments in quotes "" should be enough.
Review of attachment 195578 [details] [review]: Looks good.
Review of attachment 195579 [details] [review]: ::: common.mk @@ +12,3 @@ UNINSTALLED_INTROSPECTION_SRCDIR=$(top_srcdir) \ UNINSTALLED_INTROSPECTION_BUILDDIR=$(top_builddir) \ + $(PYTHON) $(top_builddir)/g-ir-scanner It should be sufficient to just do $(top_builddir)/g-ir-scanner without the $(PYTHON), right?
Review of attachment 195575 [details] [review]: Is G_OS_WIN32 right here instead? Also, your comment on the attachment is good, but you should duplicate that into the git commit message. See http://git.fishsoup.net/cgit/git-bz for a tool to help with this.
Review of attachment 195576 [details] [review]: A quick test shows we don't need this on Linux/glibc either; I assume it's leftover, so I've just killed it in git.
(In reply to comment #8) > Review of attachment 195579 [details] [review]: > > ::: common.mk > @@ +12,3 @@ > UNINSTALLED_INTROSPECTION_SRCDIR=$(top_srcdir) \ > UNINSTALLED_INTROSPECTION_BUILDDIR=$(top_builddir) \ > + $(PYTHON) $(top_builddir)/g-ir-scanner > > It should be sufficient to just do $(top_builddir)/g-ir-scanner without the > $(PYTHON), right? The problem here is that we don't want to use system's python, but the windows version instead. When cross-compiling, we are generating dll's and _giscanner.dll would not be imported otherwise.
(In reply to comment #12) > (In reply to comment #8) > > Review of attachment 195579 [details] [review] [details]: > > > > ::: common.mk > > @@ +12,3 @@ > > UNINSTALLED_INTROSPECTION_SRCDIR=$(top_srcdir) \ > > UNINSTALLED_INTROSPECTION_BUILDDIR=$(top_builddir) \ > > + $(PYTHON) $(top_builddir)/g-ir-scanner > > > > It should be sufficient to just do $(top_builddir)/g-ir-scanner without the > > $(PYTHON), right? > > The problem here is that we don't want to use system's python, but the windows > version instead. When cross-compiling, we are generating dll's and > _giscanner.dll would not be imported otherwise. But I think I'm going in the wrong direction here and we should the system's g-ir-scanner if we are cross-compiling instead of through wine. I didn't know there is also some compilation involved and using it through wine means you have to use a toolchain compiled for windows too, which is too much overhead already :)
Created attachment 195589 [details] [review] Use the real version of python instead of a hardcoded one
Created attachment 195590 [details] [review] Fix compilation with Mingw Added comment to the git commit and changed _WIN32 to G_OS_WIN32
Created attachment 195591 [details] [review] Let the user override the default compiler Instead of using 'cc' as a compiler let the user override it like CC=i686-w64-mingw32, as it's usually done in cross-compilation environments.
Created attachment 195592 [details] [review] Fix usage of g-ir-scanner when cross compiling When cross compiling we should use the system's g-ir-scanner.
As far as cross builds go, see https://bugzilla.gnome.org/show_bug.cgi?id=592311
Review of attachment 195591 [details] [review]: I tweaked this patch to look like this: - cpp_args = ['cc', '-E', '-C', '-I.', '-'] + cpp_args = [os.environ.get('CC', 'cc'), '-E', '-C', '-I.', '-'] I just think it's cleaner.
Review of attachment 195592 [details] [review]: ::: common.mk @@ +20,3 @@ UNINSTALLED_INTROSPECTION_BUILDDIR=$(top_builddir) \ + $(top_builddir)/g-ir-scanner +endif I don't understand what you're trying to do here. It looks like we're dropping $(EXEEXT) which should be a separate patch. Actually I'll just do that now. Dropping the _SRCDIR variable is going to be broken in srcdir != builddir builds.
(In reply to comment #13) > > But I think I'm going in the wrong direction here and we should the system's > g-ir-scanner if we are cross-compiling instead of through wine. The fundamental problem is that g-ir-scanner needs to run target code. That's what bug 592311 is all about. Now, in addition to the options listed in that bug, one that I didn't mention is that g-ir-scanner could explicitly understand cross builds, and to do the dump on the host. But that would require compiling the code twice, which ends up being hideous to do in automake at least.
It's the first time I hack on gobject-instrospection, and at the moment my unique goal was trying to get it cross-compiling for Windows. But like I see it's not straightforward :) So I tried 2 approaches: * Run everything with wine to simulate the target host. I dropped it in a first instance, but after reading your comments it might be a possible solution after installing the same toolchain in the wine environment, * Use the system's g-ir-scanner. This requires more development like an alternate impletation of _resolve_non_libtool using 'objdump' instead of 'ldd' I'm building a cross-compiling build system for windows based on jhbuild[1] and I was trying to get this cross-compiled as it's a dependency for recent pygobject, but I'll probably use an older version :) Feel free to close this bug report as now it's a duplicate of bug 592311. [1] https://github.com/ossbuild/ossbuild-main
(In reply to comment #22) > It's the first time I hack on gobject-instrospection, and at the moment my > unique goal was trying to get it cross-compiling for Windows. But like I see > it's not straightforward :) Yeah, sucks =( If only I had a time machine to go back and fix GType... > So I tried 2 approaches: > * Run everything with wine to simulate the target host. I dropped it in a > first instance, but after reading your comments it might be a possible solution > after installing the same toolchain in the wine environment, Right, basically don't cross compile everything that depends on g-i (pretending Wine is the same as Windows). > * Use the system's g-ir-scanner. This requires more development like an > alternate impletation of _resolve_non_libtool using 'objdump' instead of 'ldd' This doesn't help you because code from each project using g-ir-scanner needs to be run. The problem isn't just cross-building g-i (which is what builds GLib-2.0.gir, GObject-2.0.gir, and Gio-2.0.gir), but cross building every library and program that uses g-i. > I'm building a cross-compiling build system for windows based on jhbuild[1] and > I was trying to get this cross-compiled as it's a dependency for recent > pygobject, but I'll probably use an older version :) I'm also a jhbuild developer; please send patches as you run into problems. I will say though that as far as long term goes, I have truly fallen in love with Yocto (and its core tool, bitbake): http://www.yoctoproject.org/ Not to discourage you from jhbuild patches, but I'd at least look at bitbake if you haven't.
[Mass-moving gobject-introspection tickets to its own Bugzilla product - see bug 708029. Mass-filter your bugmail for this message: introspection20150207 ]
Ok, I managed to cross-compile gobject-introspection using wine and the tools on the build side. It has been quite a tough job. gobject-instrospection itself needs only few adjustment. I have 11 small commits on GitHub [1] rebased on 1.43.3. After that, the procedure for cross-compiling is far from easy but at least it is feasible: export CC=$arch-gcc export PKG_CONFIG=$arch-pkg-config export DLLTOOL=$arch-dlltool export PYTHON=/usr/bin/python2 export GI_OS_NAME=nt export INTROSPECTION_LAUNCHER="/usr/bin/env WINEARCH=win$bits WINEPREFIX=$srcdir/win$bits DISPLAY= /usr/bin/wine" export XDG_DATA_DIRS="/usr/$arch/share:$XDG_DATA_DIRS" export GI_TYPELIB_PATH="/usr/$arch/lib/girepository-1.0:$GI_TYPELIB_PATH" # First pass: build the giscanner Python module with build == host ./configure \ --disable-silent-rules \ --enable-shared \ --enable-static make scannerparser.c _giscanner.la # Second pass: build everything else with build != host (i.e. cross-compile) ./configure \ --build=... \ --host=... \ --prefix=... \ --disable-silent-rules \ --enable-shared \ --enable-static \ --disable-doctool \ --disable-giscanner # Avoid overriding what built in the first pass make -t scannerparser.c _giscanner.la make INTROSPECTION_LAUNCHER="$INTROSPECTION_LAUNCHER" Using the above procedure I succesfully built the whole GTK+2/GTK+3 MinGW toolchain with introspection enabled [2]. Further details on the related article I just published [3]. [1] https://github.com/ntd/gobject-introspection/tree/xbuild [2] https://github.com/ntd/aur-fedora-mingw [3] http://ntd.github.io/#mingw-gobject-introspection
Created attachment 305647 [details] [review] Use pkg-config to get python2 flags
Created attachment 305648 [details] [review] Pass to python the proper path separator
Created attachment 305649 [details] [review] Use $(PKG_CONFIG) instead of pkg-config
Created attachment 305650 [details] [review] Directly use file descriptor on MinGW platform
Created attachment 305651 [details] [review] Avoid hard-coded programs in scanner
Created attachment 305652 [details] [review] Do not hardcode .exe extension for 'nt'
Created attachment 305653 [details] [review] Allow overriding of os.name
Created attachment 305654 [details] [review] Remove leading $SHELL in libtool call
Created attachment 305655 [details] [review] Assume gcc use the same path separator as python
Created attachment 305657 [details] [review] Use a launcher to execute g-ir-scanner
Created attachment 305658 [details] [review] Allow to skip the check for Python headers
Review of attachment 305654 [details] [review]: Hum...but clearly that code was added for a reason in https://git.gnome.org/browse/gobject-introspection/commit/?id=f3fcdf97 Might this be a no-automake versus automake issue?
Review of attachment 305657 [details] [review]: This looks like a dup of: https://bugzilla.gnome.org/show_bug.cgi?id=696773 Which, sounds like it's desired and is clearly easy to support, so I'm OK with this.
Review of attachment 305654 [details] [review]: If SHELL is required I'd say to add it directly to get_libtool_command instead: there are two other places where it is used. In my tests I just noticed the shell was called twice (yes, I was using automake... I didn't even suppose you couldn't). Beware I passed throught *a lot* of trial and error stages, so maybe some patch is not required. If you think is worth I can test the whole thingy without this patch applied.
How much does this intersect with the work in bug 753428 that landed?
I just checked and it seems the intersection is an empty set. Although the goal is similar they use a really different approach: bug 753428 is about native compilation with Visual C++ while bug 658126 is about cross compilation with GCC. I doubt there will ever be some code in common.
Review of attachment 305649 [details] [review]: Superseded by bug 746669.
Review of attachment 305650 [details] [review]: Superseded by bug 733535.
A couple of similar patches has been merged upstream from other bugs so, in an attempt to move this issue on, I split the more streamlined patches on their own: - https://bugzilla.gnome.org/show_bug.cgi?id=761983 - https://bugzilla.gnome.org/show_bug.cgi?id=761984 - https://bugzilla.gnome.org/show_bug.cgi?id=761985 - https://bugzilla.gnome.org/show_bug.cgi?id=761982 - https://bugzilla.gnome.org/show_bug.cgi?id=761981 The patch for the launcher has been submitted to: - https://bugzilla.gnome.org/show_bug.cgi?id=696773
Review of attachment 305653 [details] [review]: Superseded by bug 761985
Review of attachment 305654 [details] [review]: Superseded by bug 761982
Review of attachment 305657 [details] [review]: Superseded by bug 696773
Review of attachment 305655 [details] [review]: Superseded by bug 761981
Review of attachment 305651 [details] [review]: Superseded by bug 761984
Created attachment 321064 [details] [review] Allow to skip the check for Python headers
Created attachment 321065 [details] [review] Hack Just to be able to build the project
Created attachment 321066 [details] [review] mingw gcc during cross-compilation is an ibrid compiler
Review of attachment 305647 [details] [review]: Obsolete
Review of attachment 305648 [details] [review]: Obsolete
Review of attachment 305652 [details] [review]: Obsolete
By appling the patches provided in bug 761983, bug 761984, bug 761985, bug 761982, bug 761981 and bug 696773 an the three dirty patches provided here, i.e. attachment 321064 [details] [review], attachment 321065 [details] [review] and attachment 321064 [details] [review], I am able to cross-compile HEAD (cd355e86a28b) with something similar to the following script: # Force the use of the python-2 interpreter export PYTHON=/usr/bin/python2 # Build the python module with build == host ./configure" make scannerparser.c _giscanner.la export arch=i686-w64-mingw32 # or x86_64-w64-mingw32 export bits=32 # or 64 # Cross-compile the rest of the package export CC=$arch-gcc export PKG_CONFIG=$arch-pkg-config export DLLTOOL=$arch-dlltool export GI_HOST_OS=nt export GI_LAUNCHER="env WINEARCH=win$bits WINEPREFIX=$srcdir/win$bits DISPLAY= /usr/bin/wine" export GI_TYPELIB_PATH="/usr/$arch/lib/girepository-1.0:$GI_TYPELIB_PATH" export XDG_DATA_DIRS="/usr/$arch/share:$XDG_DATA_DIRS" ./configure" --host="$arch" --prefix="/usr/$arch" \ --disable-doctool --disable-giscanner # Avoid overriding what built with build == host make -t scannerparser.c _giscanner.la make
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gobject-introspection/issues/57.