GNOME Bugzilla – Bug 681820
Windows/MSVC support for building gobject-introspection on Windows
Last modified: 2015-02-07 17:00:53 UTC
Hi, (Sorry if I did not post this in the correct location-I can't seem to find the gobject-introspection (g-i) product on the Bugzilla.) I have managed to come up with additions and updates to g-i such that g-i can be built with Visual C++, along with the .gir files for GLib (GLib-2.0.gir, GModule-2.0.gir, GObject-2.0.gir, Gio-2.0.gir, GIRepository-2.0.gir and their corresponding .typelib files). Please note that this initiative is not meant to supersede Dieter's efforts in Bug 620566, but this is meant to be an add-on to his efforts there. Thank you, with blessings!
First of
Created attachment 221107 [details] [review] Remove C99isms from cmph sources (Sorry for the noise just now as I entered keywords without removing the text in the comment block) First up is the update to cmph. The bundled cmph sources are in C99, but they aren't hard to fix for MSVC/C89 as the C99isms involved are mainly declarations of variables in the middle of the block. I also replaced the use of buflen in chd.c with packed_cr_size as it might not be defined for all platforms, as packed_cr_size seems to represent the value that is to be printed/displayed by the debugging output. This has been checked with the cmph-bdz-test and gthash-test test programs on both Windows/MSVC and Ubuntu 12.04 (x64)/GCC.
Created attachment 221108 [details] [review] Don't include unistd.h in girffi.c unconditionally This simply guards the #include with #ifdef HAVE_UNISTD_H... since we are already including config.h here, as unistd.h is not universally available.
Created attachment 221109 [details] [review] Avoid C99ism in gisacnner/scannerparser.y Like cmph, avoid declaring variables in the middle of the block.
Created attachment 221118 [details] [review] Fix up libgirepository_internals code There are a few C99isms in the sources listed in libgirepository_internals, so fix those up as well. Also, since "interface" may be a compiler keyword, replace that with "giinterface" to avoid trouble there. Enhance the code a bit to make g-ir-compiler relocatable, so that it can find the .gir files using g_win32_get_package_installation_directory_of_module().
Created attachment 221119 [details] [review] Use os.name instead of os.uname()[0] Since os.uname()[0] is not universally available, use os.name instead. This ran fine on Linux (Ubuntu 12.04), but I might need the OpenBSD people to check this.
Created attachment 221120 [details] [review] Enhance dumper.py to support MSVC compiler/linker flags Update dumper.py so that Visual C++ can be supported for compiling, linking and running the "dumper" program, as compiler/linker flags are different in a few ways with that compiler. This was checked on Ubuntu to ensure that the original program logic is not interfered.
Created attachment 221123 [details] [review] Update giscannermodule.c for handling CRT issues and C99isms This incorporates Dieter's patch in bug 620566 with enhancements to deal with CRT differences as the official Python binaries are built with the following compiler: Python 2.5.x: Visual C++ 2003 Python 2.6.x-3.2.x: Visual C++ 2008 Python 3.3 and later: Visual C++ 2010 The code also allows for not using the CRT workarounds if one is using a Visual C++ version that matches the version of Visual C++ that Python is built with. Also, deal with NEW_CLASS macro so that we aren't declaring arrays without a length, which is C99. I will continue to submit more patches later. With blessings, thank you!
Created attachment 221216 [details] [review] Add a pre-configured config.h(.win32.in) This adds a pre-configured config.h(.win32.in) that is suitable for use when building with Visual C++. The version info are put in during the autogen stage, like what is now done in the GTK+/Clutter stack.
Created attachment 221217 [details] [review] Add girepository.symbols file This is used to create the girepository.def file to create the .lib file corresponding to the (lib)girepository DLL, and possibly for other systems to export symbols.
Created attachment 221218 [details] [review] Add .def files for the various tests This adds .def files to export symbols for the DLLs that are built for the tests, so that corresponding .lib files can be generated, which is needed to create the .gir/.typelib files for them.
Created attachment 221219 [details] [review] Use GCC/MinGW as a helper to preprocess the sourecs/headers This is one interesting part... As the Visual C++ compiler/preprocessor (cl.exe) does not accept input from stdin (the "-" cflag), we need to have GCC/MinGW (gcc.exe) as a helper to preprocess the headers so that the dumper program source file can be generated. There may be other workarounds for this, but this is what I have for now. The dumper program source is then compiled and linked with the Visual C++ compiler/linker, and is then run.
Created attachment 221220 [details] [review] sourcescannermain.py: Add --filelist option This is my take on dealing with the 8192 character command line length limit on Windows, so this may probably go to Bug 620566. The thing that causes concern for the command line length is the possible sheer number of headers (such as in GTK+) that are passed to g-ir-scanner. It might make sense to first gather the headers/sources for g-ir-scanner to process in a simple listing text file (with one header/source per line), and then have g-ir-scanner go through the listing file to process those headers/sources. The original method of putting all sources/headers in a single command line is retained by not using this new option.
Created attachment 221221 [details] [review] Add Windows .bat file to generate GLib/GIRepository .gir/.typelib files This is the interesting part... This adds a Windows .bat file to create the .gir/.typelib files for GLib (GLib, GModule, GObject, GIO) and girepository, which requires the following: 1. Visual C++ command line tools/environment (no bash-style shell needed!). 2. Python installation (it must match the configuration that is being built[1]) 3. GCC/MinGW installation.[2] 4. (lib)girepository, _giscanner Python module compiled with the same Visual C++ version used to create the .gir/.typelib files 5. pkg-config with the .pc files for the GLib components [1]: That is, Win32 (x86) or x64 (x86-64), both are available as official Windows binaries. [2]: Attachment 221219 [details] (comment 12) of this bug I will continue to post more patches later, with the Visual C++ project files that will drive this Windows .bat file in the build process. With blessings, thank you!
Created attachment 221225 [details] [review] Add a text file containing sample commands to generate .gir files This is a text file to show people the commands to use in the Visual C++ command line environment (cmd.exe) to generate the .gir files for the various testing DLLs found in tests/ This would also act as samples to show ways to generate the .gir files for various uses under cmd.exe.
Created attachment 221229 [details] [review] Add a text file containing sample commands to generate .gir files This is a text file to show people the commands to use in the Visual C++ command line environment (cmd.exe) to generate the .gir files for the various testing DLLs found in tests/ This would also act as samples to show ways to generate the .gir files for various uses under cmd.exe.
Created attachment 221339 [details] [review] Instructions for building g-i with Visual C++ (Sorry for double posting the previous patch and comment by accident!) This outlines the process where one would build g-i (and the GLib/girepository .gir/.typelib files) with Visual C++ 2008/2010. I would update the mentioned GNOME Live! page once this initiative gets the nod.
Created attachment 221340 [details] [review] The Visual C++ 2008/2010 projects These are the Visual C++ 2008/2010 project files to build the following items of g-i: -The (lib)girepository DLL, along with the needed cmph static .lib -The _giscanner Python module. -The glib_print and cmph-bdz-test sample programs -The various test DLLs in tests/ -g-ir-compiler -g-ir-generator -The .gir/.typelib files for the GLib components and (lib)girepository by calling the gengir.bat file (comment 14, attachment 221221 [details] [review]) The projects for the girepository DLL, cmph and g-ir-compiler (due to use of libgirepository_internal sources), along with the property sheets (due to the various header/Python files to "install") are done with templates so that additions/removal/renaming of sources need only be done in the autotools files, like what is done now.
Created attachment 221341 [details] [review] The master Visual C++ 2008/2010 .sln files These are the master Visual C++ 2008/2010 solution files which opens and builds the project files in the previous patch (attachment 221340 [details] [review]). This patch, along with attachment 221339 [details] [review] will need to be in Windows/DOS EOL in order to work/be presented correctly.
Created attachment 221342 [details] [review] Updates to the autotools files for the MSVC additions As the last part of the proposed patches, I have also updated the autotools files to: -Create the actual Visual C++ projects for (lib)girepository, cmph and g-ir-compiler during "make dist", and create the config.h.win32 with the correct version info. -Distribute the new proposed files (including the newly-"generated" ones) in the tarball Most of the changes are done in the new Makefile-msvcproj.am and Makefile.am's under build/, so to reduce clutter in the existing Makefile*.am's. I have managed to check all these patches using "make dist" and "make distcheck" on Ubuntu 12.04 x86-64, where all the compiling tests ran fine (though the distclean test failed at cleaning the .gir's under tests/ where some of the .gir's were not cleaned away in the process). Thank you for your time, with blessings!
I will look at these (and Dieter's changes) very early in the 3.7 cycle. If I don't get to it early please remind me then. Thanks!
Created attachment 222508 [details] [review] giscanner/dumper.py: support MSVC compiler/linker [take 2] I have updated this patch as the original passing in of the --msvc-syntax flag to pkg-config is not correct. With blessings, thank you!
Created attachment 222521 [details] [review] giscanner/dumper.py: support MSVC compiler/linker [take 3] oops, I forgot to remove a debugging print...sorry.
Created attachment 222522 [details] [review] Windows .bat to generate the .gir/.typelib files for GLib and g-i [take 2] The original .bat proposal missed on a -I item for the GIRepository-2.0.gir part. With blessings, thank you!
Review of attachment 221217 [details] [review]: This patch will require the most ongoing maintenance; is there a way we can use say the "dllexport" annotation on the header files instead?
Review of attachment 221218 [details] [review]: Similar for this one...these symbol lists are going to be annoying extra work to maintain. If there's a way to use an attribute in the header files, or failing that, generate these files, that'd be much preferred.
Otherwise excellent work Chun-wei! Besides that, these patches look pretty good to me.
Attachment 221220 [details] pushed as 5c70ef2 - sourcescannermain.py: Add --filelist option
(In reply to comment #26) > Review of attachment 221218 [details] [review]: > > Similar for this one...these symbol lists are going to be annoying extra work > to maintain. If there's a way to use an attribute in the header files, or > failing that, generate these files, that'd be much preferred. Nevermind, I solved this with: http://git.gnome.org/browse/gobject-introspection/commit/?id=3ee1c9f984a71903a8eeef693c6c187291f25a0b Pushed all patches to master, thanks!
:) Thanks for the review, with blessings!
The addition of the --filelist option is breaking running of update-glib-annotations.py for me. It might work well to change giscanner/annotationsmain.py to use giscanner/scannermain.py:_get_option_parser to construct the initial options instead of only get_preprocessor_option_group (perhaps renamed as a public function get_option_parser). This would also allow the removal of duplicate -library-path and --pkg options from annotationmain.py. This could also help avoid these types of problems in the future as we can rely on the entire options struct to always have valid defaults for anything used by scannermain.py. What do you all think?
Review of attachment 221119 [details] [review]: This breaks on OpenBSD as os.name returns 'posix', as could've been found in the Python documentation. I'll create another ticket to unbreak this diff.
This needed https://git.gnome.org/browse/gobject-introspection/commit/?id=6697c86548f809bd65d3a5f736ac060ced7ccc6c
(In reply to comment #33) > This needed > https://git.gnome.org/browse/gobject-introspection/commit/?id=6697c86548f809bd65d3a5f736ac060ced7ccc6c Ah, I see, didn't think about the clang stuff when I did the MSVC work for g-i..., sorry 'bout that. I think ccache wouldn't work well for Visual C++ anyways, if at all, so that will do. A side note: I guess probably I will also try to see whether I could get the MSVC builds of .gir files to not depend on a GCC installation (and perhaps pkg-config, as pkg-config was more designed for autotools builds than MSVC builds), but that would probably be for the next dev cycle or so. Thanks though, with blessings!
[Mass-moving gobject-introspection tickets to its own Bugzilla product - see bug 708029. Mass-filter your bugmail for this message: introspection20150207 ]