GNOME Bugzilla – Bug 696773
Make gobject introspection more cross-compilation friendly
Last modified: 2016-03-24 08:26:09 UTC
I am working on adding g-i support to Yocto, and it's not an easy thing to support in a cross compilation environment. :) It would be simpler if the package did not combine 3 different things together: the tools, libgirepository and collection of random selection of girs; lumping tools together with things that are built with them is always cross-compilation unfriendly, but that's a minor annoyance. The tools have to be run on target architecture; in case of the compiler this is a simple task of running it under qemu, which seems to work ok. The scanner is more problematic. As a python tool, this can be run on the host, except it builds a binary, which needs to be cross compiled, and then run on the target. The cross compilation part can be worked around by setting up the INTROSPECTION_SCANNER_ENV/ARGS variables, but the run-me part currently requires patching the scanner. My suggestion is to add a command line argument --dump-wrapper (or possibly an env variable INTROSPECTION_SCANNER_DUMP_WRAPPER), that could be used to specify a wrapper for executing the binary to be injected it in front of the normal command line, e.g., g-ir-scanner --dump-wrapper=sysrootpath/qemuwrapper .... gdumpster.py: def _execute_binary_get_tree(self): .... args = [] args.append('syrootpath/qemuwrapper') args.extend(self._binary.args) args.append('--introspect-dump=%s,%s' % (in_path, out_path)) This wrapper would be specific to the dump, i.e., tools like gcc would not be using it. If that seems like a sensible thing to add, I can cook up a patch.
I'm travelling right now so I can't dive into this, but I wanted to add some quick comments: * See previous bugs like https://bugzilla.gnome.org/show_bug.cgi?id=592311 https://bugzilla.gnome.org/show_bug.cgi?id=658126 There's another one too I'm forgetting, one of the Collabora people worked on this a bit. The thing that's a real pain is it's not just g-i that would need invasive fixes for cross compilation; everything *using* it would also need to invoke qemu in the same way too. Maybe we can wedge that somehow automatically into the scanner.
(In reply to comment #1) > The thing that's a real pain is it's not just g-i that would need invasive > fixes for cross compilation; everything *using* it would also need to invoke > qemu in the same way too. Maybe we can wedge that somehow automatically into > the scanner. This is not really that hard to make work under Yocto, just needs little bit of lateral thinking. I got it very close to working in just a few hours of playing with it. As far as the scanner is concerned, only need 2LOC patch to get there at the moment, all the heavy lifting can be done on the Yocto end.
[Mass-moving gobject-introspection tickets to its own Bugzilla product - see bug 708029. Mass-filter your bugmail for this message: introspection20150207 ]
Created attachment 321062 [details] [review] Proposed solution
Review of attachment 321062 [details] [review]: ::: giscanner/gdumpparser.py @@ +165,3 @@ + + # Prepend the launcher command and arguments, if defined + launcher = os.environ.get('GI_LAUNCHER') How about GI_CROSS_LAUNCHER? Might as well be a bit more explicit.
(In reply to Colin Walters from comment #5) > Review of attachment 321062 [details] [review] [review]: > > ::: giscanner/gdumpparser.py > @@ +165,3 @@ > + > + # Prepend the launcher command and arguments, if defined > + launcher = os.environ.get('GI_LAUNCHER') > > How about GI_CROSS_LAUNCHER? Might as well be a bit more explicit. No preferences in this regard.
Created attachment 321072 [details] [review] Use GI_CROSS_LAUNCHER environment variable
Review of attachment 321072 [details] [review]: OK.
Nicola, do you have commit access?
(In reply to Colin Walters from comment #9) > Nicola, do you have commit access? No.
Folks, did you actually test this solution with qemu and cross compilation? We've just merged gobject-introspection support to openembedded/yocto, and to make it really work with qemu when cross compiling for a completely different architecture, a lot more changes are required: http://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta/recipes-gnome/gobject-introspection/gobject-introspection I recommend that you revert this patch, and I'll submit our patchset for your review soon.
We tested at the time. The commits will be reverted, or fixed, after you open a new bug to attach your patches and we have reviewed them.
(In reply to Alexander Kanavin from comment #11) > Folks, did you actually test this solution with qemu and cross compilation? > ... Please, see bug 658126 comment 56 before opening new bugs: all the issues in your patchset are addressed, but in different ways. We can discuss on a per issue basis instead of fighting with patchsets.
There are some issues that your approach doesn't fully solve for us or further clarification is needed: we definitely can't mix binaries built for different architectures in the same build tree; we need proper sysroot support for pkg-config; we need to use a custom ldd command; we need to run g-ir-compiler through the same emulation wrapper (since we don't trust that it's output is not arch-specific); we need to disable building of introspection data (but still build the tools and libraries) when building for platforms on which qemu usermode doesn't work. I'll open a new bug and link it to this one.
(In reply to Alexander Kanavin from comment #14) > ... > we definitely can't mix binaries built for > different architectures in the same build tree; Why? > we need proper sysroot > support for pkg-config; You can already set the PKG_CONFIG precious variable to whatever you want when calling configure. > we need to use a custom ldd command; Bug 761984. > we need to run > g-ir-compiler through the same emulation wrapper (since we don't trust that > it's output is not arch-specific); AFAIK this is the only real issue here. Once bug 764116 is confirmed we can add an endianness parameter to g-ir-compiler instead or whatever. In any case it deserves a bug by itself. > we need to disable building of > introspection data (but still build the tools and libraries) when building > for platforms on which qemu usermode doesn't work. If it is not already feasible, also this one deserves a bug by itself.
> > ... > > we definitely can't mix binaries built for > > different architectures in the same build tree; > > Why? Because a) it's a hack and goes against how you're supposed to build software with autotools. 'configure; make; make install' - if anything else is needed, you have a bug. b) openembedded build system is very specific and strict about separating different architectures to different build trees and sysroots, so they can be properly packaged and installed on targets. Mixing them is simply not allowed. > > we need proper sysroot > > support for pkg-config; > > You can already set the PKG_CONFIG precious variable to whatever you want > when calling configure. It's not about where pkg-config command is, it's about where the sysroot directory is when building g-i dependent software and needing to calling g-i compiler and scanner. This needs to be set in g-i.pc file. > > we need to use a custom ldd command; > > Bug 761984. Excuse me? This is not the same issue at all. We need a custom ldd command in a different place altogether, and definitely not when compiling on Windows. By the way, I don't think magic undocumented environment variables that change g-i tool behavior behind your back is a good approach. I would prefer to explicitly configure these things on the command line.
(In reply to Alexander Kanavin from comment #16) > ... > a) it's a hack and goes against how you're supposed to build software with > autotools. 'configure; make; make install' - if anything else is needed, you > have a bug. When cross-compiling 'configure; make; make install' will never work. > > > we need to use a custom ldd command; > > > > Bug 761984. > > Excuse me? This is not the same issue at all. We need a custom ldd command > in a different place altogether, and definitely not when compiling on > Windows. I don't see the problem in using: ./configure LDD=/whatever/place/custom-ldd-command I am *not* compiling on Windows, I am cross-compiling on linux for Windows with MinGW and wine. I need a launcher, I need a custom ldd, I have a custom pkg-config with its set of .pc files, I have a GCC toolchain in a different sysroot etc. (From my comment #13) > We can discuss on a per issue basis instead of fighting with patchsets. That said, you are obviously free to propose whatever you want and if accepted upstream I will adapt my procedure.