This kind of thing is amazingly hard to debug; my guess is something in the stack is overwriting LD_LIBRARY_PATH incorrectly, but I don't see that being done currently in gobject-introspection or gir-repository at least.

Note suggested workaround: remove gir-repository from system install paths.

Created attachment 158329 [details]
make with error

Hi there,

I have something like that here. Using jhbuild, linking stops at gobject-introspection because of missing symbols.  I realized it was looking at the wrong path (my system /usr/lib path), then I discovered that the issue was caused by a wrong order in LD_LIBRARY_PATH in the file Gio-2.0 (wrapper to .libs/Gio-2.0 ELF binary) generated by the libtool script.  Modifying this one at the related line, I was able to control the order in LD_LIBRARY_PATH in Gio-2.0 wrapper script and having a straitforward linking.

But the same trouble happened at the next module...

I think it's a libtool or autoconf/automake issue.

Attaching my affected 'make'.  Hope this help.

Giuseppe

I used to have a libgirepository RPM installed as part of the gobject-introspection meta-RPM in openSUSE, just like Sandy.

However, when *that* is present, gobject-introspection fails to build under jhbuild (and probably also fails if you build it by hand):

[jhbuild] linux-7b5d$ make V=1
env LPATH=.libs PYTHONPATH=..:.. UNINSTALLED_INTROSPECTION_SRCDIR=.. UNINSTALLED_INTROSPECTION_BUILDDIR=.. ../tools/g-ir-scanner --verbose -I.. --add-include-
path=. --add-include-path=../gir --add-include-path=. --add-include-path=../gir --namespace=Everything --nsversion=1.0 --libtool="/bin/sh ../libtool"  --libra
ry=libgirepository-everything-1.0.la --pkg=gobject-2.0 --pkg=cairo --pkg=gio-2.0 --include=GObject-2.0 --include=cairo-1.0 --include=Gio-2.0    everything.h e
verything.c Gio-2.0.gir libgirepository-everything-1.0.la --output Everything-1.0.gir
introspecting class TestObj
Traceback (most recent call last):

+ Trace 222360

make: *** [Everything-1.0.gir] Error 1  

Some interesting things:

- No file in gir/tmp-introspectXXXX/* contained the "parents" string.

- In that directory, dump.xml had <class name="TestObj" get-type="test_obj_get_type" parent="GObject"> - but not "parents" - maybe that is created by the older, system-installed libgirepository.so.

- The binaries in gobject-introspection/tools/.libs are linked to /usr/lib/libgirepository*.so - or so ldd reports.  This looks wrong, but I'm not sure if that path is supposed to change if you actually run those binaries through libtool's wrapper scripts.

I couldn't figure out if something was wrong libtool-wise.  Maybe when running the tools/ binaries, they need to be called explicitly with "libtool --mode=execute" or something.  I had to "rpm -e" the libgirepository RPM; then rebuilding gobject-introspection actually worked.

*** Bug 621500 has been marked as a duplicate of this bug. ***

Random guess: this occurs when the scanner is run outside of jhbuild, for example:

$ jhbuild run ./configure --prefix=...
$ make

As opposed to what I do which:

$ jhbuild run ./configure --prefix
$ jhbuild run make

In the first, LD_LIBRARY_PATH is not set, so we'll be picking up the system libgirepository.so.  In the latter, we grab the one from the development root.

The only fix I can think of here is to have the scanner detect where it's installed, and make a last ditch effort to synthesize a LD_LIBRARY_PATH from that.  This gets into lib/lib64 hell though.

(In reply to comment #6)
> Random guess: this occurs when the scanner is run outside of jhbuild, for
> example:
> 

I'm 99% sure that I only ran things from "jhbuild build" or "jhbuild shell" when I was trying to debug this.

Even selecting the "wipe directory and start over" option didn't work as long as the system's version of the library was installed.

But it worked on another box with the system library installed.  Go figure.

Sadly; same problem here - this is not good (TM) ;-) wasted a day or so.

With a bit of debugging - it seems like we compile some magic C program:

/usr/opt/hgnome/src/pango/pango/tmp-introspectQe9RKo/PangoXft-1.0

which mercifully is not deleted on failure; so you can see it is a libtool wrapper, it contains the immortal line:

  if test -f "$progdir/$program"; then
    # Add our own library path to LD_LIBRARY_PATH
    LD_LIBRARY_PATH="/opt/hgnome/src/pango/pango/.libs:/usr/opt/hgnome/src/pango/pango/.libs:/usr/lib:/opt/hgnome/lib:$LD_LIBRARY_PATH"

which matches the strace madness:

25400 open("/opt/hgnome/src/pango/pango/.libs/libgirepository-1.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
25400 open("/usr/opt/hgnome/src/pango/pango/.libs/libgirepository-1.0.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
25400 open("/usr/lib/libgirepository-1.0.so.0", O_RDONLY) = 4

which in turn is created by libtool:

25198 execve("/bin/sh", ["/bin/sh", "../libtool", "--mode=link", "--tag=CC", "--silent", "gcc", "-o", "/usr/opt/hgnome/src/pango/pango/tmp-introspectQe9RKo/PangoXft-1.0", "-L.", "libpangoxft
-1.0.la", "-pthread", "-Wl,--export-dynamic", "-L/opt/hgnome/lib", "-lgio-2.0", "-lgirepository-1.0", "-lffi", "-lgobject-2.0", "-lgmodule-2.0", "-lgthread-2.0", "-lrt", "-lglib-2.0", "/usr/
opt/hgnome/src/pango/pango/tmp-introspectQe9RKo/PangoXft-1.0.o"

Which oddly enough looks just fine. The jhbuild shell environment contains:

$ echo $LD_LIBRARY_PATH 
/opt/hgnome/lib:/usr/lib/mpi/gcc/openmpi/lib

which looks ok too. So - where does that bogus /usr/lib come from ? :-)

This is written by the libtool script here:

	# Export our shlibpath_var if we have one.
	if test "$shlibpath_overrides_runpath" = yes && test -n "$shlibpath_var" && test -n "$temp_rpath"; then
	  $ECHO "\
    # Add our own library path to $shlibpath_var
    $shlibpath_var=\"$temp_rpath\$$shlibpath_var\"

debug shows $temp_rpath contains the /usr/lib corruption.

Hmm; as the temp_rpath is built up, I added some debug and (it seems) the badness comes from:

	    # We need to hardcode the library path
	    if test -n "$shlibpath_var" && test -z "$avoidtemprpath" ; then
	      # Make sure the rpath contains only unique directories.
	      case "$temp_rpath:" in
	      *"$absdir:"*) ;;
	      *) temp_rpath="$temp_rpath$absdir:" ;;
	      esac
	    fi

+	    echo "temp rpath contains: '$temp_rpath' from '$absdir' '$libdir' '$deplib'" >> /dev/stderr

	    # Hardcode the library path.
	    # Skip directories that are in the system default run-time

yielding:
...
temp rpath contains: '/opt/hgnome/src/pango/pango/.libs:/usr/opt/hgnome/src/pango/pango/.libs:/usr/lib:' from '/usr/lib' '/usr/lib' '/usr/lib/libXrender.la'

Interestingly; the compile and link rpaths have this system badness removed from them, but not the temp_rpath that we use; most odd.

I seem to have libtool-2.2.6 installed by both bootstrap, and packaged locally; urk ! ... ;-(

Weell ... it seems it is possible / easy to work around this by ensuring we pass the .la files to libtool -after- the libraries that we want to link from pkgconfig; thus:

diff --git a/giscanner/dumper.py b/giscanner/dumper.py
index 14540d9..813b0ef 100644
--- a/giscanner/dumper.py
+++ b/giscanner/dumper.py
@@ -197,6 +197,7 @@ class DumpCompiler(object):
         # Search the current directory first
         args.append('-L.')
 
+        libtool_args = []
         uninst_builddir = os.environ.get('UNINSTALLED_INTROSPECTION_BUILDDIR')
         # hack for building GIRepository.gir, skip -lgirepository-1.0 since
         # libgirepository-1.0.la is not in current directory and we refer to it
@@ -206,7 +207,7 @@ class DumpCompiler(object):
                 self._options.libraries[0] == 'girepository-1.0'):
                 continue
             if library.endswith(".la"): # explicitly specified libtool library
-                args.append(library)
+                libtool_args.append(library)
             else:
                 args.append('-l' + library)
 
@@ -221,6 +222,8 @@ class DumpCompiler(object):
             if not os.path.exists(source):
                 raise CompilerError(
                     "Could not find object file: %s" % (source, ))
+        # need to be late - cf. bug#605156
+        args.extend(libtool_args);
         args.extend(list(sources))
 
         subprocess.check_call(args)


Review / permission to commit appreciated :-)

(In reply to comment #11)
> Weell ... it seems it is possible / easy to work around this by ensuring we
> pass the .la files to libtool -after- the libraries that we want to link from
> pkgconfig; thus:

Thanks a lot for looking at this Michael, and I apologize for the lost time debugging it; I know debugging twisted build stuff is really painful.

I actually had a similar realization about the library ordering yesterday when someone else hit this but didn't get a chance to commit a fix.  Yours is blunt but that's fine by me; it seems exceedingly unlikely that someone is going to want to link a libtool library before a regular one (and actually I'm fine not supporting that).

So go ahead and commit.  I'm going to attach a different patch to this bug which I think would also fix it.

Created attachment 164614 [details] [review]
Move pkg-config calls before --library and --program

We need our just-built library path to override what we have
from pkg-config.

Ho hum; well my 'fix' seems to cause other problems, or at least I stumble upon other XML weirdness when I use my patch; so yours looks rather better ;-)

Things like this:

  GISCAN Pango-1.0.gir
/usr/opt/hgnome/src/pango/pango/pango-color-table.h:762: syntax error, unexpected identifier in '    guint16 name_offset;' at 'guint16'
/usr/opt/hgnome/src/pango/pango/pango-color-table.h:768: syntax error, unexpected identifier, expecting ',' or ';' in 'static const ColorEntry color_entries[] = {' at 'color_entries'
/usr/opt/hgnome/src/pango/pango/pango-language-sample-table.h:52: syntax error, unexpected identifier in 'LANGUAGE(' at 'LANGUAGE'
  GISCAN PangoFT2-1.0.gir
  GISCAN PangoXft-1.0.gir
Traceback (most recent call last):

+ Trace 222581

Sigh. Is it only me that suffers these extrem fragility problems, and/or gets nervous about this heap of python in the build process ? ;-)

Comment on attachment 164614 [details] [review]
Move pkg-config calls before --library and --program

Attachment 164614 [details] pushed as def3d91 - Move pkg-config calls before --library and --program

Altering the g-ir-scanner argument order has no effect on the ultimate libtool argument order it seems (sadly).

Pushed - with your fix too Colin.

(In reply to comment #14)
> Ho hum; well my 'fix' seems to cause other problems, or at least I stumble upon
> other XML weirdness when I use my patch; so yours looks rather better ;-)
> 
> Things like this:
> 
>   GISCAN Pango-1.0.gir
> /usr/opt/hgnome/src/pango/pango/pango-color-table.h:762: syntax error,
> unexpected identifier in '    guint16 name_offset;' at 'guint16'
> /usr/opt/hgnome/src/pango/pango/pango-color-table.h:768: syntax error,
> unexpected identifier, expecting ',' or ';' in 'static const ColorEntry
> color_entries[] = {' at 'color_entries'
> /usr/opt/hgnome/src/pango/pango/pango-language-sample-table.h:52: syntax error,
> unexpected identifier in 'LANGUAGE(' at 'LANGUAGE'
>   GISCAN PangoFT2-1.0.gir
>   GISCAN PangoXft-1.0.gir
> Traceback (most recent call last):
> 

This is the same problem, the xml generated by the tmp-introspect program is using libgirepository.so from /usr/lib and not from the prefix.

hmm, maybe I misunderstood something but don't you want .la files (built as part of your own package) *before* system libraries, topmost dependency first, so that the linker prefers the in-tree version instead of whichever old version is already installed?

or at least that was the right order in LIBADD/LDADD automake variables

(In reply to comment #18)
> hmm, maybe I misunderstood something but don't you want .la files (built as
> part of your own package) *before* system libraries, topmost dependency first,
> so that the linker prefers the in-tree version instead of whichever old version
> is already installed?
> 
> or at least that was the right order in LIBADD/LDADD automake variables

Yes - this patch appears to break things for me, because the generated binary has an RPATH which lists my $PREFIX first, and that means it picks up the old library.

I think the problem people are running into here that we actually need the library path order to be:

.:$BUILD_ROOT:/usr

We're not actually trying to link the generated binary against the $PREFIX libgirepository.  I'm going to try making a patch which reverts this one, and does that.

Created attachment 164648 [details] [review]
Revert "fix bug#605156 by ordering libtool archives that may pull in system"

This reverts commit e14bdebab725263c3d6f97267090cbf5d06b99d2.  While
that commit fixed things in some cases, it broke others.  I think the
problem people are running into here that we actually need the library
path order to be:

.:$BUILD_ROOT:/usr

Ensure that the middle happens by explicitly calling pkg-config.

Michael can you try this new patch?

Seems to work for me; it'd be good to get in [ given my breakage I guess ]. Ultimately, though I'm prodding the libtool maintainers - since it seems the underlying problem affects other things, and we can't even run simple apps like gtk-query-immodules-2.0 inside the build tree with system X server .la files with the world in it's current state - which is lamer than lame :-)

Attachment 164648 [details] pushed as 8c48a31 - Revert "fix bug#605156 by ordering libtool archives that may pull in system"

I think the .la files should be deleted at the RPM layer, don't ship them for system packages.

[Mass-moving gobject-introspection tickets to its own Bugzilla product - see bug 708029. Mass-filter your bugmail for this message: introspection20150207 ]