GNOME Bugzilla – Bug 474278
OS X builds need to pad library headers for possible rewrites
Last modified: 2013-05-03 20:33:37 UTC
It will be some time before its reasonable to expect users to have Gtk-OSX pre-installed when installing an application that uses GTK. As a result, most such applications will be bundling Gtk-OSX with the app, and this can require the use of an OS X tool that rewrites shared library headers before packing them into an application "bundle". Under some circumstances, the libraries can end up with a header that doesn't have enough spare room to contain the result of the the rewrite. This can be avoided by adding -Xlinker -headerpad -Xlinker 2048 to the linker commands, typically via libtool_opts in the GTK Makefile.am's. 2048 is just a guess at the required size - I've run into this problem with several libraries, including libgtk and some of my own, and 2048 seems to have been big enough to solve the problem. Without this, its not possible to reliably package libgtk or other libraries up as a bundle. Note that this applies to other GNOME libraries as well.
Can't application bundles specify "the lib is in this bundle too" instead of having to rewrite the explicit path to it? Otherwise this sounds like it would (should have been) be a long-standing, wide-spread, and not-gtk-specific problem, since many programs include many libraries.
Its not that simple. In theory, the app could include a plist file specifying a DYLD_LIBRARY_PATH (equivalent and supplemental to LD_LIBRARY_PATH) that pointed inside the session. However, specifying it there has two problems: a) its too late - its set for the environment of the exec'd application, but not for the exec(2) call, so the runtime linker won't find the required libraries b) Apple has disabled several aspects of specifying runtime linker search paths in the latest release of OS X for security reasons. The only alternative is to include a helper script, which is what every linux->OS X port so far has done following the GIMP's example. In that case, the "app" is just a stub shell script that sets some environment variables and then execs the real thing. This is precisely to deal with the differences in the way the runtime linker works on linux versus OS X. I found this to be a kludge, and I also found what I consider to be a better solution that doesn't leverage a script to fix this. It does, however, require that every library and executable inside the bundle have its Mach-O header rewritten to reflect the intra-bundle location of the libraries it depends on (which is easy to arrange). I want to stress that I don't like the way Apple has designed their bundle system. Before I started working with it, I had assumed that apps would automatically find libraries within the bundle first, before any search was done outside the bundle (though there are security implications there). I also think that building the *full path* of a library dependency into an application was an idea that went out with SunOS. However, this *is* how OS X works, and it would be nice if GTK and its dependency stack would be helpful in accomodating a non-kludge solution.
"@executable_path" is the standard absolute-path way a relocatable .app bundle directs dyld to "look in this bundle". % otool -L /Applications/AbiWord.app/Contents/MacOS/AbiWord @executable_path/../Frameworks/png.framework/Versions/1.2/png (compatibility version 1.0.0, current version 1.0.0) @executable_path/../Frameworks/wv.framework/Versions/1.0/wv (compatibility version 1.0.0, current version 1.0.0) @executable_path/../Frameworks/enchant.framework/Versions/A/enchant (compatibility version 1.0.0, current version 1.0.0) @executable_path/../Frameworks/fribidi.framework/Versions/A/fribidi (compatibility version 1.0.0, current version 1.0.0) @executable_path/../Frameworks/popt.framework/Versions/1.6/popt (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.0.0) /System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 9.0.0) @executable_path/../Frameworks/libglib-2.0.dylib (compatibility version 1.0.0, current version 1.6.0) @executable_path/../Frameworks/libgmodule-2.0.dylib (compatibility version 1.0.0, current version 1.6.0) @executable_path/../Frameworks/libgthread-2.0.dylib (compatibility version 1.0.0, current version 1.6.0) @executable_path/../Frameworks/libiconv.2.dylib (compatibility version 5.0.0, current version 5.0.0) @executable_path/../Frameworks/libintl.1.dylib (compatibility version 2.0.0, current version 2.1.0) @executable_path/../Frameworks/libxml2.2.dylib (compatibility version 8.0.0, current version 8.4.0) @executable_path/../Frameworks/libgobject-2.0.dylib (compatibility version 1.0.0, current version 1.6.0) @executable_path/../Frameworks/libgsf-1.1.dylib (compatibility version 13.0.0, current version 13.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.1.4) % otool -L /Applications/AbiWord.app/Contents/Frameworks/libgobject-2.0.dylib /Applications/AbiWord.app/Contents/Frameworks/libgobject-2.0.dylib: @executable_path/../Frameworks/libgobject-2.0.dylib (compatibility version 1.0.0, current version 1.6.0) @executable_path/../Frameworks/libglib-2.0.0.dylib (compatibility version 1.0.0, current version 1.6.0) @executable_path/../Frameworks/libintl.1.dylib (compatibility version 2.0.0, current version 2.1.0) @executable_path/../Frameworks/libiconv.2.dylib (compatibility version 3.0.0, current version 3.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 63.0.0)
grr. you say this as if somehow I don't know this. try running otool -L on the libraries built as part of a regular GTK build, either from jhbuild or a conventional tarball build. they do NOT include @executable_path For example, this is what the jbhuild does to glib: otool -L /opt/gtk/lib/libglib-2.0.0.dylib /opt/gtk/lib/libglib-2.0.0.dylib: /opt/gtk/lib/libglib-2.0.0.dylib (compatibility version 1401.0.0, current version 1401.1.0) /usr/lib/libiconv.2.dylib (compatibility version 5.0.0, current version 5.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.9) /opt/gtk/lib/libintl.8.dylib (compatibility version 9.0.0, current version 9.1.0) /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon (compatibility version 2.0.0, current version 128.0.0) Anything that is linked against this library will specify that it requires "/opt/gtk/lib/libglib-2.0.0.dylib", NOT "libglib-2.0.0.dylib" as would be common on Linux. The whole point of what I am asking for is to as to be able to rewrite the headers to alter the contents to specify @executable_path (including the library name itself (the first line in the output above). For most libraries, doing the rewrite works just fine without any effort, but in recent builds of GTK (and it may be semi-random, depending on other factors during the build), the app that does the rewrite will report that there isn't enough room in the header to do the rewrite. Hence my request for explicit header-padding so that this can be done without a complete recompile and hacking the Makefiles to make the header big enough. Then the resulting libraries can be included in the bundle just like the ones you quoted from Abiword. BTW, I'm glad to see that at least Abiword has apparently packaged stuff "the right way" rather than use stub script "helper".
Everything you said made it appear you had no clue that the encoded dyld links could be relative. And you certainly know that a "regular" build of *anything* won't do non-standard stuff like this relative link game. I haven't tried it yet, but does the --libdir flag allow you to specify these sorts of @executable_path locations? I don't think doing this by default is a good idea (especially if the chosen value is chosen as being "arbitrary but large enough in one test", not based on a reason why/whether it's "right". If one uses a prefix that is at least as long as whatever your install_name_tool script will change it to, isn't that enough? Or can you pass those -Xlinker flags via LDFLAGS when you configure/build the libraries?
By all means looks like a libtool bug/feature-request.
The point of this bug filing is not to request that the build rewrites the headers. The request is that the build makes it possible to rewrite the headers without futzing with the Makefiles and doing a rebuild. Behdad, nothing needs to be added to libtool to make this happen, but I guess that one could (mistakenly, IMHO) view libtool as the nexus of "how to build a shared library" within the GNOME stack. In which case, yes, I guess its true that adding --headerpad N to libtool would make sense. However, that would still need to be invoked somehow, either by default in the Makefiles, or via a ./configure option or LDFLAGS. I have to admit that I have not had much luck propagating my settings of LDFLAGS into a build since I started using jhbuild ...
(In reply to comment #7) > The point of this bug filing is not to request that the build rewrites the > headers. The request is that the build makes it possible to rewrite the headers > without futzing with the Makefiles and doing a rebuild. I would be more positive if it *was* about rewriting the headers, if that's the right way to build usable shared libs. > Behdad, nothing needs to be added to libtool to make this happen, but I guess > that one could (mistakenly, IMHO) view libtool as the nexus of "how to build a > shared library" within the GNOME stack. In which case, yes, I guess its true > that adding --headerpad N to libtool would make sense. However, that would > still need to be invoked somehow, either by default in the Makefiles, or via a > ./configure option or LDFLAGS. I have to admit that I have not had much luck > propagating my settings of LDFLAGS into a build since I started using jhbuild > ... Libtool *is* the "how to build a shared library" indeed. If we can't agree on that, I'm not sure we can reach any agreement. What is libtool in your opinion then, if not a layer to hide platform-specific magic?
ok, i will agree (for the sake of argument) that libtool is there to hide platform-specific magic. that implies that somebody needs to add --flag-to-set-header-part-of-shared-libs-so-that-it-can-be-rewritten to libtool, as a distinct task. should it take a size argument? i don't know. does this apply to any platform other than OS X? i don't know. but even when that is done, there is a separate distinct task of providing a way to have this option invoked when building GTK/Quartz. As I mentioned earlier, I haven't had much success at using either LDFLAGS or LIBTOOL_EXPORT_OPTIONS to push extra arguments to libtool.
Sounds like this has coalesced into a libtool usage problem. You'll certainly want to try using -Wl,-FOOBAR instead of just -FOOBAR to get an LDFLAGS passed all the way through to the linker via libtool.
The flag to pass is '-Wl,-headerpad_max_install_names'.