GNOME Bugzilla – Bug 776216
Problems executing binaries on older systems than on which they were built
Last modified: 2016-12-23 19:23:42 UTC
Commit https://github.com/jralls/gtk-osx-build/commit/ec0aea57aebe5e57c1bb78566cecb92bdbf5782d makes "target" of setup_sdk() equal to the OS X version where they are built. After looking at the implementation of setup_sdk(), it appears the only thing this parameter does is that it's used for "-mmacosx-version-min=" which determines the oldest OS X version where the generated binary can be run. In my case this means that when I build on OS X 12.12, users on older systems cannot execute the generated binaries, see the problem reported in https://github.com/geany/geany/issues/1335 (skip the first half related to code-signature which is irrelevant to this and go to the comment mentioning the crash resulting to EXC_BAD_INSTRUCTION) After manually changing the value to "10.9", the user is able to run the binary without any problems. Defaulting to the latest OS X doesn't sound like a good idea to me as quite a significant portion of users are on older systems. The previous values with "10.7" for any newer system made much more sense to me, I'm not quite sure what was wrong with that.
The default is for building "native", meaning with the current running system's SDK and targeting the current running system's version. That default is aimed at someone who wants to build a package for local use, which is the general case. Developers packaging their products for other users are expected to configure the build as necessary for the older versions that they want to support. Note that SDKs tend to work only with the version of Xcode that they're shipped with, especially since Xcode 6, so in order to build with an older SDK you need to also use the corresponding Xcode. They're available at developer.apple.com/downloads. 10.9 is magic because that's when Apple introduced a new linker that works with a new style of dylib stubs, also introduced in 10.9. Before that the dylib stubs indicated the dylib version which the app/library was built with and the linker would refuse to load if the actual dylib was older than that. That meant that you had to build against the SDK for the oldest version of MacOS X that you intended to support. It's my understanding that that still holds for versions older than 10.9, but I haven't yet tested it.
(In reply to John Ralls from comment #1) > The default is for building "native", meaning with the current running > system's SDK and targeting the current running system's version. That > default is aimed at someone who wants to build a package for local use, > which is the general case. > > Developers packaging their products for other users are expected to > configure the build as necessary for the older versions that they want to > support. Note that SDKs tend to work only with the version of Xcode that > they're shipped with, especially since Xcode 6, so in order to build with an > older SDK you need to also use the corresponding Xcode. They're available at > developer.apple.com/downloads. > > 10.9 is magic because that's when Apple introduced a new linker that works > with a new style of dylib stubs, also introduced in 10.9. Before that the > dylib stubs indicated the dylib version which the app/library was built with > and the linker would refuse to load if the actual dylib was older than that. > That meant that you had to build against the SDK for the oldest version of > MacOS X that you intended to support. It's my understanding that that still > holds for versions older than 10.9, but I haven't yet tested it. Are you sure about that? That sounds really strange and from my experience at least on iOS with which I have most experience this has never been the case - you can create binaries with newer SDK which work with older OS versions. Also when you create a new project in Xcode (latest 8.2 version), it lets you specify deployment target of the project up to 10.6 so the thing about 10.9 being the earliest version that allows this doesn't seem right.
Yes, I'm sure. There's a huge difference between linking frameworks (which have all sorts of backwards-compatible magic in them) and dylibs (which didn't until 10.9). Many of the Gtk stack's dependencies are dylibs, while Xcode's GUI is oriented towards frameworks and Objective-C/Swift (which adds another layer of backwards-compatibility magic).
OK, strange though... I'll try to install older OS X versions into a VM and see how they behave (the oldest one I have access to from the App Store is 10.8).
I have older VMs (and computers) already set up if you want to try to build with, say, target=10.7, SDK=native and attach it here. Do I understand correctly that target=10.9, SDK=native built on 10.12 does work on a 10.9 machine (virtual or not)? I've *still* not actually tested that myself.
(In reply to John Ralls from comment #5) > I have older VMs (and computers) already set up if you want to try to build > with, say, target=10.7, SDK=native and attach it here. Great, that would be a time saver. This one is built with target 10.7 on OS X 12.12. https://www.dropbox.com/s/tweb5yhs0eal8is/geany-1.29_osx.dmg?dl=0 > > Do I understand correctly that target=10.9, SDK=native built on 10.12 does > work on a 10.9 machine (virtual or not)? I've *still* not actually tested > that myself. I had only 10.10.5 OS X installed inside a VM (similarly to the user who opened the bug on Github for Geany) and it worked both on the VM I had and for the user too.
Also if you want to compare the results with other targets used during build, there are some more binaries to try: target 10.9: http://download.geany.org/snapshots/geany-1.29_osx_nosign.dmg target 10.12: http://download.geany.org/geany-1.29_osx.dmg all of them built on OS X 10.12.
Cool. I tested the 10.7 one on a 10.7 VM, seems to work fine. Tried it on a 10.6 VM just for yuks and it failed with a link error (unknown symbol in libpixman). So it seems that the new dylib stubs are backward-compatible at least as far as 10.7 and perhaps 10.6. That makes building backward-compatible bundles much easier, especially since it avoids using the buggy early versions of llvm. Time to update the wiki. I still think that native/native is the correct default to pass to setup_sdk, so I'm inclined to close this as "not a bug".
(In reply to John Ralls from comment #8) > Cool. I tested the 10.7 one on a 10.7 VM, seems to work fine. Tried it on a > 10.6 VM just for yuks and it failed with a link error (unknown symbol in > libpixman). > So it seems that the new dylib stubs are backward-compatible at least as far > as 10.7 and perhaps 10.6. That makes building backward-compatible bundles > much easier, especially since it avoids using the buggy early versions of > llvm. Time to update the wiki. Great! Just out of curiosity, I built Geany with target 10.6 here https://www.dropbox.com/s/60adq49lx073q0c/geany-1.29_osx_target_10_6.dmg?dl=0 Would you try if it works on OS X 10.6? To me 10.6 seems like the oldest OS X version worth supporting so it would be nice if it worked there too. > > I still think that native/native is the correct default to pass to > setup_sdk, so I'm inclined to close this as "not a bug". OK. But it would be also good to update the comment in .jhbuildrc-custom - the only information regarding target which is there now is # Target can be "10.4", "10.5", or "10.6". It sets # MACOS_DEPLOYMENT_TARGET and the -macosx-version-min CFLAG. ... # Set up a particular target and SDK: For default operation, set the # architecture and SDK for the native machine: The comment should make it clearer what consequences setting target to the native machine has.
The 10.6 bundle worked after I changed LSMinimumSystemVersion to 10.6 in Info.plist. I see no reason to amplify "sets... -macosx_version_min", that seems pretty obvious. If it isn't then I'd expect you to Google it yourself. Note as well that in the actual documentation on wiki.gnome.org is a section https://wiki.gnome.org/Projects/GTK%2B/OSX/Building#Building_on_Older_Versions_of_Mac_OS_X which in turn points to https://wiki.gnome.org/Projects/GTK%2B/OSX/OSX%20Version%20Notes That page needs some serious rework, particularly that matching target and SDK isn't necessary for Xcode6 and later.
(In reply to John Ralls from comment #10) > The 10.6 bundle worked after I changed LSMinimumSystemVersion to 10.6 in > Info.plist. Ah, yeah, forgot about that. Good to know it works, thanks for testing! > > I see no reason to amplify "sets... -macosx_version_min", that seems pretty > obvious. If it isn't then I'd expect you to Google it yourself. > > Note as well that in the actual documentation on wiki.gnome.org is a section > https://wiki.gnome.org/Projects/GTK%2B/OSX/ > Building#Building_on_Older_Versions_of_Mac_OS_X which in turn points to > https://wiki.gnome.org/Projects/GTK%2B/OSX/OSX%20Version%20Notes > That page needs some serious rework, particularly that matching target and > SDK isn't necessary for Xcode6 and later. The trouble is that when one first starts using gtk-osx, he indeed finds this page https://wiki.gnome.org/Projects/GTK%2B/OSX/Building and does things based on what's there. And there's not a single word about that one should adjust setup_sdk() in .jhbuildrc-custom. Section "Building on Older Versions of Mac OS X" doesn't apply to someone like me who is building on the latest OS X with the latest XCode. I'm not saying it because of myself, after all this discussion I know what needs to be updated, but for new users it isn't obvious at all from the documentation IMO. Finally, the whole problem from this bug report came from the fact that the defaults got changed in quite an important way (I actually reinstalled my build environment so my original .jhbuildrc-custom got replaced by the new one and I didn't notice there was such a fundamental change in it). Before one could build binaries for reasonably old OS X versions by default. This isn't the case any more and if it's going to stay this way, it should be documented IMO.
>Before one could build binaries for reasonably old OS X versions by default. This >isn't the case any more and if it's going to stay this way, it should be documented >IMO. That was bug 766337. It was always supposed to be the default to build for native, native as the comment above says (unchanged since I wrote setup_sdk() in 2009). >I actually reinstalled my build environment so my original .jhbuildrc-custom got >replaced by the new one and I didn't notice there was such a fundamental change >in it). Then either you deleted your old one yourself or there's a bug in the install script, it's supposed to leave an existing .jhbuildrc-custom in place. The example is exactly that, an example to get you started. > Section "Building on Older Versions of Mac OS X" doesn't apply to someone like me > who is building on the latest OS X with the latest XCode. Except that it does, it just should be "Building for Older Versions of Mac OS X". The section says "Gtk-OSX is maintained mostly for the current stable release of Mac OS X and Xcode, but it can be used to build applications which run on older systems as well. For particular tricks and tweaks for making this work, read OSX Version Notes. " Repeat: "but it can be used to build applications which run on older systems as well." Pretty clear, IMO. Anyway, the wiki page is overdue for a rewrite anyway, even more so in light of actually proving that the linker works at least as far back as 10.6. That frees people of the need to keep old SDKs and Xcode compilers around and means that for the most part "Building for Older Systems" can go away and be replaced with a one-liner, "Change the target passed to setup_sdk()". All of the tricks and tips are obsolete for anyone who's on at least 10.10.
(In reply to John Ralls from comment #12) > >Before one could build binaries for reasonably old OS X versions by default. This > >isn't the case any more and if it's going to stay this way, it should be documented > >IMO. > > That was bug 766337. Which I think was opened just because of https://github.com/jralls/gtk-osx-build/commit/371fd2f61d334ae2a9290c4ea331b1b899915a30 which got fixed by https://bugzilla.gnome.org/show_bug.cgi?id=776219 I think it would build correctly for the user now even with the previous values inside .jhbuildrc-custom. > It was always supposed to be the default to build for > native, native as the comment above says (unchanged since I wrote > setup_sdk() in 2009). Alright. But when I saw the code my impression was it meant "use 10.7 for all newer systems and older targets only if on older systems" which kind of made sense. > > >I actually reinstalled my build environment so my original .jhbuildrc-custom got > >replaced by the new one and I didn't notice there was such a fundamental change > >in it). > Then either you deleted your old one yourself or there's a bug in the > install script, it's supposed to leave an existing .jhbuildrc-custom in > place. The example is exactly that, an example to get you started. There's not bug in the install script, I really wiped out my account for building and made install from scratch - but assuming 10.7 was the target used for binaries, I didn't check its contents. > > > Section "Building on Older Versions of Mac OS X" doesn't apply to someone like me > > who is building on the latest OS X with the latest XCode. > > Except that it does, it just should be "Building for Older Versions of Mac > OS X". The section says > "Gtk-OSX is maintained mostly for the current stable release of Mac OS X and > Xcode, but it can be used to build applications which run on older systems > as well. For particular tricks and tweaks for making this work, read OSX > Version Notes. " > > Repeat: "but it can be used to build applications which run on older systems > as well." Pretty clear, IMO. > > Anyway, the wiki page is overdue for a rewrite anyway, even more so in light > of actually proving that the linker works at least as far back as 10.6. That > frees people of the need to keep old SDKs and Xcode compilers around and > means that for the most part "Building for Older Systems" can go away and be > replaced with a one-liner, "Change the target passed to setup_sdk()". All of > the tricks and tips are obsolete for anyone who's on at least 10.10. Yep, totally agree, the page deserves an update. Feel free to close this bug, maybe another bug should be opened requesting the wiki page update.
I've updated the wiki with more details about ~/.jhbuildrc, ~/.jhbuildrc-custom, and calling setup_sdk(). I'll close this, but if you have comments on the changes you can still post them here.
Great, that's much better, thanks! One last thing I can think of which should be mentioned somewhere is that by default the build is made without any optimization flags. There is setup_release() and setup_debug() in .jhbuildrc but these are not mentioned in the default .jhbuildrc-custom neither on the wiki. As people who want to make binary releases probably want to enable optimizations, these should be mentioned somewhere: there could be a comment in .jhbuildrc-custom with commented-out setup_release() below saying something like "By default, no optimization flags are used. If you want to enable them, comment-out the line below to pass the -O2 option to the compiler". Apart from that no more suggestions. I wrote Geany build instructions here https://github.com/geany/geany-osx so you might take a look if something I mention there should be also described on the wiki. But I think all the reset is covered.
OK, done, thanks.