GNOME Bugzilla – Bug 100697
Problems with -pthread and shared libraries
Last modified: 2011-02-18 16:09:33 UTC
Just discovered that libgthread is not getting linked against -lpthread with GCC. Two problems: A) libtool doesn't pass thread the -pthread flag when linking libraries, though it does when linking executables. (You can force it to do so with -Wc,-pthread, but we can't put that into the `pkg-config --libs gthread-2.0` out.) B) Even if you do -pthread through to GCC, the current GCC specs don't use -lpthread when linking shared libraries on most platforms; e.g., the relevant specs portion for Linux is: *lib: %{shared: -lc} %{!shared: %{mieee-fp:-lieee} %{pthread:-lpthread} %{profile:-lc_p} %{!profile: -lc}} Both quite arguably bugs in the tools, but still something that we may have to deal with. Don't really have much of an idea about an answer yet, but I thought I'd file a bug to track the problem.
IIRC I came to the conclusion that this is a feature, not a bug. If shared libraries would be linked against libpthread, they would automatically load libpthread, when being loaded into some program self. But there (in general at least, not necessarily on linux) is more to multithreaded programs, than just being linked to libpthread. Thus the executable of a multithreaded program _has_ to be explictitly linked with -pthread to make sure the linker produces the right binary for a multithreaded program. Thats why having traces of libpthread in libgthtread.so or libgthtread.la is a bad thing. Close the bug, if you're content with that. Bug me further otherwise...
But the libgthread library really should be linked to -lpthread as well, at least on Linux, if not generally. The two reasons that were brought up are: - prelink; in order for prelink to preresolve relocations for a library must not have undefined symbols; any undefined symbols will always have to handled at run time. - Symbol versioning; the binding of symbols to particular versions happens when the library is linked. If -lpthread not on the link line, then the symbol will get the base version at run time, not the correct newer one. For a concrete example that happens currently with gthread, to quote from a mail by Jakub Jelinek: === The [...] symbol to look at is pthread_create, which is solely in libpthread, as GLIBC_2.0 and GLIBC_2.1 version. Unversioned pthread_create will resolve to pthread_create@@GLIBC_2.0, aka __pthread_create_2_0, which if attr != NULL makes its own copy of the attributes on the stack and resets to default guardsize, stack{addr{,_set},size}. === Also, note that on some platforms (like AIX) undefined symbols in a library just don't work at all. On these platforms -pthread probably works right with -shared, but the libtool problem is still an issue.
I filed two bugs. One for gcc: http://gcc.gnu.org/cgi-bin/gnatsweb.pl?database=gcc&cmd=view+audit-trail&pr=8888 and one for libtool: http://mail.gnu.org/pipermail/libtool-patches/2002-December/002274.html ----------- Do we really want to add a bandaid to that into configure.in? I agree, this problem isn't nice, but at least things should _work_, if not optimal (no prelinking, no new versions). I would rather not patch configure.in again. The whole threading stuff is already a real mess (but OTOH I don't think, it could be much cleaner due to the many different setups). If we now try to add -lsomething to the link lines, I'm afraid, things will get broken again, which might not be wise shortly before release.
For linux this is fixed in CVS gcc, as seen here. http://subversions.gnu.org/cgi-bin/viewcvs/gcc/gcc/gcc/config/linux.h I don't know, what I was smoking then, but I thought, I took the right link for the libtool bug report (which now points to some other message), but somehow the report completely disapeared. Have to send it again maybe.
Not really sure what way to go here; this has turned out to be a pretty big problem for GNOME packages, though it can be worked around; the problem is that people do things like: testprog_LIBS = $(top_builddir)/src/libmylibrary.la But libtool doesn't store -pthread in the .la file, so it doesn't get pulled when linking testprog. I don't think we can depend on people having newer-than-new libtool and gcc, especially considering that libtool is a bit undermaintained from my perspective. (I say that because I use a libtool with multiple patches to build the GLib/GTK+ tarballs and there has been no action on any of them.) We could make the problem go away for people on "typical" platforms pretty easily by doing: case $host in *-*-linux*) G_THREAD_LIBS_FOR_GTHREAD="`echo $G_THREAD_LIBS | sed s/-pthread/-lpthread/`" ;; *) G_THREAD_LIBS_FOR_GTHREAD="`echo $G_THREAD_LIBS | sed s/-pthread/-Wc,-pthread/`" ;; fi (Despite appearances, the above doesn't discount the fact that it might be -pthreads; we know it wont' be that for Linux and the second line should work fine.) And use G_THREAD_LIBS_FOR_GTHREAD in gthread/Makefile.am. That still leaves things broken for any plaform where the thread libraries have to be on the final compile line for the executable other than linux.
Created attachment 13042 [details] [review] Suggested hack as a patch
I've gone ahead and committed my hack now; I'm not entirely happy with the approach, but I don't want this issue to get in the way of people testing the rest of GLib/GTK+.
I'm really not happy with the hack either. The first thing to notice: Producing a multithreaded executable is in general more than just linking with libpthread.so. E.g. on BSD you have to use a different libc (namely libc_r). So every multithreaded program should explicitly link with -pthread (or whatever the appropriate option is). That is: every program should link with what `pkg-info --libs gthread-2.0` outputs. I saw, that some programs in GNOME CVS didn't properly link because of link errors. But the solution must be to add gthread-2.0 to the pkg-config dependencies of that package and fix the Makefiles. Not making libgthread.so pull in libpthread.so automatically. That might haunt us later big time. Also note, that fixing libtool to include -pthread in its dependency_libs line will not work, as libtool currently doesn't understand that in .la-files and any change here would make all old libtool versions unusable, and I doubt, that they would do it. So while I fully concur, that not passing -pthread is a bug for both libtool and gcc, it is as well a bug to link a multithreaded program without explicitly saying so. Alas now that hack is released. It's up to you. I just wanted to state my opinion.
Here's my reasoning in detail: Linking -lgthread-2.0 against -lpthread is important for reasons discussed above. To achieve that, we must: Fix GCC for Linux, by: a) Requiring people to patch their GCC's specs files b) Require as-yet-unreleased GCC 3.3 c) Pass -lpthread instead of -pthread when linking -lgthread-2.0 a) and b) aren't reasonable in general and would have the same effect as c) in the end anyways. Convince libtool to pass -pthread to GCC on platforms other than Linux, by: a) Adding another patch to the GLib/GTK+ "fork" of libtool b) Using -Wc,-pthread instead of -pthread The only advantage of a) is that it would add -pthread to dependency_libs in the generated .la files, which, as you say, old libtool's wouldn't understand anyways. Plus the exact details of the required changes weren't clear to me. So, that's the reasoning behind the changes. What's the effect on things "downstream". What you seem to be concerned about is that: libmylib_la_LIBADD = `pkg-config --libs gthread-2.0` mytestprog_LDADD = libmylib.la Now will work on Linux but not on some other platforms (that is, BSD). Given the necesary solib dependencies for libpthread, there is no way around this. Either BSD users will get the downstream programs fixed, or libtool will be fixed. I don't think it should be our guiding concern.
I see and am happy, that letting libpthread.so be pulled in automatically by libgthread.so is not the reason for this patch. So adding -lpthread is just an optimization (prelink, lib versioning) for linux (I would still argue, that it is not strictly necessary and would fully dissolve, once libtool and gcc is fixed.) But why doing case $host in .... *) G_THREAD_LIBS_FOR_GTHREAD="`echo $G_THREAD_LIBS | sed s/-pthread/-Wc,-pthread/`" ;; esac then. I wouldn't fix it, if it isn't broken on the other platforms, and we basically do not know much about the other platforms (ok, libtool is still broken, but we can patch it, its a small patch) So I'm content with the linux branch, but wouldn't like the other branch unless we really have evidence, that it is causing real problems.
For Linux it's not just an optimization... the symbol versioning part is actually a question of correctness. The old symbols might not match the current header files. But the need to link libraries against their dependencies is not Linux specific -- for example I doubt AIX will even work at all without the -Wc,-pthread change... undefined symbols in AIX shared libraries are generally not possible.
Ok, you convinced me. [But the second branch can be deleted, once the libtool version shipped with glib is fixed]
Putting on 2.4.0 - hopefully by then there will be a fixed libtool out.
The current libtool (1.5.2) does the right thing. Once we use it in GLib (which we currently do not), we can delete the lines *) G_THREAD_LIBS_FOR_GTHREAD="`echo $G_THREAD_LIBS | sed s/-pthread/-Wc,-pthread/`" ;; in configure.in and close this bug. I don't know, how much the GLib version of libtool is, so that's up to Owen to decide, when to upgrade. (The other hack for gcc < 3.3 will certainly live on forever....)
I'm using 1.5.2 to build 2.3.5 (we were at 1.5.0 before); I don't want to make this change immediately before I release 2.3.5 however.
2004-12-06 Matthias Clasen <mclasen@redhat.com> * configure.in: Remove a no longer needed hack for libtool < 1.5.2. (#100697, Owen Taylor, Sebastian Wilhelmi)