GNOME Bugzilla – Bug 667788
Reduce circular build dependy on pkg-config
Last modified: 2012-02-26 13:37:35 UTC
Created attachment 205111 [details] [PATCH] Reduce circular build dependy on pkg-config This has been sitting in my tree for a while - the underlying problem is that new pkg-config depends on glib, and glib uses pkg-config. It turns out that glib really doesn't need pkg-config (cf patch). The only mess left is gtk-doc and I haven't found round tuits for that... From the patch commit log: glib trivially uses pkg-config to look for: - zlib - libffi - pcre - optionally dbus-1 for the serialization test The zlib use was superfluous as the correct checks were already in place. libffi is a standard library check - my replacement looks slightly more involved, just to make the environment variable overrides LIBFFI_{CFLAGS,LIBS} available. Bug 640261 which introduced the check for PCRE 8.11 is symptomatic. The bug showed that glib needs a PCRE which defines PCRE_ERROR_SHORTUTF8 to compile. Instead of simply testing for the existence of that define as per this patch, the version number was looked up and pkg-config introduced. Given the use of dbus-1 - just to run a serialization test - I didn't try too hard, especially as dbus-1 spreads its includedirs around, but I'm sure that someone keen on running the serialization test can trivially, e.g., ./configure DBUS1_CFLAGS="-I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/includ The reason this patch reduces but doesn't remove the circular (unnecessary) dependency on pkg-config is due to glib's hard dependency on gtk-doc, just made much worse by gtk-doc commit a81a1746, c.f., bug 649269, but that is the subject of a separate patch.
No. We want to use pkg-config more, not less. Cyclic build deps are an annoyance, but a fact of life.
Why do you want to use pkg-config more? In my ideal world it doesn't exist, and sensible m4 files are written instead. (I don't understand why pkg-config is good.)
The cyclic dependency is trivially worked around at configure time by the person doing the compilation. As someone who has worked at building distributions, specifically cross-compiling sandboxed ones, I can say that more m4 macros are *not* the solution you are looking for.
you mean those same m4 macros that made the configure.ac size balloon out of control? your ideal world strongly resembles hell; and though you may wish to subject yourself to pain and torment, it doesn't seem to me that us sane people should be strung along. circular dependencies are a fact of life; you can set CFLAGS and LIBS when building pkg-config - it's not exactly rocket science.
Can you explain why? It is precisely the cross-compile case in which m4 tests are likely to give the correct answer without the need to fiddle paths and environment variables.
We had a bit of a collision - comment 5 goes with comment 3. As to comment 4, you don't have all of glib in a single source file, so why have all of configure.ac in a single source file? (If size is a problem.) If you have m4 problems, please feel free to pass them on to me :-) (resisting the urge to rant) I am still interested in knowing why "more pkg-config" is good.
(In reply to comment #0) > The only mess left is gtk-doc and I haven't found round tuits for that... Note I recently made a gtk-doc-stub module that you can use to bootstrap a build with no documentation: http://git.gnome.org/browse/gtk-doc-stub I'm using it myself to temporarily cut the circular build dependency too.
As far as your patch, I think the core problem is it punishes people who *do* have pkg-config already. Your patch appears to hardcode e.g. -ldbus-1, but: $ pkg-config --libs dbus-1 -L/lib64 -ldbus-1 -lpthread -lrt There are two things we should do: 1) Make it easy to get a copy of some of the GLib source to be embedded in one's project. If you just want some data structures like GList this shouldn't be too hard. 2) Change pkg-config (again) to use a copy of the glib subset it needs, but have a sane process to sync up the sources. We could also eventually use this to replace Plymouth's random reimplementation of glib data structures too. However I could imagine taking a patch like the one you have as long as it's like: AC_PATH_PROG([PKG_CONFIG], [pkg-config]) if test -z "$PKG_CONFIG"; then # hand crafted checks go here else # use pkg-config here fi The problem we'll hit is that the hand crafted checks are likely to bitrot, which is why I really lean more towards making pkg-config not depend on glib at build time by default.
(In reply to comment #8) > 2) Change pkg-config (again) to use a copy of the glib subset it needs, but > have a sane process to sync up the sources. or just port pkg-config to python...
(In reply to comment #8) > 1) Make it easy to get a copy of some of the GLib source to be embedded in > one's project. If you just want some data structures like GList this shouldn't > be too hard. I wonder how this would even look like. > 2) Change pkg-config (again) to use a copy of the glib subset it needs, but > have a sane process to sync up the sources. for this, libnih and eglib already fill the ecological niche of "small subset of GLib to embed into another project". in the beginning is easy; then distributions start adding patches to allow using the system copy to minimize security issues and we're back to square one.
More Pkg-config is good because a) it gives the same API to get the information across different libraries. You just need to look at the situation from the late 90s, early 2000s before pkg-config was fairly ubiquitous, where each m4 macro was named something different, did things subtley different and broke nearly every other week. b) pkg-config is easy to use in 99.99% of the cases. Yes, you've found the one case where it's a bit weird but I'd argue that this is a bug in pkg-config (and agree with Colin in comment 8), and should be fixed there, not fixed by penalising every user of pkg-config c) pkg-config can be used outside of the configure script: gcc -o gtk-app gtk-app.c `pkg-config --cflags --libs gtk+-2.0` I also dislike your insinuations that the developers are not able to use m4. m4 macros were common 10 years ago, the developers saw the myriad of problems with them and pkg-config was created to solve those problems and has solved those problems. The developers know how to fix m4 issues when they arise, it is the normal users who have the problems, or are you going to be the 24hr help desk for all m4 issues? We need to make writing a configure.ac file easier, not harder and looking at your patch where every removal of a single pkg-config line is replaced with 10 lines of arcane looking script, you have failed in this goal.
(In reply to comment #8) > As far as your patch, I think the core problem is it punishes people who *do* > have pkg-config already. In what way? > Your patch appears to hardcode e.g. -ldbus-1, but: > > $ pkg-config --libs dbus-1 > -L/lib64 -ldbus-1 -lpthread -lrt Which is why I wrote above: glib trivially uses pkg-config to look for: - optionally dbus-1 for the serialization test Given the use of dbus-1 - just to run a serialization test - I didn't try too hard, especially as dbus-1 spreads its includedirs around, but I'm sure that someone keen on running the serialization test can trivially, e.g., ./configure DBUS1_CFLAGS="-I/usr/include/dbus-1.0 -I/usr/lib/dbus-1.0/includ dbus is *only* used in the test, not in glib as such. I say "I didn't try too hard" precisely because of this. dbus is optional. If writing code to find DBUS correctly is all that it takes to remove "WONTFIX", stay tuned! > The problem we'll hit is that the hand crafted checks are likely to bitrot, I really wouldn't call them (pcre, ffi, zlib) hand crafted. There is nothing special about them.
(In reply to comment #11) Reordering to: > a) it gives the same API to get the information across different libraries. You > just need to look at the situation from the late 90s, early 2000s before > pkg-config was fairly ubiquitous, where each m4 macro was named something > different, did things subtley different and broke nearly every other week. > I also dislike your insinuations that the developers are not able to use m4. m4 > macros were common 10 years ago, the developers saw the myriad of problems with > them and pkg-config was created to solve those problems and has solved those > problems. The developers know how to fix m4 issues when they arise, it is the > normal users who have the problems, or are you going to be the 24hr help desk > for all m4 issues? because I think they go together. I would like to be the 24hr help desk for all your m4 issues. Please take me up on the offer! What I have experienced over the last twenty years has been sending patches (in general, not glib) which fixed problems in configure scripts which would then simply sit in bug trackers. Developers get commit access based on their skill in coding the project, not by writing configure scripts. So it is easier to say "I don't understand, so can't review" (fair enough) than take the risk that some unknown reporter did the right thing. That way the configure scripts simply get a bad name as the brokeness doesn't get fix, and alternatives get reinvented. (An example is how many configure scripts test for GCC and if it is false, then assume that we are on solaris with sun pro. That is clearly wrong, is not an autotools problem, but it gets copied by humans.) (I had resisted ranting above but so be it - another historical one is firefox sticking with autoconf 2.13 - even though eg some systems had converted to elf long ago and even "old" versions of the autotools already new this.) The situation in glib seems rather better, especially with Colin being involved, so I thought it is worth writing patches which fix things :-) > b) pkg-config is easy to use in 99.99% of the cases. Yes, you've found the one > case where it's a bit weird but I'd argue that this is a bug in pkg-config (and > agree with Colin in comment 8), and should be fixed there, not fixed by > penalising every user of pkg-config I still don't see how I am penalising users of pkg-config but not calling it from the configure.ac script. I am particularly interested in one of the comments above which seems to suggest that having pkg-config makes cross compiling easier. I am sure the opposite is true - feel free to educate me... > c) pkg-config can be used outside of the configure script: gcc -o gtk-app > gtk-app.c `pkg-config --cflags --libs gtk+-2.0` Please carry on using it outside of the configure script :-) That is also an easy case: you are compiling on your machine for your machine. > We need to make writing a configure.ac file easier, not harder and looking at > your patch where every removal of a single pkg-config line is replaced with 10 > lines of arcane looking script, you have failed in this goal. The "arcane" was just keeping the Some influential environment variables: LIBFFI_CFLAGS C compiler flags for libffi LIBFFI_LIBS linker flags for libffi ability you have grown accustomed to from pkgconfig. They can be removed if you don't find them useful!
(In reply to comment #13) > > I would like to be the 24hr help desk for all your m4 issues. Please take me up > on the offer! Heh. > I am particularly interested in one of the comments above which seems to > suggest that having pkg-config makes cross compiling easier. I am sure the > opposite is true - feel free to educate me... Yeah, pkg-config isn't a help for cross building over just AC_CHECK_LIB, but neither does it hurt - iain is right that the main value is the consistency it brings to the stack. I'm not totally opposed to this patch but the main issue I see is that before, we cared about what was in e.g. zlib.pc, now we no longer do. If one of those dependency pkg-config files get updated, just in newer versions of glib we don't pick up those updates. Granted the current zlib.pc and pcre.pc files are basically just -I includedir and -L libdir, maybe that's unlikely to change, so it's not a problem. So...this patch would probably look less ugly if we kept the check for libdbus-1.pc, and you could pass e.g. --disable-modular-tests after bug 667806 has landed. One other thing: - PCRE_REQUIRES=libpcre - AC_SUBST(PCRE_REQUIRES) That will break static linking right? glib-2.0.pc.in:Requires.private: @PCRE_REQUIRES@
Created attachment 208440 [details] [review] Second attempt, putting the Requires.private: libpcre back As before but: - don't remove the PCRE_REQUIRES which I mistakenly removed last time - by popular demand leave pkg-config for the optional dbus (rather than sticking to flags as suggested first time around)