GNOME Bugzilla – Bug 652475
use srcdir != builddir by default
Last modified: 2016-05-31 13:51:17 UTC
Very very often, when build fails with weird errors, 'make distclean' isn't enough, while 'git clean -fdx' is. This command is hard to find out by yourself, and saves a lot of bandwidth compared with "Wipe everything and start over". So I think it should be suggested in the list of actions when a build fails. It will probably save much time to GNOME developers that don't know about this trick, or don't think of it when encountering errors (I know I often try to fix absurd bugs for minutes until I think of it ;-). One could even think it could be useful to have a 'jhbuild clean --hard' option that would run 'git clean -fdx' when you're tired of getting many errors. Or even an option to try that solution every time a module fails to build. This kind of thing can really help new contributors that keep fighting with the stack instead of doing real work!
Thank you for the report. I like the idea. I'll implement when I can. Patches welcome too.
The dangerous thing about this is that if you're actively hacking on any of the modules, and you didn't "git add" the relevant files, they'll be lost forever. My two cents on this is that the best fix is to use srcdir != builddir by default, and change the "distclean" target to simply: "rm -rf _build" (almost the same amount of cleaning, but much safer). Thus, I'm hijacking this bug =)
Rationale for srcdir != builddir by default: * Debian is using it for their multi-architecture builds * For people who believe in "make distcheck" (not me, but that's an aside), it's something we need to support anyways, so we should test it more * It's the harder case to think about, so if it works, it's more likely that srcdir == builddir works too * We no longer need to maintain .gitignore very much (this is a big one) * We can get rid of the "distclean" target in favor of "rm -rf _build" * We can support mutiple builds more easily (as a developer, I often use/test both unoptimized builds (gcc -g -O0) as well as optimized (gcc -g -O2); with this it's much easier to do both (even simultaneously!) ) Concerns: * Every existing jhbuild checkout will need to be rebuilt (autoconf refuses to run if the srcdir is configured too). This is a one-time cost. * Developer transition - people will have to get used to "cd _build; make". This is probably the biggest issue, but I think in the big picture, this cost is totally worth the benefits above.
Of course that would be nice, but I think git -fdx can be useful in other cases: for examples, we had problems with gdk-pixbuf PO files that were present on disk, but added later in git, leading git pull to fail. In that case, instread of "Wipe everything", and additional option "Hard clean" would help. That doesn't mean the removal of builddir shoulnd't replace the "Distclean" option.
(In reply to comment #4) > Of course that would be nice, but I think git -fdx can be useful in other > cases: for examples, we had problems with gdk-pixbuf PO files that were present > on disk, but added later in git, leading git pull to fail. The gdk-pixbuf problem is that *something* in the normal "make" build is causing "make update-po" to run, which modifies all the .po files. git pull then fails. git clean -dfx doesn't help this situation; git reset --hard origin/master will, which is quite a different thing. The thing to fix here is gdk-pixbuf's build, since only it is doing that. I tried but failed to figure it out =(
(In reply to comment #2) > The dangerous thing about this is that if you're actively hacking on any of the > modules, and you didn't "git add" the relevant files, they'll be lost forever. I always use git clean -dfX which avoids this problem. It assumes a properly maintained .gitignore, otherwise it might not clean up everything, but keeping that up-to-date is a good thing anyway, in my opinion. > My two cents on this is that the best fix is to use srcdir != builddir by > default, and change the "distclean" target to simply: "rm -rf _build" (almost > the same amount of cleaning, but much safer). As far as I know, autotools supports srcdir != builddir builds only when building from dist tarballs, so I'm not sure whether that's really usable for jhbuild. Otherwise you'll run into issues with files that are generated and distributed as they are expected to exist in srcdir.
(In reply to comment #6) > (In reply to comment #2) > > The dangerous thing about this is that if you're actively hacking on any of the > > modules, and you didn't "git add" the relevant files, they'll be lost forever. > > I always use git clean -dfX which avoids this problem. It assumes a properly > maintained .gitignore, otherwise it might not clean up everything, but keeping > that up-to-date is a good thing anyway, in my opinion. I didn't know about -X, that is useful. That is indeed probably safe enough to use from jhbuild. > As far as I know, autotools supports srcdir != builddir builds only when > building from dist tarballs, so I'm not sure whether that's really usable for > jhbuild. Otherwise you'll run into issues with files that are generated and > distributed as they are expected to exist in srcdir. This is a good point; I noticed something like this in the context of bison/yacc. So it looks like with my current work on this in gobject-introspection, automake puts the generated scannerparser.c in the toplevel directory in the tarball, as would happen for a srcdir == builddir build, which is fine; the build rules look there. So this issue merits more investigation; but I wouldn't categorize it yet as a blocker. We may be able to easily work around the cases where it appears. For jhbuild, I think the motivations of: 1) Supporting both optimized and unoptimized builds 2) Being able to rm -rf _build (in fact, do this by default) remain.
See: https://mail.gnome.org/archives/desktop-devel-list/2011-July/msg00088.html For diagnosis and possible fixes for the gdk-pixbuf problem without brute force.
Created attachment 324585 [details] [review] Default to builddir != srcdir Enable a separate buildroot under XDG_CACHE_HOME. We need to ensure that building GNOME with jhbuild exercises the same build and code paths as building GNOME with Continuous. We also need to ensure that modules distcheck more easily. See discussion on desktop-devel-list: https://mail.gnome.org/archives/desktop-devel-list/2016-February/msg00070.html
New discussion on d-d-l: https://mail.gnome.org/archives/desktop-devel-list/2016-February/msg00070.html Summary: everyone was in favor of the change of default. I've been running builds with buildroot != None for a while, and most everything works out of the box, with the exception of some Vala projects, in some cases. For those, I suggested we use "supports-non-srcdir-builds" in the moduleset, like we already do.
Can we please get this in, once and for all?
Review of attachment 324585 [details] [review]: Lets merge this now that we are in the beginning of the cycle
Thanks, Javier. Attachment 324585 [details] pushed as 000f820 - Default to builddir != srcdir