GNOME Bugzilla – Bug 691312
Support building from tarballs
Last modified: 2013-03-10 21:42:34 UTC
The main reason for this was building a better webkit, which is required to build epiphany master. I didn't want to patch webkitgtk-tarball-releases because it looks like a giant hack to me, and because it's against my religion to commit generated files in git. :) It turned out to be a nice cleanup, IMHO, and it allows to sync gnome-ostree with jhbuild on the lower layers, while also updating some of the github forks. Only build tested.
Created attachment 232926 [details] [review] ostbuild: introduce an object oriented abstraction for VCS utilities This will allow to support building from sources different than git.
Created attachment 232927 [details] [review] Split GIT specific code out of Vcs.Module This will make it possible to have modules that don't use GIT, and at the same time it restores 'local:' keytype support.
Created attachment 232929 [details] [review] Add support for building from tarballs Add a new module class that uses wget, tar, cp and patch and supports building from tarballs, without requiring git mirrors (that often become outdated).
Created attachment 232931 [details] [review] Automatically run autoreconf when the build system uses autoconf If no autogen.sh script is present, but the build system uses autoconf, we can run autoreconf ourselves.
Created attachment 232932 [details] [review] Switch to building from tarballs for certain modules For modules of which we don't want to do continuous integration, switch to building from tarballs, from the latest stable version, or the minimum required by GNOME modules.
(In reply to comment #0) > The main reason for this was building a better webkit, which is required to > build epiphany master. > I didn't want to patch webkitgtk-tarball-releases because it looks like a giant > hack to me, and because it's against my religion to commit generated files in > git. :) Yes, but... > It turned out to be a nice cleanup, IMHO, and it allows to sync gnome-ostree > with jhbuild on the lower layers, while also updating some of the github forks. > Only build tested. I see the jhbuild argument, but I had hoped that this would go the other way, and we'd switch jhbuild to use spidermonkey-tarballs-as-git for example. Anyways, it isn't just the continuous integration argument for git versus tarballs. * I used to have a script in ostbuild which could show a "git shortlog" or "git log" or "git diff" between any two source snapshots. This was actually quite cool. On the other hand it's true that with tarball imports the commits aren't useful anyways...you have pollution from generated "configure" scripts and the like. * While I haven't looked in detail at your patches, so far I feel like there's been a lot of code simplification from the "everything is a git repository" model. Think about stuff like how the build system can use "git clean -dfx". * Supporting tarballs takes off pressure from people who are git resisters (nss, I'm thinking of you). But I guess I was being too naive in thinking gnome-ostree would convince people like that =/ Hmm. What about a model where we accept tarballs as input, but auto-convert them to repositories locally? Named like "tarball-imports/$name". That way we get the space savings of git, and when you do an 'ostbuild checkout intltool', you know you have a git repository to work with, and you can at least use "git grep", "git clean" and stuff like that.
Review of attachment 232931 [details] [review]: Have you seen http://people.gnome.org/~walters/docs/build-api.txt ? I am leaning a bit towards saying we should patch the modules to include an autogen.sh. Probably a lot of them already have one, it's just missing from EXTRA_DIST.
(In reply to comment #7) > Review of attachment 232931 [details] [review]: > > Have you seen http://people.gnome.org/~walters/docs/build-api.txt ? > > I am leaning a bit towards saying we should patch the modules to include an > autogen.sh. Probably a lot of them already have one, it's just missing from > EXTRA_DIST. You can't, without git am. patch supports setting file modes only starting from 2.7.1 (Fedora 19), and it's buggy when it comes to new files. Btw, with tarballs you should be able to use the distributed configure. The problem is when you're patching .am files, so the real solution is to patch .in files instead and let ./configure && make && make install do the job without ever involving automake or autoconf. I'm not particularly keen on this patch, really, it was just a faster hack to get texinfo building.
(In reply to comment #8) > (In reply to comment #7) > > Review of attachment 232931 [details] [review] [details]: > > > > Have you seen http://people.gnome.org/~walters/docs/build-api.txt ? > > > > I am leaning a bit towards saying we should patch the modules to include an > > autogen.sh. Probably a lot of them already have one, it's just missing from > > EXTRA_DIST. > > You can't, without git am. patch supports setting file modes only starting from > 2.7.1 (Fedora 19), and it's buggy when it comes to new files. Hrm. Well, we could automatically make autogen.sh executable. A bit of a hack...but probably the least bad? > The > problem is when you're patching .am files, so the real solution is to patch .in > files instead and let ./configure && make && make install do the job without > ever involving automake or autoconf. Been there...usually I end up getting frustrated and just forcing autogen.sh/autoreconf. > I'm not particularly keen on this patch, really, it was just a faster hack to > get texinfo building. Right.
Just a note that I haven't forgotten about this, and if you don't get a chance to try the "convert tarballs to git internally" approach soon, I'll likely try doing it this weekend.
Well, I haven't tried the tarballs-to-git approach really, as I've been otherwise busy, but this bug probably needs a heavy rebase.
Ok, I ended up doing this a bit differently: https://git.gnome.org/browse/gnome-ostree/commit/?id=dfe6adbc877ec4dde5e5efc3ee3709308dae17fe Going to mark this as fixed, but we should probably at least move libical and libpwquality to tarballs too.