GNOME Bugzilla – Bug 653842
DESTDIR usage breaks installation of Pango, gdk-pixbuf, rsvg, etc
Last modified: 2011-07-13 15:02:40 UTC
Excerpted from gdk-pixbuf Makefile.am: RUN_QUERY_LOADER_TEST=test -z "$(DESTDIR)" install-data-hook: install-ms-lib install-def-file @if $(RUN_QUERY_LOADER_TEST) ; then \ $(mkinstalldirs) $(DESTDIR)$(libdir)/gdk-pixbuf-2.0/$(GDK_PIXBUF_BINARY_VERSION) ; \ $(top_builddir)/gdk-pixbuf/gdk-pixbuf-query-loaders > $(DESTDIR)$(libdir)/gdk-pixbuf-2.0/$(GDK_PIXBUF_BINARY_VERSION)/loaders.cache ; \ else \ echo "***" ; \ echo "*** Warning: loaders.cache not built" ; \ echo "***" ; \ echo "*** Generate this file manually on host" ; \ echo "*** system using gdk-pixbuf-query-loaders" ; \ echo "***" ; \ fi Same exact thing occurs in Pango. In rsvg, you'll find: install-data-hook: if test -z "$(DESTDIR)"; then \ $(mkinstalldirs) $(DESTDIR)$(gdk_pixbuf_binarydir) ; \ $(GDK_PIXBUF_QUERYLOADERS) > $(DESTDIR)$(gdk_pixbuf_cache_file) ; \ fi In gnome-icon-theme-standard, we have: install-data-hook: if test -z "$(DESTDIR)" ; then \ $(GTK_UPDATE_ICON_CACHE) -q $(DESTDIR)$(themedir); \ fi Because it of course doesn't make sense to update the icon cache if we aren't putting anything there. Basically, the DESTDIR doesn't work without some sort of "post" mechanism to handle stuff that has to be done after putting files into their final location. Maybe this is just hardcoding a bunch of rules into the Python code for all the things that we want to install. Maybe it's: <trigger directory="<....>"> <exec command=""> </trigger> rule in the moduleset. I don't know. Need to figure this or revert the DESTDIR stuff temporarily, since it's generating lots of obscure problems that most potential builders of GNOME aren't prepared to handle.
*** Bug 653965 has been marked as a duplicate of this bug. ***
Created attachment 191332 [details] [review] Add post-installation scripts Several things in the GNOME stack have file-based caches, like gdk-pixbuf's loaders.cache. These need to be regenerated, and historically that worked in jhbuild because these modules include bits in their Makefile to run then relevant commands, if DESTDIR is not set. Package managers have post-installation scripts in packages, and OS builders typically support this as well. Jhbuild historically relied on the post-installation scripts in modules, but in (see bug 647231) we switched to using DESTDIR for all module types. So we need to move closer to the package model here. This patch adds a proof-of-concept post-installation script for gdk-pixbuf loaders.
Created attachment 191349 [details] [review] Add post-installation trigger scripts Several things in the GNOME stack have file-based caches, like gdk-pixbuf's loaders.cache. These need to be regenerated, and historically that worked in jhbuild because these modules include bits in their Makefile to run then relevant commands, if DESTDIR is not set. Package managers have post-installation scripts in packages, and OS builders typically support this as well. Jhbuild historically relied on the post-installation scripts in modules, but in (see bug 647231) we switched to using DESTDIR for all module types. So we need to move closer to the package model here. We call them "triggers" as they're run after any kind of build operation (including "uninstall"). This patch adds a proof-of-concept trigger script for gdk-pixbuf loaders.
Created attachment 191350 [details] [review] triggers: Add pango module trigger
Does it really need to match on regexp? can't it match on directory/file? $libdir/gdk-pixbuf-2.0/2.10.0/loaders/ would be more readable imho. Also I ponder mentioning required triggers in the moduleset, to avoid "your jhbuild was too old, please pull a new version to get new triggers"; just like modulesets are taken live from git.gnome.org nowadays. Going that way, perhaps the trigger should be part of the moduleset itself, like adding such a snippet to the gdk-pixbuf module: <trigger filename="$libdir/gdk-pixbuf…"><command>gdk-pixbuf-query-loaders …</command></trigger>.
(In reply to comment #5) > Does it really need to match on regexp? can't it match on directory/file? > $libdir/gdk-pixbuf-2.0/2.10.0/loaders/ would be more readable imho. Sure, I guess. I'll keep the current REMatch key, and add a new LiteralMatch. > Also I ponder mentioning required triggers in the moduleset, to avoid "your > jhbuild was too old, please pull a new version to get new triggers"; just like > modulesets are taken live from git.gnome.org nowadays. Hmmmm. I can't see us introducing modules which have triggers too often. There are only ~3 in the stack right now. > Going that way, perhaps the trigger should be part of the moduleset itself, > like adding such a snippet to the gdk-pixbuf module: > <trigger filename="$libdir/gdk-pixbuf…"><command>gdk-pixbuf-query-loaders > …</command></trigger>. Urgh =/ Shell script in XML will get ugly (like redirections > and <).
(In reply to comment #6) > > Going that way, perhaps the trigger should be part of the moduleset itself, > > like adding such a snippet to the gdk-pixbuf module: > > <trigger filename="$libdir/gdk-pixbuf…"><command>gdk-pixbuf-query-loaders > > …</command></trigger>. > > Urgh =/ Shell script in XML will get ugly (like redirections > and <). Well, it's no worse than escaping dots in regular expressions :) and there's CDATA section. Really I'd like triggers not to be tied to jhbuild sources, like patches that can live next to a remote moduleset (so it may not be <command> but <script>triggers/gdk-pixbuf</script>, that would be found relatively to the moduleset).
Created attachment 191372 [details] [review] Add post-installation trigger scripts * Load from jhbuild install prefix (PKGDATADIR), not self.config.prefix * Support LiteralMatch in addition to REMatch
Created attachment 191373 [details] [review] pango.trigger: New trigger for pango modules Reattaching for ordering
Created attachment 191374 [details] [review] glib.trigger: New trigger for GSettings schemas
Created attachment 191376 [details] [review] pango.trigger: New trigger for pango modules Fix installation directory
Created attachment 191377 [details] [review] gtk+.trigger: New gtk+ trigger for icon cache
Review of attachment 191372 [details] [review]: ::: triggers/README @@ +5,3 @@ +A .trigger file is a set of regular expressions which are evaluated +against files installed after any "make install DESTDIR=", combined +with a shell script to run if any of them match. Elsewhere you say that the triggers are also run on uninstall - what gives ? Also, it would be good to be a bit more specific as to what the matches match against here - paths of files or directories that are created / deleted / modified ? any of the above, or just the first ? Is the literal match a substring match ? or a match thats restricted to path components ? What exactly is JHBUILD_PREFIX set to ? the prefix where jhbuild is installed, or the installation prefix ? Also, it is probably worth mentioning that a trigger for module A can be run for an install of module B - or is that not the case ? I think it would be required for this to make sense...
(In reply to comment #14) > > Elsewhere you say that the triggers are also run on uninstall - what gives ? Well...they're more run on any change to the manifest which contains files that match the lines. What we should probably change is on uninstall, look at both the *old* manifest and the *new* one. And we want this anyways to implement the whole rationale for DESTDIR in the first place, which was to remove no-longer-shipped files on upgrades. > Also, it would be good to be a bit more specific as to what the matches match > against here - paths of files or directories that are created / deleted / > modified ? any of the above, or just the first ? Is the literal match a > substring match ? or a match thats restricted to path components ? I'll do so. > What exactly is JHBUILD_PREFIX set to ? the prefix where jhbuild is installed, > or the installation prefix ? That was added in bug 653048. > Also, it is probably worth mentioning that a trigger for module A can be run > for an install of module B - or is that not the case ? I think it would be > required for this to make sense... Yep, it is.
One issue matthias raised that is important to fix is the case of partial builds. If say we're *not* building glib, we still need to run the *system* glib-compile-schemas if the module ships schemas. I think the easiest way to handle this is to drop the current logic associating a trigger to a module. Basically, we just: test -x glib-compile-schemas && ... So we transparently pick up the system one if we have it, otherwise we use the jhbuild one. Note that we really should be adding dependencies in our moduleset where triggers may be used. For example as matthias said, gnome-icon-theme doesn't depend on gtk+, but it probably should.
(In reply to comment #16) > > test -x glib-compile-schemas && ... Of course, this requires your shell fragment to be simplistic enough to be treated this way. Alternatively, you could add the moral equivalent of a TryExec key...
*** Bug 653977 has been marked as a duplicate of this bug. ***
Colin, any update on this ?
Created attachment 191639 [details] [review] Add post-installation trigger scripts * Correctly run after each module build, instead of after a whole operation * Support IfExecutable * Update README a bit
Created attachment 191640 [details] [review] pango.trigger: New trigger for pango modules Use IfExecutable
Created attachment 191641 [details] [review] glib.trigger: New trigger for GSettings schemas Use IfExecutable
Created attachment 191642 [details] [review] gtk+.trigger: New gtk+ trigger for icon cache Use IfExecutable
Attachment 191639 [details] pushed as c350169 - Add post-installation trigger scripts Attachment 191640 [details] pushed as d3809d1 - pango.trigger: New trigger for pango modules Attachment 191641 [details] pushed as a79c568 - glib.trigger: New trigger for GSettings schemas Attachment 191642 [details] pushed as aa84ad1 - gtk+.trigger: New gtk+ trigger for icon cache
I get a crash at buildscript.py:run_triggers because PKGDATADIR is None. (with 'jhbuild buildone gnome-games')
+ Trace 227748
all_triggers = trigger.load_all(os.path.join(PKGDATADIR, 'triggers'))
elif path == '' or path.endswith('/'):
(In reply to comment #25) > I get a crash at buildscript.py:run_triggers because PKGDATADIR is None. > (with 'jhbuild buildone gnome-games') Fixed, but please open *new* bugs for things like this. See bug 654555.