GNOME Bugzilla – Bug 743127
packagedb: move to using separate files for package info
Last modified: 2015-02-01 11:28:20 UTC
The main motivation here is in order to facilitate packing up of a single (installed) package as a .tar file that can be "reinstalled" simply by unpacking the .tar file again. The .tar file can simply contain the info/pkgname and manifest/pkgname files along with all of the files listed in the manifest. After unpacking, it is only necessary to rerun the triggers. An added advantage of this approach is that it vastly simplifies the code, removing the need for locking and cache consistency checks, with very little added overhead.
Created attachment 294793 [details] [review] packagedb: move from_xml method Earlier patches put methods in an illogical order. Fix that up so that the to/from xml functions are together at the bottom. Sorry for the noise.
Created attachment 294794 [details] [review] fileutils: add safe mkdir_with_parents() helper Add a GLib-style mkdir_with_parents() helper that doesn't throw exceptions in the case that the directory already existed. Replace some racy code in packagedb with a call to it.
Created attachment 294795 [details] [review] packagedb: don't keep/pass around manifests_dir Store 'dirname' as the toplevel packagedb directory instead of storing the manifests_dir. Pass dirname into the individual package objects and let them do the path pasting for themselves. Also: hold off on actually creating the manifests directory until we actually go to write a file there. This removes all logic about the location of the 'manifests' directory from the toplevel PackageDB object.
Created attachment 294796 [details] [review] packagedb: store a separate 'info' file Store a separate XML file for each package in the info/ subdirectory. This will allow the entire content of a package to be easily turned into a .tar file (since we don't have to deal with splitting out the separate information from the packagedb).
Created attachment 294797 [details] [review] packagedb: remove __main__ sniplet at end of file This code fragment supported dumping out the contents of the packagedb if the file was run directly. Remove it, because that's about to stop working.
Created attachment 294798 [details] [review] packagedb: use info/ files as authoritative source Move over to using the individual files in info/ as an authoritative source for package information, falling back to the packagedb.xml file only in the case that we can't find such a file. We no longer update the packagedb.xml file. In order to ensure that we don't detect stale packagedb.xml entries as installed packages, ensure that we only fall back to reading an entry from that file if we are missing an info/ file but have a manifests/ file. This commits removes caching of the packages in the _entries variable and therefore also removes the code for statting in order to see if the cache needs to be updated. Since we no longer write to the file, we also remove the need for locking.
Created attachment 294799 [details] [review] utils/lockfile: remove unused file Remove the now-unused utils/lockfile.py.
Created attachment 294805 [details] [review] packagedb: store relative paths in manifest This change does not actually touch any packagedb code, but rather changes its users to assume that the 'manifest' property is populated with relative pathnames. All readers pass the list through fileutils.filter_files_by_prefix() so modify that to take either paths relative to the given prefix or absolute paths (which will be filtered to limit them to the prefix). The code in process_install() is the only writer, so simply modify it to use relative paths. Although much of the software that jhbuild builds does not like to have its install prefix moved (due to hardcoding of paths), this change means that jhbuild itself theoretically has no problem with it. It also makes the process of generating tar files using the manifest slightly cleaner.
Fine.
Attachment 294793 [details] pushed as a440e54 - packagedb: move from_xml method Attachment 294794 [details] pushed as 1080d09 - fileutils: add safe mkdir_with_parents() helper Attachment 294795 [details] pushed as d5931ca - packagedb: don't keep/pass around manifests_dir Attachment 294796 [details] pushed as 6312ea0 - packagedb: store a separate 'info' file Attachment 294797 [details] pushed as ee89ba0 - packagedb: remove __main__ sniplet at end of file Attachment 294798 [details] pushed as 10789b5 - packagedb: use info/ files as authoritative source Attachment 294799 [details] pushed as b345bbb - utils/lockfile: remove unused file Attachment 294805 [details] pushed as 4b838a4 - packagedb: store relative paths in manifest Live to you from the GNOME booth at FOSDEM :)