GNOME Bugzilla – Bug 568842
Cache versioning
Last modified: 2015-02-07 16:59:08 UTC
Current gobject-introspection uses $HOME/.cache/g-ir-scanner/ to persist information between invocations. However, this cache is not necessarily compatible between different g-ir-scanner versions. This means that multiple versions may attempt to use the cache in incompatible ways, causing unexpected results. We encounter this in our buildbot environment where we simultaneously test different versions of gobject-introspection and the scanner throws an exception due to the format of the cache. Thanks.
Owen suggested on irc that we should use the mtime of the relevant .py files so we don't have to manually bump the cache version when something changes. A quick check shows that we should base the cache version on the mtime of the following files: - ast.py - transformer.py - girparser.py And the following to play safe: - g-ir-scanner - cachestore.py (just to be safe) - config.py - sourcescanner.py
Created attachment 163309 [details] [review] [cachestore] Add versioning Version the cache by checking the SHA1 of the content of all python source files. If the SHA1 hash differs, just regenerate the cache.
Review of attachment 163309 [details] [review]: ::: giscanner/cachestore.py @@ +34,3 @@ +_CACHE_VERSION_FILENAME = '.cache-version' + +import giscanner Note here the cache is adding extra I/O and CPU time to reread and digest the python sources. If we're going to keep this, I'd prefer we generated the SHA1 at build time and substituted it into config.py from the Makefile. @@ +78,3 @@ + def _check_cache_version(self): + + self._check_cache_version() Leftover debug line?
(In reply to comment #3) > Review of attachment 163309 [details] [review]: > > ::: giscanner/cachestore.py > @@ +34,3 @@ > +_CACHE_VERSION_FILENAME = '.cache-version' > + > +import giscanner > > Note here the cache is adding extra I/O and CPU time to reread and digest the > python sources. I thought the same, so I measured that, it takes about 10ms on my box (3 years old core2). Shouldn't be a problem in practice. It's about 2ms when using mtime, but that is not as correct as using the content. If IO is a problem we could switch over to using .pyc files which are already paged in (or will be shortly) > If we're going to keep this, I'd prefer we generated the SHA1 at build time and > substituted it into config.py from the Makefile. There's no build-time for python, so you might run into this problem while developing without re-running make or configure
The following fix has been pushed: e472079 [cachestore] Add versioning
Created attachment 163319 [details] [review] [cachestore] Add versioning Version the cache by checking the SHA1 of the content of all python source files. If the SHA1 hash differs, just regenerate the cache.
[Mass-moving gobject-introspection tickets to its own Bugzilla product - see bug 708029. Mass-filter your bugmail for this message: introspection20150207 ]