After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 742298 - various improvements for packagedb
various improvements for packagedb
Status: RESOLVED FIXED
Product: jhbuild
Classification: Infrastructure
Component: general
unspecified
Other All
: Normal normal
: ---
Assigned To: Jhbuild maintainers
Jhbuild QA
Depends on:
Blocks:
 
 
Reported: 2015-01-04 04:27 UTC by Allison Karlitskaya (desrt)
Modified: 2015-01-04 23:22 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
packagedb: properly terminate manifest files (993 bytes, patch)
2015-01-04 04:27 UTC, Allison Karlitskaya (desrt)
committed Details | Review
packagedb: remove ancient compat code (1.37 KB, patch)
2015-01-04 04:27 UTC, Allison Karlitskaya (desrt)
committed Details | Review
packagedb: write only one manifest (2.69 KB, patch)
2015-01-04 04:27 UTC, Allison Karlitskaya (desrt)
committed Details | Review
packagedb: remove manifest files on uninstall (1.33 KB, patch)
2015-01-04 04:28 UTC, Allison Karlitskaya (desrt)
reviewed Details | Review
packagedb: move safe file writing to fileutils (3.52 KB, patch)
2015-01-04 04:28 UTC, Allison Karlitskaya (desrt)
committed Details | Review
packagedb: remove unused parameter (1.20 KB, patch)
2015-01-04 04:28 UTC, Allison Karlitskaya (desrt)
committed Details | Review
packagedb: remove manifest files on uninstall (1.87 KB, patch)
2015-01-04 20:26 UTC, Allison Karlitskaya (desrt)
committed Details | Review

Description Allison Karlitskaya (desrt) 2015-01-04 04:27:48 UTC
See patches.
Comment 1 Allison Karlitskaya (desrt) 2015-01-04 04:27:50 UTC
Created attachment 293690 [details] [review]
packagedb: properly terminate manifest files

Manifest files are missing the final newline.  Add it.
Comment 2 Allison Karlitskaya (desrt) 2015-01-04 04:27:54 UTC
Created attachment 293691 [details] [review]
packagedb: remove ancient compat code

Nobody stores manifests in .xml anymore.
Comment 3 Allison Karlitskaya (desrt) 2015-01-04 04:27:57 UTC
Created attachment 293692 [details] [review]
packagedb: write only one manifest

Because of the fact that the 'manifest' attribute on PackageEntry items
is a property and because of the fact that we check it each time we
convert a node to XML, we end up reading every manifest file every time
we go to update the packagedb (ie: every time we install a package).

Even worse: because self.manifest is now populated, we also write every
single manifest file each time we update the packagedb.

Avoid the problem by making the write explicit for the module being
added, and no other.
Comment 4 Allison Karlitskaya (desrt) 2015-01-04 04:28:00 UTC
Created attachment 293693 [details] [review]
packagedb: remove manifest files on uninstall

These have been left dangling.  Make sure we delete them.
Comment 5 Allison Karlitskaya (desrt) 2015-01-04 04:28:04 UTC
Created attachment 293694 [details] [review]
packagedb: move safe file writing to fileutils

Add a SafeWriter class to fileutils that handles the usual dance of
creating a tmpfile, writing to it, flushing it, fsyncing it, closing it
and renaming it.

Use it from the two locations that were doing this in packagedb.
Comment 6 Allison Karlitskaya (desrt) 2015-01-04 04:28:07 UTC
Created attachment 293695 [details] [review]
packagedb: remove unused parameter
Comment 7 Colin Walters 2015-01-04 13:47:44 UTC
Review of attachment 293690 [details] [review]:

Ok.
Comment 8 Colin Walters 2015-01-04 13:52:40 UTC
Review of attachment 293691 [details] [review]:

I'd like to see some at least very brief attempt at backing the assertion in the commit message; something like "The manifests were switched to external files ~3.5 years ago in bug 655417, and we expect jhbuild users to stay close to git master".
Comment 9 Colin Walters 2015-01-04 13:54:34 UTC
Review of attachment 293692 [details] [review]:

Looks fine to me.
Comment 10 Colin Walters 2015-01-04 13:57:04 UTC
Review of attachment 293693 [details] [review]:

::: jhbuild/utils/packagedb.py
@@ +106,3 @@
 
+    def remove(self):
+        os.unlink(os.path.join(self.manifests_dir, self.package))

Would prefer a bit if we handled the ENOENT case, so we don't traceback if we're interrupted.
Comment 11 Colin Walters 2015-01-04 13:59:35 UTC
Review of attachment 293694 [details] [review]:

Definitely a nice cleanup.
Comment 12 Colin Walters 2015-01-04 13:59:51 UTC
Review of attachment 293695 [details] [review]:

OK.
Comment 13 Allison Karlitskaya (desrt) 2015-01-04 20:26:17 UTC
Created attachment 293724 [details] [review]
packagedb: remove manifest files on uninstall

These have been left dangling.  Make sure we delete them.

Add a ensure_unlinked() function to fileutils to help with the job.
Comment 14 Colin Walters 2015-01-04 21:46:25 UTC
Review of attachment 293724 [details] [review]:

Yep.
Comment 15 Allison Karlitskaya (desrt) 2015-01-04 23:22:26 UTC
Attachment 293690 [details] pushed as 6686953 - packagedb: properly terminate manifest files
Attachment 293691 [details] pushed as e385291 - packagedb: remove ancient compat code
Attachment 293692 [details] pushed as c2904cd - packagedb: write only one manifest
Attachment 293694 [details] pushed as 62a7097 - packagedb: move safe file writing to fileutils
Attachment 293695 [details] pushed as a83b728 - packagedb: remove unused parameter
Attachment 293724 [details] pushed as 5dd4fd9 - packagedb: remove manifest files on uninstall