After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 729585 - higher level metadata
higher level metadata
Status: RESOLVED FIXED
Product: ostree
Classification: Infrastructure
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: OSTree maintainer(s)
OSTree maintainer(s)
Depends on:
Blocks:
 
 
Reported: 2014-05-05 17:13 UTC by Colin Walters
Modified: 2014-09-07 16:59 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
fetcher: Close request body on error paths (1.06 KB, patch)
2014-08-20 16:14 UTC, Colin Walters
committed Details | Review
Bump GIO dependency to 2.38 for GTask (1.52 KB, patch)
2014-08-20 16:14 UTC, Colin Walters
committed Details | Review
Add repository "summary" file and metalink support (56.52 KB, patch)
2014-08-20 16:15 UTC, Colin Walters
committed Details | Review

Description Colin Walters 2014-05-05 17:13:27 UTC
* Branches should be able to have a textual description

* Allow enumerating available branches
Comment 1 Colin Walters 2014-05-16 08:57:53 UTC
Also, if we support tags (bug 727564) it seems clear to me that we should also allow enumerating the available tags.
Comment 2 Colin Walters 2014-08-04 21:15:12 UTC
For 729388 we are going to need a single file which contains at a minimum the branch names and the commits they point to, if we want to support chaining like this:

* Retrieve metalink via TLS, resolves to target url + checksum
* Retrieve target URL via HTTP (or ftp), verify checksum
Comment 3 Colin Walters 2014-08-04 21:15:22 UTC
bug 729388
Comment 4 Colin Walters 2014-08-06 13:39:42 UTC
Ok, let's make this a generic property bag of a{sv}.  That will maximize our future flexibility.

Let's call it the repository "summary".  It will appear as repo/summary.

Additionally, it should support inline GPG signatures.  We can detect whether the content starts with ----BEGIN PGP...---- and automatically verify/unwrap it.

Example code: https://github.com/wgwoods/fedup/blob/master/fedup/download.py#L366
Which is basically using gpgme under the hood.

Do we want to support detached signatures as well for this file?

The a{sv} will contain at least the following keys:

* commits: a(ayay) - array of pairs of commit checksum to commit - this is an optimization for when multiple refs point to the same commit.  Commits are in sorted order by checksum, to allow bsearch.
* refs: a{say} - mapping from name to commit id
Comment 5 Colin Walters 2014-08-20 16:14:56 UTC
Created attachment 283979 [details] [review]
fetcher: Close request body on error paths

Otherwise, we're potentially holding up subsequent requests.

I was hitting this when testing the metalink code, where we want to
continue doing more fetches after hitting a 404.
Comment 6 Colin Walters 2014-08-20 16:14:59 UTC
Created attachment 283980 [details] [review]
Bump GIO dependency to 2.38 for GTask

I'd like to use GTask, so bump the dependency.
Comment 7 Colin Walters 2014-08-20 16:15:03 UTC
Created attachment 283981 [details] [review]
Add repository "summary" file and metalink support

For Fedora and potentially other distributions which use globally
distributed mirrors, metalink is a popular solution to redirect
clients to a dynamic set of mirrors.

In order to make metalink work though, it needs *one* file which can
be checksummed.  (Well, potentially we could explode all refs into the
metalink.xml, but that would be a lot more invasive, and a bit weird
as we'd end up checksumming the checksum file).

This commit adds a new command:

$ ostree summary -u

To regenerate the summary file.  Can only be run by one process at a
time.

After that's done, the metalink can be generated based on it, and the
client fetch code will parse and load it.

TODO:
 - Decide whether this file can contain an inline signature, or should
   be duplicated
 - Add size of commit object
Comment 8 Colin Walters 2014-08-21 16:40:23 UTC
Also, we should use the size metadata to limit request sizes smaller than the 10MiB limit.
Comment 9 Colin Walters 2014-08-21 16:51:59 UTC
Also, we should obey maxconnections.
Comment 10 Colin Walters 2014-08-27 18:00:55 UTC
Comment on attachment 283979 [details] [review]
fetcher: Close request body on error paths

Attachment 283979 [details] pushed as c08a450 - fetcher: Close request body on error paths
Comment 11 Colin Walters 2014-09-03 17:24:15 UTC
(In reply to comment #9)
> Also, we should obey maxconnections.

Though...that element was designed for package systems; let's see if people object to us being more aggressive by default.  If they do, we can change things.
Comment 12 Colin Walters 2014-09-07 16:58:58 UTC
James approved this on IRC.

Attachment 283980 [details] pushed as 3571418 - Bump GIO dependency to 2.38 for GTask
Attachment 283981 [details] pushed as f8f5da2 - Add repository "summary" file and metalink support