After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 707379 - rework commit objects
rework commit objects
Status: RESOLVED FIXED
Product: ostree
Classification: Infrastructure
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: OSTree maintainer(s)
OSTree maintainer(s)
Depends on:
Blocks:
 
 
Reported: 2013-09-03 12:54 UTC by Colin Walters
Modified: 2013-09-09 21:12 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
core: Add detached metadata, readd metadata to commits (18.40 KB, patch)
2013-09-05 12:57 UTC, Colin Walters
committed Details | Review

Description Colin Walters 2013-09-03 12:54:35 UTC
From the TODO:

- Drop the a{sv} per https://bugzilla.gnome.org/show_bug.cgi?id=673012
- Optional non-object metadata; e.g., "detached" GPG signatures which
   are in the same file (to avoid double HTTP requests)
- Extended validation; SHA384+SHA256 checksum of all metadata along
  with content object metadata (file size in particular) wouldn't take
  too much extra time per commit, and would greatly strengthen resistance
  to active hash collision attacks.
- Total size of all objects
- Largest object size: Prevent DoS via filling up the disk.  The client
  can ensure it never writes an object larger than this, and combined
  with total size should act as an effective cap.


Something like this:

commit = header content

header = guint64 <a(say)>       # 64 bit flags, plus mappings of strings to data
content = header <ay> <ay> <ay> # verified headers, plus parent, dirmeta, dirtree

Then we need known verified headers:

ts = timestamp, varint seconds since the epoch
parent = checksum, parent comm
log = freeform UTF-8 text
total-size = varint total number of bytes for all objects
max-object-size = size of largest object
sizes = mapping of checksum -> size for each object?

And known detached headers:

gpgsigs = [array of detached gpg signatures]
Comment 1 Colin Walters 2013-09-03 13:00:15 UTC
Implementation thoughts:

Create a new object type COMMIT2 ?  Except...we only have one checksum per ref.  Which brings us back to basically just breaking binary compatibility =/
Comment 2 Colin Walters 2013-09-03 13:09:47 UTC
Ok, new plan here:

Work around bug 673012 by simply having *one* key in the a{sv} which is

metadata -> ay

And then the ay contains our "header" above.  Hopefully we can avoid adding any indexing to GVariant for the case where there's only one key.

Then fit all of the rest of the stuff into our new metadata format.
Comment 3 Colin Walters 2013-09-04 19:05:13 UTC
For stuff like GPG signatures, we could add a generic "detached metadata".  This would just be an a{sv}, stored in the FS as .commit-metadata.

It'd probably only contain GPG signatures, since...most other things you would want to have protected by the signature.  But maybe say cached data like .sizes don't need to be GPG signed, and can be computed after the fact.
Comment 4 Colin Walters 2013-09-05 12:57:02 UTC
Created attachment 254178 [details] [review]
core: Add detached metadata, readd metadata to commits

Previously I thought we'd have to ditch the current commit
format to avoid a{sv} due to

See https://bugzilla.gnome.org/show_bug.cgi?id=673012

But I realized that we don't really have to care about
unpacking/repacking commit objects, so let's just re-expose the
existing metadata a{sv} in commits in the API.

Also, add support for "detached" metadata that can be updated at any
time post-commit.  This is specifically designed for GPG signatures.
Comment 5 Colin Walters 2013-09-09 21:12:18 UTC
Attachment 254178 [details] pushed as ac2d61d - core: Add detached metadata, readd metadata to commits