GNOME Bugzilla – Bug 756266
Enhance size metadata functionality
Last modified: 2018-08-17 18:58:40 UTC
Currently there is no code in ostree to parse the size index even though there is code to generate it. I have some patches to add a function to read this data to the libostree api and also a command (currently named size-summary) to print it on the command line.
Created attachment 312928 [details] [review] commit: Store objtype in size index If the objtype is not stored in the size index, the index will be ambiguous if the size of anything other than regular files is stored. This adds the objtype in a single byte following the existing data, ensuring that code designed to read the old format will continue to work with this addition.
Created attachment 312929 [details] [review] tests: Fix sizes test to check objtype Test the new objtype field of the size index.
Created attachment 312930 [details] [review] commit: Use 64 bit types for size metadata
Created attachment 312931 [details] [review] commit: Store sizes for all object types Since the metadata can account for a non-negligible amount of size, add size index entries for other types of objects as well.
Created attachment 312932 [details] [review] commit: Implement ostree_repo_get_commit_sizes to read size index There is currently no code to read the size index. Add a function ostree_repo_get_commit_sizes to read the total and new size of the objects in a given commit in terms of both archived size (i.e. download size) and unpacked size (i.e. on disk size).
Created attachment 312933 [details] [review] Implement the size-summary command size-summary prints information about the download size, on disk size, and number of files necessary to pull a given branch.
I haven't looked at your patches here yet in depth, but in the big picture - do you agree that the static deltas will obsolete this?
I don't think that static deltas will obsolete this. I agree that static deltas will handle the common case, but it doesn't seem feasible to generate static deltas from every existing old version every time a new version is released. The common case can be handled by static deltas, but not the long tail of really old versions being upgraded to latest. Also, how do I determine the size of a static delta accurately before it's downloaded? I see that ostree_repo_static_delta_generate computes it, but I don't see it being stored anywhere. Were you hoping for some code to store and read that data?
Picking this back up again, and sorry about the delay. Nowadays it's better to submit patches to github BTW. Since we've talked, the core ostree now displays the static delta sizes: https://github.com/ostreedev/ostree/commit/4beb5f4eaf619f1c47762bb872099e24f3efe2f0 As far as covering deltas everywhere - one thing that the delta *design* supports (although the code isn't really tested), is delta recursion. Basically let's say you have 5 commits. v0 v1 v2 v3 v4 The idea is we generate a delta tree, like: v0-v1 v1-v2 v2-v3 v3-v4 Then: v0-v2 v2-v4 And finally: v0-v4 - v0-v2 - v2-v4 It's also be possible of course for v0-v2 to simply recurse v0-v1 v1-v2. But while I haven't done any real world math on this I suspect for most uses having a binary tree would work well.
*** Bug 709050 has been marked as a duplicate of this bug. ***
OSTree has moved to Github a while ago. Furthermore, GNOME Bugzilla will be shut down and replaced by gitlab.gnome.org. If the problem reported in this Bugzilla ticket is still valid, please report it to https://github.com/ostreedev/ostree/issues instead. Thank you! Closing this report as WONTFIX as part of Bugzilla Housekeeping.