First of all, thanks a lot.

Not entering into the details, for the overall picture I would prefer things changed that way:

- turn debian-sysdeps.py into jhbuild/commands/debiansysdeps.py (or whatever, I like generate-debian-sysdeps-cache but it's quite long)
- make it output a json file into ~/.cache/jhbuild/ instead of debian_data.py (with files only relevant to the user distribution)
- on relevant distributions, generate the cache on first run (maybe displaying a message if it's too old?)

Does that sound reasonable to you?

That's not a bad approach, but it would certainly be slower than the current system.  It also means a very large download for every user.

It does have the advantage of us not having to keep the files up to date.

If we did your approach, I'd skip the separate command and just have the download and data generation occur on-demand the first time you tried to use sysdeps.

There is another problem, though: jhbuild may not have the full moduleset information available at the time.  That's both an advantage and a disadvantage.  If the user has a custom moduleset, for example, they could end up getting better results (ie: it could find custom sysdeps that are not in our modules) but it could also mean that we cache an invalid result for when they go back to using our modulesets.

A way to avoid that issue could be to download the Contents file and cache that.  Each time the user does 'sysdeps --install' we could manually do the query at that time.  That would be slow, but still _much_ faster than the current backend (I measure debian-sysdeps.py against jessie as taking under 6 seconds on my relatively speedy laptop).

But now we're into a place where we start asking "why are we caching Contents files in the user's homedir?  Isn't this what apt-file is supposed to do in /var?"  But then we have to tell the user "install apt-file and keep it up to date"...

Lots of interesting questions :)


One more aside: I was chatting with one of the freebsd-gnome guys last night.  He noticed the branch and thought it was interesting.  It might make sense to add something similar for FreeBSD, but in that case, the updating of the data would be a lot more manual (ie: a substantially static database with many modifications by-hand).  Since we already crossed the barrier of shipping static package data for Debian, it didn't seem so evil to do that for FreeBSD, but now that you propose that we stop doing that, ...

Writing things as I think about them...  ok, what about having data/$sysid.json, I have less problems with hosting generated data than generated code, and this would allow for a freebsd file.  The easiest way is certainly to get it in the jhbuild repository but this may be the tipping point to start using the gnome-modulesets repository, and the moduleset file could be extended with <package-matching sysid="debian-8" href="https://git.gnome.org/browse/gnome-modulesets/plain/data/debian-8.json"/> lines.

That may be even more issues to think about, so I propose to cut it down to the simplest thing; data/$sysid.json files in the jhbuild repository, and later we'll see how to transition to the gnome-modulesets location, and that may include extending the moduleset with new tags.

In what way is the existing PackageKit sysdeps backend insufficient for Debian?

(In reply to comment #4)
> In what way is the existing PackageKit sysdeps backend insufficient for Debian?

Debian doesn't know how to resolve pkgconfig names into package names.  It also can't do filenames.  That's what makes it worse than yum.  Additionally, we are contemplating (in bug 691394) adding support for some new types of sysdeps: python modules and xml stylesheets.  Neither of those is supported either.

(In reply to comment #3)
> Writing things as I think about them...  ok, what about having
> data/$sysid.json, I have less problems with hosting generated data than
> generated code, and this would allow for a freebsd file.

The current file is "just data" as well... it's hard to argue that this is generated code.  I used python syntax just because that was the most obvious choice for a tool written in python.

I'm happy to move it to data/ or to use ast.literal_eval to parse it...

> The current file is "just data" as well... it's hard to argue that this is
> generated code.  I used python syntax just because that was the most obvious
> choice for a tool written in python.
> 
> I'm happy to move it to data/ or to use ast.literal_eval to parse it...

But someday those files may be hosted and downloaded automatically, and then I'll be happier with json.load().

I'd say my most pressing concern is that JSON can't even do comments...

-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/jhbuild/-/issues/206.