After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 778619 - Download image data at runtime
Download image data at runtime
Product: recipes
Classification: Other
Component: data
Other Linux
: Normal enhancement
: 2.0
Assigned To: Recipes maintainer(s)
Recipes maintainer(s)
: 780224 (view as bug list)
Depends on:
Blocks: 779223
Reported: 2017-02-14 18:47 UTC by Matthias Clasen
Modified: 2017-04-04 02:07 UTC
See Also:
GNOME target: ---
GNOME version: ---

Description Matthias Clasen 2017-02-14 18:47:36 UTC
We currently have 18 recipes by 13 chefs included in our data, and our 0.12.0 tarball weighs 73M. Clearly, we can't keep going like this and we need to think about some solution for hosting at least the image data somewhere else.
Comment 1 Matthias Clasen 2017-02-25 15:41:50 UTC
With 26 recipes, we have 100M of images.

A rough outline for a plan, based on what gnome-software does for screenshots:

1) Include the default image for each recipe and the chef images with the app, so we can populate the tiles without delay.

2) When opening the detail page for a recipe, start downloading any missing images.

3) We may or may not need to do the same trick gnome-software does, where it first downloads a thumbnail and then proceeds to load the full-size image.

4) Use ${sha256}-${original-name} to locate the images.

5) Store downloaded images in XDG_CACHE_DIR
Comment 2 Matthias Clasen 2017-02-25 18:01:22 UTC
Once we start getting new recipes and chefs by downloading data, the 'preinstall some images' part does not make too much sense anymore; we better just  make sure to download the default image for each new recipe quickly and make tiles deal with images that are still being downloaded.
Comment 3 Matthias Clasen 2017-03-06 00:35:05 UTC
There's an image-download branch that has an implementation.
Comment 4 Jeremy Bicha 2017-03-18 00:17:49 UTC
*** Bug 780224 has been marked as a duplicate of this bug. ***
Comment 5 Matthias Clasen 2017-03-30 11:48:02 UTC
Image content will be hosted at Some example content is there already.

To do before this is ready for prime time:

- Verify that setting local images during recipe import and in other places works cleanly

- Store some negative cache entries so we don't try a failing url more than, say, once per day

- Consider a low-bandwidth setting that would try to load lower-resolution versions of the images, which will have to be provided on the site

- Handle change of image content, somehow (look what gnome-software does)
Comment 6 Matthias Clasen 2017-03-30 11:49:02 UTC
- Store version information in the recipe and chef databases:

Comment 7 Matthias Clasen 2017-03-30 12:08:26 UTC
- Make sure we load only images that are actually visible
Comment 8 Matthias Clasen 2017-03-30 12:10:58 UTC
- make sure libsoup builds and works on OS X