Created attachment 364755 [details] [review]
HACKING: Document the requirements for adding weather sources

Review of attachment 364755 [details] [review]:

::: HACKING
@@ +38,3 @@
+- the API should be documented
+- access to the API can require the use of a freely available application
+  token ID, but one should be provided in a patch for testing purposes

And you should be reasonably confident that lots of people running `make check` won’t cause the token to be revoked and break everyone’s tests (very often).

It should be easy to change the token in the test code so distros can change it in patches if they need to.

@@ +39,3 @@
+- access to the API can require the use of a freely available application
+  token ID, but one should be provided in a patch for testing purposes
+- the result of requests must be machine parseable

And the parsing code must have unit tests, since it’s parsing untrusted content off the internet.

HTTPS must be used if at all possible.

The user’s privacy must be maintained as much as possible (don’t include unnecessary detail in requests, don’t allow fine-grained location tracking, don’t include other identifying information in the requests if possible).

The user’s bandwidth must be preserved where possible (make use of caching, use HEAD requests and caching headers to minimise downloads when polling and nothing has changed on the server).

Created attachment 364766 [details] [review]
HACKING: Document the requirements for adding weather sources

Attachment 364766 [details] pushed as 29c682e - HACKING: Document the requirements for adding weather sources

I disagree with the requirement that the data must always be available for commercial use.

Both Yahoo Weather and (AFAIK) OpenWeatherMap are non-commercial, and that's why they are not the default, but if an application is not commercial (eg a research project using libgweather, or a non-profit website, or who knows what) they should be allowed to opt in to different backends.

The reason is that the quality and the coverage of the different sources varies a lot, and yr.no, while having good world coverage, is not always great in terms of predictive power.
(Additionally, METAR, the default and only source of current weather, has very spotty coverage, with many airports around the world reporting only once a day or once every few days).

Yes, that might mean that some sources will be more tested than others, and thus less buggy. That's what unit tests and bug reports are for. If people care, they'll file bugs.


As for caching, it's worth mentioning libsoup provides a full-featured on-disk HTTP/1.1 cache (and libgweather makes use of it). As long as the upstream server has useful Cache-Control, ETag and If-None-Match (or Last-Modified, If-Modified-Since), the client code does not need any special support.

(In reply to Giovanni Campagna from comment #5)
> I disagree with the requirement that the data must always be available for
> commercial use.
> 
> Both Yahoo Weather and (AFAIK) OpenWeatherMap are non-commercial, and that's
> why they are not the default, but if an application is not commercial (eg a
> research project using libgweather, or a non-profit website, or who knows
> what) they should be allowed to opt in to different backends.
> 
> The reason is that the quality and the coverage of the different sources
> varies a lot, and yr.no, while having good world coverage, is not always
> great in terms of predictive power.
> (Additionally, METAR, the default and only source of current weather, has
> very spotty coverage, with many airports around the world reporting only
> once a day or once every few days).
> 
> Yes, that might mean that some sources will be more tested than others, and
> thus less buggy. That's what unit tests and bug reports are for. If people
> care, they'll file bugs.

I've updated the HACKING file about that, noting that this fact must be mentioned in the service selection bitfield.

> As for caching, it's worth mentioning libsoup provides a full-featured
> on-disk HTTP/1.1 cache (and libgweather makes use of it). As long as the
> upstream server has useful Cache-Control, ETag and If-None-Match (or
> Last-Modified, If-Modified-Since), the client code does not need any special
> support.

I've mentioned that the servers must support features that'll stop libsoup from re-downloading.

commit 8250359e995af09112e38ab95f6f45b3e84b9a63
Author: Bastien Nocera <hadess@hadess.net>
Date:   Sat Dec 2 16:39:50 2017 +0100

    HACKING: Update some backend requirements
    
    As per https://bugzilla.gnome.org/show_bug.cgi?id=791086#c5