After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 706044 - Add a nominatim instance or proxy on gnome.org
Add a nominatim instance or proxy on gnome.org
Status: RESOLVED OBSOLETE
Product: sysadmin
Classification: Infrastructure
Component: Other
unspecified
Other Linux
: Normal enhancement
: ---
Assigned To: GNOME Sysadmins
GNOME Sysadmins
Depends on:
Blocks:
 
 
Reported: 2013-08-15 02:02 UTC by Mattias Bengtsson
Modified: 2013-11-21 15:17 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Mattias Bengtsson 2013-08-15 02:02:18 UTC
Geocode-glib (and therefore GeoClue and Maps) uses Nominatim[1]
for geocoding[2]. 
OpenStreetMaps[3] and MapQuest[4] host the only two free and public
Nominatim-instances that I know of. During the 3.9.x series we have
used the Nominatim instance on OSM directly, but that probably needs
to change in time for 3.10. 

===

The usage policy for OSM's instance[5] states (among other) the 
following:

"No heavy uses (an absolute maximum of 1 request/s)."

"Note that the usage limits above apply per website/ application: 
 the sum of traffic by all your users should not exceed the limits."

"Apps must make sure that they can switch the service at our request
 at any time (in particular, switching should be possible without
 requiring a software update). If anyhow possible, set up a proxy and
 also enable caching of requests."

===

The Terms of Use for the MapQuest instance[6] basically states that
we can use it freely if we: 
 1) give OSM credit
 2) give MapQuest credit 
 3) mail them and let them know we'll start using their services
 4) mail them if we expect jumps in usage
 5) abide to the legal stuff stated in 9a here: 
    http://developer.mapquest.com/web/info/terms-of-use

Regarding #2 here we'd actually need to show a MapQuest logo and 
say “Nominatim Search Courtesy of MapQuest” next to it for all 
applications that uses it. I'm not sure exactly how that'd work
really since I think we want to use GeoClue in the control-center
for example to help us autofill stuff and possibly in other places
as well. 
Regarding #5 I have no idea. It's all gibberish to me.

===

As I see it this leaves us with three possible choices:

 - Install our own Nominatim instance
   The hardware requirements[7] are really high so this might be
   unfeasible. Might be necessary in the long run if we generate
   too much traffic for osm/mapquest. 

 - Use MapQuest Nominatim directly or via a proxy
   Someone would have to come with some input here on whether 
   MapQuests terms of use are reasonable for us. 
   If we don't put up a proxy we would also get into trouble if 
   MapQuest suddenly decided that our load is too high and blocked
   us since we'd have no way to redirect our requests to another
   server (since the url to MapQuest's nominatim would be hardcoded).

 - Set up a (caching) proxy to OSM Nominatim
   This is what I'd suggest right now. We would be pretty future
   proof while not having to install too much server software.
   We'd also not have to deal with the MapQuest Terms of Use-stuff
   (which might or might not be a deal-breaker, I have no idea).

Me and Andrea had a discussion about this on #sysadmin earlier today
and I told him I'd post here. 
Here's the log if anyone is interested: 
https://bugzilla.gnome.org/attachment.cgi?id=251641

--
1: http://wiki.openstreetmap.org/wiki/Nominatim
2: http://en.wikipedia.org/wiki/Geocoding
3: http://nominatim.openstreetmap.org/
4: http://open.mapquestapi.com/nominatim/v1/
5: http://wiki.openstreetmap.org/wiki/Nominatim_usage_policy
6: http://developer.mapquest.com/web/products/open/nominatim
7: http://wiki.openstreetmap.org/wiki/Nominatim/Installation#Hardware
Comment 1 Zeeshan Ali 2013-08-15 09:02:37 UTC
(In reply to comment #0)
> Geocode-glib (and therefore GeoClue and Maps) uses Nominatim[1]
> for geocoding[2]. 
> OpenStreetMaps[3] and MapQuest[4] host the only two free and public
> Nominatim-instances that I know of. During the 3.9.x series we have
> used the Nominatim instance on OSM directly, but that probably needs
> to change in time for 3.10. 
> 
> ===
> 
> The usage policy for OSM's instance[5] states (among other) the 
> following:
> 
> "No heavy uses (an absolute maximum of 1 request/s)."
> 
> "Note that the usage limits above apply per website/ application: 
>  the sum of traffic by all your users should not exceed the limits."
> 
> "Apps must make sure that they can switch the service at our request
>  at any time (in particular, switching should be possible without
>  requiring a software update). If anyhow possible, set up a proxy and
>  also enable caching of requests."
> 
> ===
> 
> The Terms of Use for the MapQuest instance[6] basically states that
> we can use it freely if we: 
>  1) give OSM credit
>  2) give MapQuest credit 
>  3) mail them and let them know we'll start using their services
>  4) mail them if we expect jumps in usage
>  5) abide to the legal stuff stated in 9a here: 
>     http://developer.mapquest.com/web/info/terms-of-use
> 
> Regarding #2 here we'd actually need to show a MapQuest logo and 
> say “Nominatim Search Courtesy of MapQuest” next to it for all 
> applications that uses it. I'm not sure exactly how that'd work
> really since I think we want to use GeoClue in the control-center
> for example to help us autofill stuff and possibly in other places
> as well. 

s/GeoClue/geocode-glib/ and maybe we could ask them if its OK if some of our UIs can't give them very visible credits? We could give them credits in about dialog or something. I'll also talk to Jon about this.
Comment 2 Mattias Bengtsson 2013-08-15 20:07:29 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > Regarding #2 here we'd actually need to show a MapQuest logo and 
> > say “Nominatim Search Courtesy of MapQuest” next to it for all 
> > applications that uses it. I'm not sure exactly how that'd work
> > really since I think we want to use GeoClue in the control-center
> > for example to help us autofill stuff and possibly in other places
> > as well. 
> 
> s/GeoClue/geocode-glib/ and maybe we could ask them if its OK if some of our
> UIs can't give them very visible credits? We could give them credits in about
> dialog or something. I'll also talk to Jon about this.

Thanks. We probably need designer input on this.
Comment 3 Andrea Veri 2013-08-19 18:59:28 UTC
Added a mod_proxy rule for forwarding the requests sent to nominatim.gnome.org to nominatim.openstreetmaps.org. Also got in touch with the OSM admins about a possible way to avoid being banned for doing more than 1/reqs per IP.

Also seems the OSM admins are going to whitelist our proxy's IP. In addition they'll be enforcing the 1/reqs on the User-Agent header binding it to the one provided by gnome-maps itself. 

In the case things will get worse and the traffic will be getting huge, they'll contact us for finding a possible solution but ideally the internal caching of the various apps (geocode-lib?) should also help with this.
Comment 4 Andrea Veri 2013-11-21 14:56:31 UTC
The GNOME Infrastructure Team is currently migrating its bug / issue tracker away from Bugzilla to Request Tracker and therefore all the currently open bugs have been closed and marked as OBSOLETE.

The following move will also act as a cleanup for very old and ancient tickets that were still living on Bugzilla. If your issue still hasn't been fixed as of today please report it again on the relevant RT queue.

More details about the available queues you can report the bug against can be found at https://wiki.gnome.org/Sysadmin/RequestTracker.

Thanks for your patience,

the GNOME Infrastructure Team
Comment 5 Zeeshan Ali 2013-11-21 15:14:35 UTC
How is this bug "old" or "ancient"? :)
Comment 6 Andrea Veri 2013-11-21 15:17:39 UTC
"The GNOME Infrastructure Team is currently migrating its bug / issue tracker
away from Bugzilla to Request Tracker and therefore all the currently open bugs
have been closed and marked as OBSOLETE."

We're not handling requests with Bugzilla anymore, I'm sending out an email soon with all the details.