GNOME Bugzilla – Bug 770321
Forecast not available all day, HTTP 404 for noaa.gov
Last modified: 2016-10-07 16:27:29 UTC
(org.gnome.Weather.Application:15735): GWeather-WARNING **: Failed to get METAR data: 404 Not Found. GNOME 3.20.1 as distributed in Fedora 24. My logs first mention this yesterday at 00:08 UTC and it has been happening every time I open the app since. I’m guessing that a service change went live at midnight yesterday.
http://weather.noaa.gov/mgetmetar.php returns “This Service is no longer available”
There doesn’t appear to be a replacement from NOAA.
According to https://web.archive.org/web/20160130172236/http://weather.noaa.gov:80/weather/coded.html "METAR Data Access On or about April 4th, 2012, request for METARS through this service will be redirected to: http://www.aviationweather.gov/adds/metars/ This redirection is part of a larger effort to reduce duplication of effort within the National Weather Service. The website above offers the same functionality. It also needs to be noted that the NWS Aviation Weather Center operated website above is the only FAA approved source of aviation weather over the Internet to be used for pilot briefings. Please update bookmarks, links, etc.."
Ah argh I did not know about that. Anyone of you want to take a stab at a patch using the new service? Otherwise I'll try to work on it on the weekend.
Created attachment 334100 [details] [review] Ugly monkey patch to fix METAR retrieval The attached extremely ugly patch against the libgweather git head restores METAR functionality using the replacement www.aviationweather.org text data server. (Which outputs well-formed XML that the code should probably be parsing, instead of blindly flailing around in for the raw-text METAR string, but... meh.)
Hi all, You can also use my patch for libmateweather (a fork of libgweather for MATE desktop): https://github.com/mate-desktop/libmateweather/commit/835055e0d0b950d3f66ad236de7e40942f6f7d8e Well, it's not perfect... but this new URL turned out to be a nice drop-in replacement, so the code change is quite small.
By the by, why does this problem with a US only weather service cause issues for everyone else in the world that use data from other weather services such as MET Norway? From what I can tell, the METAR data from NOAA.org is just discarded and never used.
Frank Dana and Vlad Orlov, please update your patches and switch the endpoints to HTTPS instead of HTTP. (Just change the supplied URLs from one to the other.) This will increase user privacy and help prevent any third parties from learning users’ device location from their use of libgweather/libmateweather.
The list of things that could be improved about my patch would probably exceed bugzilla's max comment length; if libgweather wants to transition to making HTTPS data requests then I'm all for it, but I think that's a separate change outside the scope of this bug report. I'd suggest opening a separate bug, if there isn't already one in the system.
Frank Dana, the only reason this hasn’t been changed to HTTPS already is that the previous API endpoint didn’t support it. Some other libgweather APIs now use HTTPS. I can open a follow-up patch later, but would be good to just get the change in sooner rather than later.
Review of attachment 334100 [details] [review]: It really needs to use HTTPS.
Created attachment 334112 [details] [review] Updated METARS patch with HTTPS requests Alrighty then. Here you go.
Created attachment 334136 [details] [review] libmateweather patch adapted Both patches work fine but the one from Vlad / libmateweather looks a lot more trivial.
More trivial, at the expense of scraping the METAR string from an active webpage, instead of a data service. By my testing, requesting http://aviationweather.gov/metar/data?ids=KLGA (for my local weather station) results in a 1.54 MEGABYTE response. While guidance on how to use aviationweather.gov's data programmatically is unfortunately spare (I've looked), that's clearly against their Best Practices (http://www.aviationweather.gov/dataserver/bestpractices), and even assuming their servers are up to the task, seems a wasteful consumption of the enduser's bandwidth.
My mistake, I'm an idiot. (I retrieved the page at 1.54MB/s, and clearly have wget output reading-comprehension issues this early in the morning.) However, the difference between the patches is still a 618 byte response (mine) and an 18.48 KByte response (libmateweather). That's a rather significant cost.
I know it's not perfect, as I said before. That way of getting the data was in the library before (and was there for a long time it seems, even since Gnome 2), so for a quick fix I did a simple drop-in replacement patch. BTW, there's one more option - to parse plain text files from here: http://tgftp.nws.noaa.gov/data/observations/metar/stations/ However, it looks like https isn't supported there, so I'm not sure about this.
Comment on attachment 334136 [details] [review] libmateweather patch adapted Marking obsolete due to high traffic count.
I think Frank's right, we should not strain the service more than really needed. I've also been browsing for documentation of aviationweather.gov services and found next to nothing. Also been trying to use the noaa.gov site Vlad mentioned. But there are at least two reasons against it: 1. it doesn't provide https 2. according to NWS the site aviationweather.gov is the only authoritative site for METAR data. So we should stick to that.
Ok. I think I'll use Frank's patch in libmateweather as well.
I can confirm that Frank's patch seems to work fine. I applied it on Debian Unstable's libmateweather 1.14.0 as well as Debian Oldstable's libgweather 3.4.1 (where a small change in the patch is required, the old location being different), and both now work again and seem to give sensible results. Thanks!
Patch doesn't apply directly on libgweather 3.20.2 (or git HEAD) because some surrounding code slightly changed, but it otherwise works fine there too.
How about fixing this in a more resilient (change-tolerant) way like gnome-maps did with their maps data provider?
(In reply to Giovanni Campagna from comment #4) > Ah argh I did not know about that. > > Anyone of you want to take a stab at a patch using the new service? > Otherwise I'll try to work on it on the weekend. Giovanni, do you have time to get releases out today or tomorrow? Otherwise, I would prefer to handle releases due to the urgency of this issue.
Apologies for not getting to this earlier. The patch currently posted looks good, and I tested it with gnome-weather it appears to work (ie, I get reasonable current weather conditions around the world). I am about to issue a new release of libgweather 3.20 and 3.18 with the patch applied. I do not plan to backport it further, but it should not be hard to do it for distros because the code did not change significantly in that area recently. Thanks for the patch in any case.
*** Bug 772262 has been marked as a duplicate of this bug. ***
I have an ancient copy of Gnome Clock 2.28.0 that stopped displaying the weather at the same time as the current version of GNOME Weather. It seems not to use libgweather. Can anyone advise me on a way to patch this one?
(In reply to Ron_ from comment #26) > I have an ancient copy of Gnome Clock 2.28.0 that stopped displaying the > weather at the same time as the current version of GNOME Weather. It seems > not to use libgweather. Can anyone advise me on a way to patch this one? Update to a more recent version. Gnome clock 2.28 is like 5 to 6 years old and probably hasn't seen an update in 5 years.
(In reply to Ron_ from comment #26) > Can anyone advise me on a way to patch this one? Please contact your Linux distribution for support questions. Thanks!