GNOME Bugzilla – Bug 89746
Weather station names in gweather can't be translated
Last modified: 2005-06-18 01:17:49 UTC
In gweather, the Location file contains strings like Central and South America, but these strings can't be translated. This file is in gnome_config format which I think isn't supported by intltool. A quite simple fix would be to convert it to xml, which intltool can translate.
Pretty nasty problem for i18n, right?
Is anybody working on this? I don't mind taking a stab. Anything to keep in mind while coding re: internalization issues?
I'm attaching a copy of an xml version of the locations file, any input on it's structure? I was trying to decide if recording zone and radar even if they were "blank" made sense. With the xml format, it's easy to add more information, such as a customized radarmap URL (<radar url="http://my.custom_address/blah.jpg"/>) or the longitude/latitude of a location.
Created attachment 12617 [details] XML version of locations file
I'm attaching a patch which contains an updated Locations.xml file and modifications to gweather-prefs.c Two issues: 1) The Makefile.am doesn't install the Locations.xml file 2) in gweather-prefs.c, I reference the Locations.xml as /tmp/Locations.xml I don't know enough about automake to fix those.
Created attachment 12641 [details] patch and updated Locations.xml file
Oh, and if someone can sort the "locations" under each state in the locations.xml file, we could append instead of prepend in gweather-prefs.c:310
Created attachment 12655 [details] [review] fixes the various reported problems
I slightly modified your patch to read Location.xml from $prefix/share/gweather (I just copied and pasted the old code), and modified Makefile.am to install it (actually I only replaced Locations with Locations.xml :p) There is still some changes to do so that Locations.xml is translatable (these changes are explained in intltool README)
Forgot to mention that replacing append with preprend was probably not necessary since the append is probably occurring on a GList where prepend and append have the same cost (maybe I totally missed your point about using prepend instead of append though :p)
*** Bug 103081 has been marked as a duplicate of this bug. ***
I looked a bit at gweather code the other day, and it seems Locations is no longer used. If I'm right, this can be closed (though another bug should be reopened since the town names are probably untranslatable in the new version)
teuf, could you please open another bug report if it's needed?
Just renaming this report.
Yah, in cvs the weather is gotten from weather.interceptvector.com so Locations is not used and the weather data is in xml format (and untranslated). What we hope to do is run our own weather server where we can then do some translations on the cities. So this bug is still valid and shouldn't be closed
No chance for having this sorted out before the string freeze, right?
Still can't contact weather.interceptvector maintainer at all?
wasn't the whole weather.interceptvector.com change reverted before 2.4.0? That means we're back to using Locations, right?
What's the status on this? I believe we can translate the Locations file as is, using intltool, since it handles this style of file. Shall I spend the effort to make Locations work, or should I just wait until we have a sexy XML server?
*** Bug 131324 has been marked as a duplicate of this bug. ***
Do I understand it correctly that this bug, among other things, want to make it possible to use non-ASCII characters in city names? Currently if I try to add that to Locations it breaks.
*** Bug 135608 has been marked as a duplicate of this bug. ***
*** Bug 140004 has been marked as a duplicate of this bug. ***
Few notes: 1. intltool cannot currently handle this format, because we've got fields like: key=something-translatable CODE BLAH --- ... (i.e. the value of a key is space-delimited, and we get the whole line if we try intltool-extract on that) 2. There's no problem with using UTF-8 in Locations file, at least I couldn't see any (and I tried it) There are a few potential solutions (perhaps many others): 1. Preprocess Locations and produce a .h file using suitable C structs (WeatherLocation) -- this wins speed (loading time), eases translation, but makes it harder to change Locations during runtime (recompilation is needed) 2. Preprocess Locations and produce a .h file containing only city and country names, which is not used in the code, but only to extract strings for translation; this is the easiest approach, and we'd need to add only a couple of gettext() or _() calls; it's not very scalable, though 3. Go with the XML or some other format (as discussed above)
I remember that several versions ago, Evolution solved this problem for the weather component in the Summary page. During that time, the translatable strings were over 5000 and I remember translators were unhappy as they could not complete the localisation. Then the strings backed out. Can anyone confirm/deny this or am I smoking something? (for the expressions sake, I do not smoke)
I can't comment on what you've been inhaling :) but I checked out the 1.4.6 evolution code and it still contains an individual string for each location, resulting in over 6000 translatable strings in 1.4.6. For the record, it appears that evolution uses solution 2 from Danilo's list above. I haven't tried the new version of evolution (1.5.7) but from looking at CVS HEAD, it appears that they have completely removed the Summary page and hence don't have a weather component anymore. I suppose you could class that as solving the problem. Of course it could be somewhere else, but I haven't been able to find it with a quick search of the code.
The latest patch didn't include the Locations.xml file. Not sure it's still going to be done in that fashion, but it should be in there for completeness.
Created attachment 28255 [details] patch plus xml.in file The attached patch, is mostly an updated version of Christophe's previous patch modified to work with the cvs head. However, I've extended it, so that the locations are now marked for translation and are properly displayed when translated. I've also included a perl script that I wrote to convert the one Locations file into Locations.xml.in, just in case some of the locations change before this patch is applied.
I think it's important to get in soon. Upping pri.
Agreeing with Kjartan this needs to get in asap. Having the following comments: 1. the .xml file that gets generated from the Locations.xml.in seems to be empty here. 2. Lack of a ChangeLog entry. 3. You might want to update to CVS HEAD however it's not conflicting yet, only a few offsets. Thanks, it's getting really close and ready for inclusion.
A nice addition to the xml file would also be custom forecast urls. The weather stations of many countries do have a text version online that could be used. Note that the current forecast is parsed and that these text versions shouldn't be parsed. What do you think about it Gareth ?
Created attachment 28532 [details] updated patch I've updated the patch to CVS head and also added ChangeLog entries for the po and gweather directories. The included Locations.xml.in file has also been updated to be in line with CVS head. The problem with the Locations.xml file not being generated, was that my script to convertion Locations -> Locations.xml.in wasn't correctly converting & to & - now it does.
With reference to comment #31. I think that would be a great addition to xml file, I've already thought of a number of additions that could be made once the xml file is in place: - custom radar entries (ie none weather.com) - splitting metar/bom/met forecast info into seperate actions - removal of blank iwin and radar entries But before looking at these I think it would be better to get this part of the xml changes commited. Else, we won't give the translators time to translate all the Location names and this commit is likely to grow exponentially.
The patch is having some severe issues here: 1. Opening the preference window takes a considerable amount of time. (5 seconds on an athlon 2600 XP) 2. The tree view is completely borked. There is a load of duplicate entries, It's randomly leaf opened, closed at places, and some more strangeness. One minor comment: You deleted the enter at the end of the ChangeLog :) Sorry it isn't working yet, but it's getting there without doubt!
*** Bug 138580 has been marked as a duplicate of this bug. ***
Created attachment 28846 [details] updated patch As Dennis pointed out there were some major problems with the last patch. This latest patch should address all these areas. To fix the speed problem, I had to parse the xml file one line at a time using the xmlTestReader API calls. To keep the code semi-modularised I've put the parsing code in two new files gweather-xml.c and gweather-xml.h. I also change the format of the XML file, so that full nodes are marked for translation rather than single attributes, I've removed the attributes and replaced them with individual nodes. This made dealing with the multiple languages much easier. The only other addition to the changes, is that you no longer have to have entries for radar and zone if the location doesn't have a radar map or iwin forecast. This means the xml file is slightly smaller. The only remaining work that needs to be done, is to re-order some of the entries in the XML file, so that they all appear in alphabetical order. This is because the order the locations appear in the XML file, is the order they appear in the locations tree. I'd like to wait until this patch has been commited before doing that, so that I don't accidentally miss any new/modified locations. As before the tar file contains a simple script to convert the current Locations file into the Locations.xml.in format. For the curious, the patch adds in the region of an extra 2500 strings that need translating! Good luck to all translators !!
Some further testing has shown that there is still some work to be done on the patch. It doesn't display the translated locations unless the language code in ALL_LINGUAS is an exact match for the locale. For example, it works for en_GB, but doesn't work for fr_FR. I'll need to have a look at gettext or how this is handled this for menu items in the gnome desktop. Unless someone can point me in the right direction.
I also would like to suggest, if this is doable and doesn't take too much resource to sort within the program and don't rely on how the XML file is sorted. Good luck with the other problem, I can't really give an suggestion because my knowledge on gettext is bellow what it should be.
Gareth: the gnome_vfs_i18n_get_language_list from gnome-vfs/libgnomevfs/gnome-vfs-i18n.c (similar functions are present in libgnome and libbonobo) returns a list of the locales which can be used for a fr_FR string (in that case it would return things like fr_FR, fr, C I guess).
Created attachment 28887 [details] updated patch Another (and hopefully final) patch to fix this. Thanks to the info from Christophe I was able to sort out the translations. Now the correct translation is displayed, even if the locale doesn't exactly match the translation name (ie fr_FR now correctly shows the French translation). Also, as suggested by Dennis the sorting is done within the applet. This means that the sorted is done in the current locale, ie the sorting is done on the translated names. Dennis, I think this should be everything, can you check this patch?
Looks incredibly good!, i've comitted in to CVS HEAD! Thanks a lot! It's fantastic ;) Could people please try CVS HEAD and see if it works for them as well ?
We definitelly DON'T want core desktop to grow for another 2500 strings (at least IMHO)! Can we have this separated out into another gettext domain (with new intltool, it should be trivial) which we'd depend on for translations. If you agree with the approach in principle, I can help with that (basically, just put "Locations.xml.in" in po/POTFILES.skip, create po-locations with POTFILES.in containing "Locations.xml.in" and suitable Makevars, and use dgettext instead of gettext for getting translations).
Gareth, Menthos what do you people think of Danilos suggestion ?
It sounds fine to me, 2500 strings is a lot for one applet. I don't think you'll actually have to use gettext, as the code reads the translations directly from the xml file and doesn't use gettext. Feel free to go ahead and make the changes.
Uhm, it's even easier than that -- no need to bother with dgettext (I just looked at the code), since translations are read directly from generated .xml file. So, it's as simple as creating a separate directory for locations. It might even help in having real PO files smaller (since these new PO files in po-locations and their generated MO files should not be installed).
Danilo, What needs to be done to implement this. I think this should go in pretty quickly as the 2500 strings are showing up against gnome-applets now on the translation status pages.
Gareth, I'm attaching a patch which basically does this: new files po-locations/POTFILES.in and po-locations/POTFILES.skip; instead of INTLTOOL_XML_RULE, use same rule (from intltool.m4) with "po" replaced by "po-locations". The only remaining issue with the patch is dist-ing po-locations/*.po files (we need them in tarballs to be able to build Locations.xml). That can be done by copying po/Makefile.in.in to po-locations, and adding "po-locations" to list of dist-ed subdirs, though this will have negative side-effect of actually installing generated ".mo" files (which we do not need). I don't have time right now to look into Makefile stuff to see how best to avoid installing .mo files, yet include .po files in dists.
Created attachment 28951 [details] [review] Separate translation domain for Locations.xml
What about use the translations from iso-codes ( http://ftp.debian.org/debian/pool/main/i/iso-codes/iso-codes_0.35.orig.tar.gz) package? We must try to share as much translations as possible.
I agree. I'd much rather see the locations translated as a package in the Translation Project or something like that. Translating them for just GNOME alone makes little sense.
Well, make up your minds, we need something for 2.8 here and it must be doable. What about just going for a separated gettext domain ? as suggested before.
I think we should first split the locations into a seperate domain, as this will then enable translators to track location translations seperately from the rest of gnome-applets translations. I agree that using something like the iso-codes would be a sensible way forward and reduce the work of the translators. But to do this wouldn't that mean that Gnome would either have to add the iso-codes package to the list pre-requistes or actually distribute it? I don't think I'm in a position to make such a decision. Also, as Danilo pointed out on the gnome-i18n list, the iso-code package (specifically ISO-3166 and ISO-3166-2) only contain country names and region names (eg US states), they do not contain individual city names. Therefore the majority of the strings marked for translation by this patch, wouldn't actually be covered by iso-codes.
The locations for the weather reports comes from the airports that exist in each country. Therefore there is no 1 to 1 association between a location and an airport. For example, capital cities typically have 3 or more airports. In addition. there are airports where no city is near (for example military airports). As a result, it will be *really* difficult to associate the "iso-codes" textual domain of the Translation Project with the Locations file of gweather. Think of "London/Heathrow", "London/Gatwick", "London/Luton", "London/City". One can claim to localise the first part but still it limits the work of the translator. In this example, I would put "London/West", "London/SouthWest", "London/North", "London/Centre". In addition, the textual domain for the Locations file should be different from that of gnome-applets, as it will add thousands of names with no chance to get translated. There should be a different textual domain, separate from gnome-applets.po, that probably will not show up in the stats (so the 100% is not affected). For the Greek language, as I envision it, we will translate the Greek airport names (time to add the full list from https://www.notams.jcs.mil/common/icao/R9C1.html !!!) and possibly those from the neighbouring countries and the capital cities of Europe. As a result, the final Locations.mo will be really small, containing only the translated names. To help better, one may msgmerge the existing iso-codes city names to this translation. Finally, - DO NOT mix the iso-codes textual domain with the WeatherLocations textual domain - Make an independent textual domain for the city names that does not appear in the general stats Reference: http://bugzilla.gnome.org/show_bug.cgi?id=140004
I've submitted Danilo's patch, so we now have a separate textual domain for locations. The distribution tar file issue still has to be sorted out. But I thought it was better to get the domain split in as soon as possible, so that the translators can get to work.
What is the status of this bug? Does it still need to be open?
This is very much fixed (for quite some time now).