GNOME Bugzilla – Bug 569829
Please support calling gettext() at runtime instead of shipping static translations
Last modified: 2013-08-18 00:42:57 UTC
Currently, .desktop files ship static translations. This has the following problems: * It duplicates information from the .mo files, which are shipped anyway for translating the actual application. * It makes it impossible to ship updated .mo files (e. g. in "language packs", which we do in Ubuntu) which update the .desktop translations as well. For this reason, we have used a patch to call gettext() at runtime in Ubuntu for many years [1]. Back then we asked for inclusion of a new optional "Gettext-Domain:" field into the desktop XDG standard, but it was rejected. But now, after so many years, it has proved to work really well, and the runtime overhead is not even measurable. Anyway, Vincent Untz ported those patches to OpenSUSE a while ago, and did some improvements. I merged them back into Ubuntu now, and did a bug fix. This patch is fully backwards compatible, i. e. if you use a standard desktop file, it will work as before. However, if the desktop file has no translations for the current language, and it has a X-GNOME-Gettext-Domain=<domain> field, then it will do a g_dgettext("<domain">, "C string") call to get the translation dynamically. With that, distros can mangle .desktop files in their build systems (i. e. drop the translations and add the X-GNOME-Gettext-Domain field). Of course this build time mangling is a hack. In an ideal world, it would look like this: intltool-merge would grow a mode to only add the gettext domain at the appropriate place instead of merging actual translations. * If all distros/upstream want to use that, it should just do that by default. * If that is disputed, we could keep the current default behaviour and instead provide an option for that alternate mode, like --only-add-domain. * Introduce a new environment variable INTLTOOL_EXTRA_ARGS where distros/builders could specify --only-add-domain and thus build all GNOME packages that way wholesale without modifying the GNOME package's build systems. However, that will take a while, since right now, GNOME tarballs have the nasty habit of shipping their own copy of intltool-merge (and others). Not that I blame you for that, it's just what intltoolize does, but that's the way it is. I feel that as a first step, we should have support for this mode in glib, then we can work with intltool developers etc. to get proper build time support. Please also see [2], which is the counterpart this bug for gconf. [1] https://wiki.ubuntu.com/LangpacksDesktopfiles [2] http://bugzilla.gnome.org/show_bug.cgi?id=568845
Created attachment 127542 [details] [review] patch One note: This patch uses X-GNOME-Gettext-Domain. Our Ubuntu patch uses X-Ubuntu-Gettext-Domain, and OpenSUSE's uses X-SUSE-Gettext-Domain. So I figure if this goes upstream, both of our distros will carry a small patch which falls back to that field if X-GNOME isn't found, until all our .desktop files are converted.
For the record, these considerations and this patch are for *.directory files as well.
(In reply to comment #0) > However, that will take a while, since right now, GNOME tarballs have the nasty > habit of shipping their own copy of intltool-merge (and others). Should not happen if tarballs are rolled with a recent-enough intltool, I believe.
You mean that "once intltool gets support for this, static copies will be okay in GNOME tarballs following those", or "recent intltoolize doesn't create static copies any more"? If you mean the latter, the current 2.25.90 tarballs that I looked at still have copies.
The tarballs I release don't have those copies, fwiw ;-)
So, a few questions: a) Why is this not being proposed as an extension for the desktop entry spec on xdg-list ? b) Will this cause every glib app to open all the worlds gettext domains ? That would be a problem, I think.
a) It was already: http://lists.freedesktop.org/archives/xdg/2005-June/005344.html But was rejected unfortunately. The primary reason was your b) question, because at that time the responders on the list didn't seem to realize that all we proposed was a new *optional* field. I'm happy to bring that up again on the list to do a second attempt, unless it's outright rejected by you guys as well. b) The need to open many .mo files is indeed real, but not quite as drastic as you allude to ("every glib app to open all gettext domains"). By and large, it's gnome-panel which has to open the domains for all displayed menu entries (i. e. in the order of 50). As I said, I did some time measurements (https://wiki.ubuntu.com/LangpacksDesktopfiles), and the overhead was not really measurable. However, please consider that this *only* happens if there are no translations in the .desktop file, and it has an X-GNOME-Gettext-Domain field. We decided that a negligible performance overhead is well paid off by the possibility to update translations easily, and even after the release of a distribution. However, we acknowledge that some distributions might have different priorities, and thus wrote the patch in a backwards compatible manner, which doesn't need any gettext overhead at all by default.
I don't see any rejection in that thread. Mark asked the same question I did. Some people claimed there is no performance problem, without providing any numbers. Then Carlos threw a fit and ran off...
> I don't see any rejection in that thread. Right, Carlos just gave up after arguing the same thing over and over. I provided performance measures, don't argue that it *will* cause a certain performance impact, and never asked anyone to use it by default, just allow distros which do want to use it (such as Ubuntu and SUSE) to use a well-defined key without going through the X-{Ubuntu,Debian,SUSE}-Gettext-Domain madness. But as I said, I'm happy to raise the issue at the list again. (Not today any more, though, have to run now.) Thanks, and have a good weekend!
The 'numbers' you point to on that wiki page are a bit meager... - It doesn't say anything about the amount of IO before/after - It doesn't say anything about the memory consumption before/after - It just says that the desktop files have been 'enriched' with the gettext domain - if that means you've just added the extra key, without removing the translations, it doesn't prove anything, since your patch prefers the included translations. On a more fundamental level, what does this do to compatibility ? If you remove the translations from the desktop files, other implementations of the desktop spec (eg kde) will suddenly loose all translations. If you don't remove the translations, then you don't win anything, since your patch prefers the included translations. Therefore, I don't think this is really suitable to be added as an optional X-foo key, and needs to become an integral part of the desktop entry spec.
> The 'numbers' you point to on that wiki page are a bit meager... OK, I'll do some more benchmarks in two weeks, when I'm back from the conference next week. > It just says that the desktop files have been 'enriched' with the gettext domain - if that means you've just added the extra key, without removing the translations, it doesn't prove anything, since your patch prefers the included translations. Indeed, the patch attached here does that. However, the previous patch which we had used in Ubuntu preferred gettext over inline translations, since we did not patch out the inline translations for compatibility reasons. Vincent used "inline > gettext" preference in OpenSUSE, and I followed suit to avoid some problems with user-defined translations, as well as consolidate the patches in OpenSUSE and Ubuntu. > If you remove the translations from the desktop files, other implementations of the desktop spec (eg kde) will suddenly loose all translations. Right, KDE needs to be patched accordingly. (The Ubuntu packages already do that). So I'll do some updated and more precise benchmarks in a bit, *if* that's actually what concerns you. Thanks, Martin
What concerns me is that I don't want to accept this until it makes it into the desktop entry spec and there is buy-in to implement this across the various users of the desktop entry spec. Meaningful numbers might help you in gathering that buy-in.
Created attachment 143224 [details] the previous version had a setlocale typo the update fixes the issue
Comment on attachment 143224 [details] the previous version had a setlocale typo the update fixes the issue the change is not correct ignore the update
(The latest patch changed setlocale(LC_MESSAGES, NULL) to setlocale(LC_MESSAGES, ""), which is obviously not something any glib function should do.) The patch only applies this to .desktop files. However in Gnome, there are many other kinds of keyfiles that might profit from this, e.g. .epiphany-extension, .gedit-plugin, .evince-backend, .gnome-settings-plugin etc files; how about supporting these too?
Wouldn't it be more sensible to prefer localizations from mo files? That way one can start to update locale packages independently, which is the original motivation, and the second step is to strip translations in the desktop files to save space, as soon as other implementations support the feature, which is just an accidental benefit anyway.
(In reply to comment #16) > Wouldn't it be more sensible to prefer localizations from mo files? Once using gettext is the One True Way of translating .desktop files, it makes sense. But for now, I think its more polite to not break third-party installed packages which ship inline translations, or local builds of software which does not do the stripping. However, in fact we had preferred mo files in Ubuntu for quite a while, I just changed it after discussing this stuff with Vincent, and settling for a common patch in Ubuntu and OpenSUSE.
Is there any progress?
this will be solved soon, with the desktop file cache that desrt is working on. no need to keep this bug open until then.