GNOME Bugzilla – Bug 535205
Incorrect alphabetical order of localized language names
Last modified: 2009-11-02 07:58:55 UTC
Please describe the problem: Langauges are in alphabetical order on sites http://l10n.gnome.org/teams and /languages. But this order is the "standard" US English one, e. g. that's why are languages with accented initial letters at the end of the Hungarian list. The default collation of the current locale would be much better (for searching a language and for the "that is localized" feeling). Steps to reproduce: Actual results: Expected results: Does this happen every time? Other information:
This is also the case for Thai. Probably, LC_COLLATE-based sorting should be applied in trans_sort_object_list() [common/utils.py] ?
The sorting problem in Python web applications is not trivial. If you find good resources about this, feel free to add them in a comment. The most promising resource I found is pyuca: http://jtauber.com/blog/2006/01/27/ but I didn't try to implement this so far. See also: http://www.cmlenz.net/archives/2008/07/the-truth-about-unicode-in-python
Created attachment 126410 [details] [review] Some idea I'm not quite familiar with Python, but hoping my C apperience applies somehow.. This is what I had in mind when proposing the idea. But it's not tested, and probably needs some encoding conversion between Python's Unicode strings and locale encodings.
The second link in my comment #2 explains why the locale module of Python is not suitable for this functionality in web applications.
*** Bug 547207 has been marked as a duplicate of this bug. ***
(In reply to comment #4) > The second link in my comment #2 explains why the locale module of Python is > not suitable for this functionality in web applications. Why? Isn't the "get_language()" as provided by Django [1] sufficient for determining the currently translated language? It's supposed to be the same one as what's used in translation, with the same criteria upon many conditions as described in Django document [2]. [1] http://djangoapi.matee.net/django.utils.translation-module.html [2] http://docs.djangoproject.com/en/dev/topics/i18n/?from=olddocs So, once the language, and then the locale, is determined, why isn't it possible to temporarily set the locale for sorting? What's described in your given link probably means Python for web in general. But as we have all necessary information from Django, I don't think we are under that limited condition.
Quoting the blog: "Python has a locale module, but unfortunately (for web applications, at least), it manages the locale as a global, process-wide setting. It's basically just a thin wrapper on top of the POSIX locale functionality. For command-line scripts and desktop applications, this works okay, but for web applications the locale depends on the current request (for example based on the Accept-Language HTTP header.) The locale module is basically unusable here if you're not using a CGI-like request handling model, that is without multi-threading or asynchronous processing. This module also provides some basic date and number formatting, but due to to the global nature of the locale, that is not usable in web applications either." The problem is basically that the locale Python module is not threadsafe. From the official docs: "setlocale() is not thread safe on most systems. Applications typically start with a call of import locale locale.setlocale(locale.LC_ALL, '') This sets the locale for all categories to the user’s default setting (typically specified in the LANG environment variable). If the locale is not changed thereafter, using multithreading should not cause problems." Now a Python expert's opinion would be much appreciated.
(In reply to comment #7) > The problem is basically that the locale Python module is not threadsafe. OK. I see. Mutex probably? Otherwise, UCA may be the last resort. But I just prefer LC_COLLATE, IMO. Locale-based customization is sometimes necessary.
PS: Sorry that I can't do much on this bug now.. My harddisk just crashed yesterday, and I'm replying from other people's machine.
Just for a reminder, http://pypi.python.org/pypi/PyICU could be a good candidate to implement correct sorting (python-pyicu package).
commit 5513502b01849c521e7e2a53ea5e9e8924ff908b Author: Claude Paroz <claude@2xlibre.net> Date: Sat Oct 31 16:34:17 2009 +0100 Sort team/languages according to current language collation (Fixes #535205) Correct string sorting is not trivial to do with standard Python libs, as the locale module is not thread-safe and requires the corresponding locales be installed at the system level. We therefore use the PyICU module which is a wrapper around the ICU libraries. http://site.icu-project.org/ Just waiting for someone installing the python-pyicu package on the server.
Hopefully fixed now!