GNOME Bugzilla – Bug 729523
[regression] python3 unicode decoding errors with configparser in _readSettingsFromConfigurationFile prevent startup when run with LANG=C or unrecognized locales
Last modified: 2014-10-22 17:53:30 UTC
On my laptop, I tried out Pitivi git again and got this traceback:
+ Trace 233553
self.settings = GlobalSettings()
self._readSettingsFromConfigurationFile()
self._config.read(conf_file_path)
self._read(fp, filename)
for lineno, line in enumerate(fp, start=1):
return codecs.ascii_decode(input, self.errors)[0]
...which then causes this other traceback, preventing startup: File "./pitivi/application.py", line 143, in _activateCb self.createMainWindow() File "./pitivi/application.py", line 150, in createMainWindow self.gui = PitiviMainWindow(self) File "./pitivi/mainwindow.py", line 176, in __init__ self._createUi() File "./pitivi/mainwindow.py", line 263, in _createUi self.medialibrary = MediaLibraryWidget(self.app, self.uimanager) File "./pitivi/medialibrary.py", line 133, in __init__ self.clip_view = self.app.settings.lastClipView AttributeError: 'NoneType' object has no attribute 'lastClipView'
Created attachment 275835 [details] sample pitivi.conf file I'm suspecting the problem is due to the paths (or section names) with accents and such...
Why are certain [Config Options] in French and others in Spanish? This could be the problem. Maybe you switched locales? Is it fixed when you create a new config file? I guess they should be all in English, since they are not exposed to the user. Funny thing is that I can load a config file with non ASCII paths, as described in 729526. I also can load the attached config.
The reason why my config file has the same config section names in different languages is that it is a result of how the dynamic detachable UI components (tabsmanager.py) works: we use the tab's label to save the attributes related to that tab. But in practice, this worked without a hitch until now. I was expecting the paths to be the problems, rather than the config section names. I'm puzzled by the fact that this occurs on my computer but not yours, with this sample config file...
Okay so I found out how to reproduce this on my end... like bug #729526, this does not occur if I simply run "pitivi" (from inside the pitivi-git-environment) with the default locale, but if I do: $ LANG=C pitivi ...then I hit this traceback. The problem with testing this thoroughly is that I *also* hit a f&%king sefault that I *can't* trace in GDB (sounds like another race) if I do: $ rm ~/.config/pitivi.conf ; LANG=C pitivi ...and close the app's welcome dialog. Which means I can't shutdown the app "normally" when I'm running under "LANG=C" to see what happens when that locale saves the config file. So the only way for me to generate the config file again is to run without "LANG=C", exit pitivi normally, and then I can get the traceback by re-running with LANG=C. And when I do so, in addition to the traceback above (AttributeError: 'NoneType' object has no attribute 'lastClipView'), I also get this traceback:
+ Trace 234034
I am not sure if UTF-8 support should work when you set LANG=C http://stackoverflow.com/questions/10561923/unicodedecodeerror-ascii-codec-cant-decode-byte-0xef-in-position-1 For me LANG is en_US.UTF-8. If you want to use a french locale, you should try for example fr_CA.UTF-8. When I run pitivi with LANG=fr_CA.UTF-8, I get the error that its not installed: (pitivi:25391): Gtk-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. (pitivi:25391): Clutter-WARNING **: Locale not supported by C library. Using the fallback 'C' locale. I installed it using the method in Arch: https://wiki.archlinux.org/index.php/Locale Now Pitivi is in French, and works with a file called /home/bmonkey/Videos/Vidéos/clé FACIL - montage final final.xges It also works with the english UTF-8 locale.
I made a fresh fedora 20 VM and set the locale to french Canadian. I was able to save and open the following file: /home/bmonkey/Vidéos/Sans titre lalalé fooêeôô.xges containting the following asset: /home/bmonkey/Vidéos/éeôô.jpg Could you please try to reproduce this issue with a fresh Fedora installation?
It does work if you run with LANG=fr_CA.UTF8 instead of LANG=C. A problem, however, is that whenever a user runs with a broken or unrecognized locale, this bug will occur. For example, if you start with LANG=fr_QC.UTF8 it will use the fallback "C" locale, cause a segfault and start a referendum. Reinstalling my distro won't change anything to that :)
Created attachment 287467 [details] [review] encoding: exit pitivi on Unicode errors and open an error dialog I think you maybe want something like this. An error dialog about the encoding. I am exiting Pitivi on encoding errors this way, since you will not be able to open a "Łódź.xges" file with LOCALE=C anyway.
I'm wondering if presenting such a dialog is considered an acceptable solution; On that question, - thiblahute doesn't seem to have an opinion - mclasen said "not sure it is really a 'solution', but I don't see much future for non-utf8 locales, so there probably is no solution" ...so I guess I could test and merge that, and call it a day.
Mathieu had an interesting idea: instead of punting the user, shouldn't this code just call setlocale to override to a normal "known working" locale, and issue a warning log/debug statement about it?
Created attachment 288284 [details] [review] modified patch Sifting through http://bugs.python.org/issue19846 I get a feeling that this is a mess that is not worth getting into. And from my testing, I fear that the setlocale trick might not even work: the patch is able to show an error dialog, but utterly unable to "abort" the application's startup (you still get the mainwindow showing up after the error dialog). Attaching a revised patch to: - Make the wording clearer (and more polite) - Reword the commit message - Add a "return" statement inside the except UnicodeDecodeError condition (to make sure we "protect" the settings file?) I think I'd be fine with pushing this patch if you folks agree.
Created attachment 288285 [details] [review] modified patch (minor correction in the commit message: link to the bug report)
Pushed the modified version as: commit c6b12b7df202f7b3467f6967ef52f37b76af84d5 Author: Lubosz Sarnecki <lubosz@gmail.com> Date: Tue Sep 30 17:36:55 2014 +0200 Show an error dialog when encountering Unicode decoding errors/broken locales Fixes bug #729523