After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 729526 - [regression] Python3 unicode decoding error when dealing with project file paths to compare backup file mtimes
[regression] Python3 unicode decoding error when dealing with project file pa...
Status: RESOLVED INVALID
Product: pitivi
Classification: Other
Component: General
Git
Other Linux
: Normal major
: 0.94
Assigned To: Pitivi maintainers
Pitivi maintainers
Depends on:
Blocks: 729527
 
 
Reported: 2014-05-04 17:34 UTC by Jean-François Fortin Tam
Modified: 2014-10-11 19:21 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Jean-François Fortin Tam 2014-05-04 17:34:40 UTC
I was trying to load this project that was showing up in my "recently used" projects in the welcome dialog:

"/home/jeff/Vidéos/xges files that won't render because they're borked/just Big Buck Bunny.xges"


...which led to this traceback, and the project not loading at all:
  • File "./pitivi/dialogs/startupwizard.py", line 106 in _loadCb
    self.app.project_manager.loadProject(uri)
  • File "./pitivi/project.py", line 190 in loadProject
    time_diff = os.path.getmtime(backup_path) - os.path.getmtime(path)
  • File "/usr/lib64/python3.3/genericpath.py", line 54 in getmtime
    return os.stat(filename).st_mtime
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 14: ordinal not in range(128)

Comment 1 Mathieu Duponchelle 2014-05-05 21:11:47 UTC
Thanks for the feedback, but can you please provide more specific ways to reproduce ? This kind of report is basically useless if we can't reproduce :)
Comment 2 Lubosz Sarnecki 2014-05-06 12:05:41 UTC
Why are certain [Config Options] in French and others in Spanish?
I guess they should be all in English, since they are not exposed to the user.

First I thought the problem is caused by the non ASCII character in your path, but I could not reproduce it with 

last-folder = /home/bmonkey/Vidéos/xges files that won't render because they're borked

The xges was located in the same folder. The project file was called "just Big
Buck Bunny.xges".


The problem is in the function which tries to determine the "mtime" of a path. The encoding error happens here, since a non ASCII character is used in a bytestring. Python 3 is very restrictive here.

time_diff = os.path.getmtime(backup_path) - os.path.getmtime(path)

Could you please add this print statement around line 190 in your project.py:

        print (type(path), path)
        print (type(backup_path), backup_path)
        
        try:
            time_diff = os.path.getmtime(backup_path) - os.path.getmtime(path)
            self.debug('Backup file is %d secs newer: %s', time_diff, backup_path)

For me the print output is:

<class 'str'> /home/bmonkey/Vidéos/xges files that won't render because they're borked/just Big Buck Bunny.xges
<class 'str'> /home/bmonkey/Vidéos/xges files that won't render because they're borked/just Big Buck Bunny.xges~

And it does not crash.

The char from your exception is '\xe9' or 'é', the first non ASCII character in "Vidéos".

In [1]: "\xe9"
Out[1]: 'é'


Could you provide your xges file?
Comment 3 Jean-François Fortin Tam 2014-08-31 21:15:19 UTC
$ LANG=C pitivi
and then opening this file, with the welcome dialog's recently used file list:

~/Vidéos/montage/2014 FACIL clé multiboot/clé FACIL - montage final final.xges


...yields this traceback:
  • File "pitivi/dialogs/startupwizard.py", line 106 in _loadCb
    self.app.project_manager.loadProject(uri)
  • File "pitivi/project.py", line 188 in loadProject
    print("path is", path)
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 14: ordinal not in range(128)




And if you try to do it directly from the commandline:
$ LANG=C pitivi "/home/jeff/Vidéos/montage/2014 FACIL clé multiboot/clé FACIL - montage final final.xges"

Failed to initialize modules
Traceback (most recent call last):
  File "/home/jeff/pitivi-git/pitivi/bin/pitivi", line 132, in <module>
    _initialize_modules()
  File "/home/jeff/pitivi-git/pitivi/bin/pitivi", line 112, in _initialize_modules
    initialize_modules()
  File "pitivi/check.py", line 255, in initialize_modules
    from gi.repository import Gdk
  File "<frozen importlib._bootstrap>", line 1567, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1534, in _find_and_load_unlocked
  File "/usr/lib64/python3.3/site-packages/gi/importer.py", line 68, in load_module
    dynamic_module._load()
  File "/usr/lib64/python3.3/site-packages/gi/module.py", line 296, in _load
    self._overrides_module = importlib.import_module('gi.overrides.' + self._namespace)
  File "/usr/lib64/python3.3/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/usr/lib64/python3.3/site-packages/gi/overrides/Gdk.py", line 396, in <module>
    initialized, argv = Gdk.init_check(sys.argv)
UnicodeEncodeError: 'utf-8' codec can't encode character '\udcc3' in position 14: surrogates not allowed



....BUT! This only happens when I run with LANG=C. When I run with my default language (French Canadian), I don't get this traceback, in both situations.
Comment 4 Lubosz Sarnecki 2014-09-24 13:36:23 UTC
I made a new Fedora 20 VM and created the following file with Pitivi:

/home/bmonkey/Vidéos/montage/2014 FACIL clé multiboot/clé FACIL - montage final final.xges

I was able to open it in the "recently used" dialog without any exceptions.

Could you try to reproduce the issue on a fresh VM?
Comment 5 Jean-François Fortin Tam 2014-10-11 19:21:07 UTC
The "fix" for bug #729523 (considering that LANG=C is not valid) makes this moot.