After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 583823 - Could not sync files (folder to folder) with umlaute in it
Could not sync files (folder to folder) with umlaute in it
Status: RESOLVED WONTFIX
Product: conduit
Classification: Other
Component: core
0.3.x
Other Linux
: Normal critical
: ---
Assigned To: conduit-maint@gnome.bugs
conduit-maint@gnome.bugs
gnome[unmaintained]
: 584254 585227 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2009-05-25 17:36 UTC by Bernd Schlapsi
Modified: 2018-07-01 09:04 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
convert strings to unicode, to prevent exception on python 2.6 (690 bytes, patch)
2009-08-13 23:22 UTC, sam tygier
none Details | Review

Description Bernd Schlapsi 2009-05-25 17:36:55 UTC
I am using the latest svn-version (Revision 1931)

[datatypes.DataType  ][DEBUG  ]  Getting Rid for file:///media/nas_data/save/Documents/dell/akku-problem/LED-Ladegerät.JPG (DataType.py:158)
Exception in thread Thread-2:
Traceback (most recent call last):
  • File "/usr/lib/python2.6/threading.py", line 525 in __bootstrap_inner
    self.run()
  • File "/opt/conduit/conduit/Database.py", line 304 in run
    self.cur.execute(req, args)
ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.


[datatypes.DataType  ][DEBUG  ]  Getting Rid for file:///media/nas_data/save/Documents/presentation/openstreetmap/ct-200809-Straßenmeister.pdf (DataType.py:158)
Exception in thread Thread-2:
Traceback (most recent call last):
  • File "/usr/lib/python2.6/threading.py", line 525 in __bootstrap_inner
    self.run()
  • File "/opt/conduit/conduit/Database.py", line 304 in run
    self.cur.execute(req, args)
ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings.

Comment 1 John Stowers 2009-05-25 20:16:37 UTC
I just saw this yesterday to. It seems python 2.6 handling of unicode has broken many things. *sigh*
Comment 2 Bernd Schlapsi 2009-06-03 14:56:52 UTC
This is a critical bug for me because I couldn't sync without a error. Is there a workaround or some plans for the bugfix?
Comment 3 Dennis Benndorf 2009-07-17 12:38:00 UTC
Bug exist also for german "umlaute". Its in the trunk and the repo-version for ubuntu.
Comment 4 sam tygier 2009-08-13 20:47:11 UTC
this is the same as bug #585227
Comment 5 sam tygier 2009-08-13 20:48:32 UTC
*** Bug 585227 has been marked as a duplicate of this bug. ***
Comment 6 sam tygier 2009-08-13 23:21:10 UTC
possible explanation and fix.
between version 2.5 and 2.6 of python there were changes in str and unicode types to make way for python3. 
this somehow makes sqlite fussy about receiving strings with interesting characters.
conduit is not the only project effected ( http://code.djangoproject.com/ticket/7921 )
a solution seems to be converting strings to unicode objects before passing to the database.

to trigger the issue in conduit you just need to synk to folders where there is a file with an interesting character.

mkdir 1 2
touch 1/foö

after the file gets copied across in Synchonization.py:put_data(), the mapping gets updated

    #Update the mapping and save
    mapping.set_source_rid(sourceDataRid)
    mapping.set_sink_rid(sinkRid)
    conduit.GLOBALS.mappingDB.save_mapping(mapping)

save_mapping() inserts a bunch of values that it gets from Mapping.values().

By forcing all those values from str to unicode the ProgrammingError is avoided.

Comment 7 sam tygier 2009-08-13 23:22:30 UTC
Created attachment 140725 [details] [review]
convert strings to unicode, to prevent exception on python 2.6
Comment 8 sam tygier 2009-08-14 00:23:39 UTC
*** Bug 584254 has been marked as a duplicate of this bug. ***
Comment 9 matmoser 2009-08-14 07:33:16 UTC
I tried the latest SVN with Sams patch and it fixed the "You must not use 8-bit bytestrings" for me. "Umlaute" like ä,ü,ö work without a problem, but still some chars don't, e.g.:

DEBUG:datatypes.DataType:Getting Rid for file:///home/matt/Dokumente/WU/FIWI PS/praesi/Pr%C3%A4sentation%20Chris%20FIWI.ppt
[Syncronization      ][CRITICAL]  UNKNOWN SYNCHRONIZATION ERROR
Traceback (most recent call last):
  • File "/home/matt/Downloads/conduit/conduit/Synchronization.py", line 743 in run
    self.one_way_sync(self.source, sink)
  • File "/home/matt/Downloads/conduit/conduit/Synchronization.py", line 496 in one_way_sync
    dataRid = data.get_rid()
  • File "/home/matt/Downloads/conduit/conduit/datatypes/DataType.py", line 161 in get_rid
    mtime=self.get_mtime(),
  • File "/home/matt/Downloads/conduit/conduit/datatypes/File.py", line 293 in get_mtime
    ts = self._file.get_mtime()
  • File "/home/matt/Downloads/conduit/conduit/platform/FileGio.py", line 78 in get_mtime
    mtime = self.fileInfo.get_attribute_uint64('time::modified')
AttributeError: 'NoneType' object has no attribute 'get_attribute_uint64'
 (Synchronization.py:751)
CRITICAL:Syncronization:UNKNOWN SYNCHRONIZATION ERROR
Traceback (most recent call last):
  • File "/home/matt/Downloads/conduit/conduit/Synchronization.py", line 743 in run
    self.one_way_sync(self.source, sink)
  • File "/home/matt/Downloads/conduit/conduit/Synchronization.py", line 496 in one_way_sync
    dataRid = data.get_rid()
  • File "/home/matt/Downloads/conduit/conduit/datatypes/DataType.py", line 161 in get_rid
    mtime=self.get_mtime(),
  • File "/home/matt/Downloads/conduit/conduit/datatypes/File.py", line 293 in get_mtime
    ts = self._file.get_mtime()
  • File "/home/matt/Downloads/conduit/conduit/platform/FileGio.py", line 78 in get_mtime
    mtime = self.fileInfo.get_attribute_uint64('time::modified')
AttributeError: 'NoneType' object has no attribute 'get_attribute_uint64'

The files name actually is "Pr%C3%A4sentation%20Chris%20FIWI.ppt", same with a file named Zwischenpr�ntation.zip (which actually is a corrupted cp1252/utf-8 filename from a windows machine)

This is just a handful of files which I can rename by now, but conduit still should now how to handle them. Anyway thanks for the patch!
Comment 10 André Klapper 2018-07-01 09:04:24 UTC
Conduit is not under active development anymore, has not seen code changes for eight years, and saw its last tarball release in 2010.
Its codebase has been archived: https://gitlab.gnome.org/Archive/conduit/commits/master

Closing this report as WONTFIX as part of Bugzilla Housekeeping to reflect reality. Please feel free to reopen this ticket (or rather transfer the project to GNOME Gitlab, as GNOME Bugzilla is deprecated) if anyone takes the responsibility for active development again.