GNOME Bugzilla – Bug 512752
The Qt theme engine interferes with Gnumeric's locale
Last modified: 2008-02-02 18:04:28 UTC
Please describe the problem: I have a csv file containing students grades. under gnumeric 1.6.3 loading the file (from the menu) rounds all decimal numbers under gnumeric 1.8.1 the import is still incorrect but less silent (and thus less dangerous) : in this case, the grade is replaced by what seems to be a time. this problem seems to be related to the fact that in my (french) file, decimal number are using comas ',' and not dots '.' very strangely, EVERYTHING IS FINE when loading from the command line Steps to reproduce: 1. export LANG=fr_FR.UTF8 2. execute gnumeric test.csv 3. see grade for student xxx is 8.5 (everything ok) 4. close gnumeric 5. execute gnumeric 6. click file, open, select test.csv 7. see grade for student xxx is 8,00 (gnumeric 1.6.3) or 0::00::09 (gnumeric 1.8.1) Actual results: the import filter fails. more importantly, under gnumeric 1.6.3 it fails SILENTLY (and caused me quite some troubles with my students) Expected results: the filter should load everything correctly or at least warn the user that something wrong is going on. moreover the behaviour should be the same when loading from the command line and loading from the menu. Does this happen every time? yes Other information: here is a csv file to test: xxx;yyy;;;;8,5;;
I cannot reproduce this. What does "printenv" output for you? Where did you get the Gnumeric binary from? Any messages on stderr?
(In reply to comment #1) > I cannot reproduce this. > > What does "printenv" output for you? SSH_AGENT_PID=29592 SHELL=/bin/bash TERM=mlterm WINDOWID=37748741 USER=wagnerf LD_LIBRARY_PATH=:/home/wagnerf/mpi/lib:/home/wagnerf/kaapi_trunk/lib:/home/wagnerf/mpi/lib:/home/wagnerf/kaapi_trunk/lib LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:su=37;41:sg=30;43:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.svgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.bz2=01;31:*.bz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36: GNOME_KEYRING_SOCKET=/tmp/keyring-N8LzTC/socket SSH_AUTH_SOCK=/tmp/ssh-TjOiY29533/agent.29533 USERNAME=wagnerf KONSOLE_DCOP=DCOPRef(konsole-29909,konsole) PATH=/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:/home/wagnerf/kaapi_trunk/bin:/home/wagnerf/mpi/bin:/home/wagnerf/kaapi_trunk/bin:/home/wagnerf/mpi/bin DESKTOP_SESSION=default QT_IM_MODULE=scim GDM_XSERVER_LOCATION=local PWD=/tmp KONSOLE_DCOP_SESSION=DCOPRef(konsole-29909,session-1) XMODIFIERS=@im=SCIM JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun GNOME_KEYRING_PID=29530 LANG=fr_FR.UTF-8 GDM_LANG=fr_FR.UTF-8 PS1=<_\u@\h_:\w> $ MLTERM=2.9.4 GDMSESSION=default HISTCONTROL=ignoredups COLORFGBG=default;default SHLVL=4 HOME=/home/wagnerf LOGNAME=wagnerf XDG_DATA_DIRS=/usr/local/share/:/usr/share/:/usr/share/gdm/ DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-OqNO6Tlx4z,guid=bb04b07f5cb6c93c39ef729e479f2b7a LESSOPEN=| /usr/bin/lesspipe %s WINDOWPATH=7 DISPLAY=:0.0 GTK_IM_MODULE=scim LESSCLOSE=/usr/bin/lesspipe %s %s XAUTHORITY=/home/wagnerf/.Xauthority COLORTERM= _=/usr/bin/printenv > Where did you get the Gnumeric binary from? gnumeric 1.8.1 is from debian unstable gnumeric 1.6.3 is from debian etch > Any messages on stderr? nothing : <_wagnerf@chippewa_:/tmp> $ gnumeric Reading file:///tmp/test.csv maybe this can help : if I do a "export LANG=C" and "gnumeric test.csv" gnumeric loads only two cells and takes "," as a separator instead of ";"
Created attachment 103963 [details] example csv file mishandled the 8,5 grade in this file loads badly
Could you add a step 8 "Save as test-181/163.gnumeric" as appropriate? Then please attach the resulting files.
I can't reproduce either (using debian in french locale).
(In reply to comment #5) > I can't reproduce either (using debian in french locale). > hi everyone, I've been testing this bug on different machines and also haved some troubles reproducing it. after a few tests I discovered that this bug was only under my user account. So i searched and finally found something if I remove my ~/.gtkrc-2.0 everything returns back to normal (tested for 1.8.1 only) here is my gtkrc : # This file was written by KDE # You can edit it in the KDE control center, under "GTK Styles and Fonts" include "/usr/share/themes/Qt/gtk-2.0/gtkrc" style "user-font" { font_name="Sans Serif 12" } widget_class "*" style "user-font" gtk-theme-name="Qt" gtk-font-name="Sans Serif 12" as you can see i'm using the theme engine from the debian package gtk-qt-engine i've also been quickly looking at stf-parse.c under gdb and the stf_parse_sheet function takes the same arguments when called from the command line and called from the file menu. so I still don't have any idea what is going on. hope this helps.
That theme seems to have gone out with SuSE 10.3. I'll have to come up with another way of testing. I am guessing that something in the Qt library (and/or the way the theme calls it) messes up the locale settings.
(In reply to comment #6) > after a few tests I discovered that this bug was only under my user account. > So i searched and finally found something > > if I remove my ~/.gtkrc-2.0 everything returns back to normal (tested for > 1.8.1 only) > > here is my gtkrc : I can reproduce it with gtk-qt-engine installed and this .gtkrc-2.0 in place (and also when it's trimmed down to just the include and gtk-theme-name lines). (In reply to comment #7) > That theme seems to have gone out with SuSE 10.3. I'll have to come up > with another way of testing. Perhaps you can use "alien" to install the .deb? > I am guessing that something in the Qt library (and/or the way the theme > calls it) messes up the locale settings. If it is, it's doing so in a fairly subtle way. For me, the whole list of locale environment variables (as queried through =getenv inside gnumeric) has the same values with or without the .gtkrc-2.0 in place when testing: LC_CTYPE,en_US.UTF-8 LC_NUMERIC,#N/A LC_TIME,#N/A LC_COLLATE,C LC_MONETARY,#N/A LC_MESSAGES,en_US LC_PAPER,#N/A LC_NAME,#N/A LC_ADDRESS,#N/A LC_TELEPHONE,#N/A LC_MEASUREMENT,#N/A LC_IDENTIFICATION,#N/A LC_ALL,#N/A
Got it (with the suse 10.2 package). Here's what I get in format_match_simple: (gdb) p text $5 = 0x85ec3ab "8,5" (gdb) set print elements 2222 (gdb) p (char*)setlocale(6,(void*)0) $11 = 0x824a1c0 "LC_CTYPE=fr_BE.UTF-8;LC_NUMERIC=C;LC_TIME=fr_BE.UTF-8;LC_COLLATE=fr_BE.UTF-8;LC_MONETARY=fr_BE.UTF-8;LC_MESSAGES=fr_BE.UTF-8;LC_PAPER=fr_BE.UTF-8;LC_NAME=fr_BE.UTF-8;LC_ADDRESS=fr_BE.UTF-8;LC_TELEPHONE=fr_BE.UTF-8;LC_MEASUREMENT=fr_BE.UTF-8;LC_IDENTIFICATION=fr_BE.UTF-8" Have a look at that LC_NUMERIC in there!
And here's the culprit:
+ Trace 187800
..and over at http://www.google.com/codesearch?hl=en&q=+qt_init_internal+setlocale+show:qQboaVyqX_g:WWOK9Y3hHF0:kIvLDZZD7c0&sa=N&cd=1&ct=rc&cs_p=ftp://ftp.trolltech.com/qt/source/qt-1.44.tar.gz&cs_f=qt-1.44/src/kernel/qapplication_x11.cpp#l451 you can see the code Qt initialization code... setlocale( LC_ALL, "" ); // use correct char set mapping setlocale( LC_NUMERIC, "C" ); // make sprintf()/scanf() work if ( XSupportsLocale() && ( qstrlen(XSetLocaleModifiers( "" )) || qstrlen(XSetLocaleModifiers( "@im=none" ) ) ) ) xim = XOpenIM( appDpy, 0, 0, 0 ); else xim = 0; which is pathetic for a library.
I installed a work-around: using an idle handler, I re-set the locale when the gui comes up. This is clearly a bug in the Qt theme and/or the qt library. This problem has been fixed in our software repository. The fix will go into the next software release. Thank you for your bug report.
I can still reproduce the problem with current svn (trunk and 1.8): when the .gtkrc-2.0 is in place, the "8,5" entry gets imported as "0:00:9"; when it is not in place, the entry gets imported correctly.
Hmm. Not good. Could you please set a breakpoint in format_match_simple and do (gdb) p (char*)setlocale(6,(void*)0) $4 = 0x85059f0 "fr_BE.UTF-8"
(In reply to comment #10) > And here's the culprit: > ... > > which is pathetic for a library. > And from a Qt Mailinglist: > I ran the following command in the 4.3 and 4.4 branches: > fgrep setlocale src/*/*/*.cpp > and found one less occurrence of setlocale in 4.4 in qcoreapplication.cpp: > setlocale(LC_NUMERIC, "C"); > > This was removed just a few months ago and the relevant comment is: > The numerical locale being the POSIX locale in > QCoreApplication::init() seems to be a largely > historical consideration, which was necessary > before the introduction of QLocale in Qt 3.3. > Problems can consequently arise when Qt is used > in conjunction with 3rd party libraries which > respect the system numerical locale. It's fixed in Qt 4.4
JHM: I have tried what I can think of and I cannot reproduce that problem. Could you try to debug it? format_match_simple needs to recognize the text as a number. (You getting a time value means that the text is not recognized as a number -- the time code should not get reached for 8,5 in a comma locale.) Thanks.
(In reply to comment #13) > Could you please set a breakpoint in format_match_simple and do > (gdb) p (char*)setlocale(6,(void*)0) With LANG=fr_FR.UTF8: $1 = 0x8605b0 "LC_CTYPE=en_US.UTF-8;LC_NUMERIC=C;LC_TIME=en_US.UTF-8;LC_COLLATE=C;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US;LC_PAPER=en_US.UTF-8;LC_NAME=en_US.UTF-8;LC_ADDRESS=en_US.UTF-8;LC_TELEPHONE=en_US.UTF-8;LC"... With LC_ALL=fr_FR.UTF8: (gdb) p (char*)setlocale(6,(void*)0) $1 = 0x879270 "LC_CTYPE=fr_FR.UTF8;LC_NUMERIC=C;LC_TIME=fr_FR.UTF8;LC_COLLATE=fr_FR.UTF8;LC_MONETARY=fr_FR.UTF8;LC_MESSAGES=fr_FR.UTF8;LC_PAPER=fr_FR.UTF8;LC_NAME=fr_FR.UTF8;LC_ADDRESS=fr_FR.UTF8;LC_TELEPHONE=fr_FR."...
Stack trace as discussed on IRC. GNU gdb 6.7.1-debian Copyright (C) 2007 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu"... Using host libthread_db library "/lib/libthread_db.so.1". (gdb) set args /tmp/test.csv (gdb) break qt_init_internal Function "qt_init_internal" not defined. Make breakpoint pending on future shared library load? (y or [n]) Breakpoint 1 (qt_init_internal) pending. (gdb) run Starting program: /home/ray/debian/packages/maintained/gnumeric/devel/gnumeric-dev/build/src/.libs/gnumeric /tmp/test.csv [Thread debugging using libthread_db enabled] [New Thread 0x2b1ebd7378e0 (LWP 10139)] Breakpoint 2 at 0x2b1ebf61087e Pending breakpoint "qt_init_internal" resolved
+ Trace 187854
Thread 47411027474656 (LWP 10139)
The program is running. Exit anyway? (y or n)
Fixed the file-from-command-line case too.