GNOME Bugzilla – Bug 323106
IMAP drawers with non-ASCII UTF-8 names do not work
Last modified: 2013-09-10 13:41:34 UTC
Version details: evolution-data-server-1.4.2.1 Distribution/Version: SuSE Linux 10.0 + supplementary GNOME update After update of evolution-data-server to 1.4.2.1 and evolution to 2.4.2, IMAP drawers with non-ASCI UTF-8 names do not work any more. These names are displayed mangled (seems equally like "iconv -f ISO-8859-1 -t utf-8" done on UTF-8 string). Clicking to this mangled name, Evolution reports folder does not exist error. Trying to subscribe also shows mangled names without chance to subscribe. How to reproduce: 1. Start evolution with evolution-data-server-1.4.0 and IMAP account. 2. Create folder containing international characters. 3. Upgrade to evolution-data-server-1.4.2.1 4. Try to read folder created in the step 2. How often it happens: Every time Additional notes: Degrade of evolution-data-server to 1.4.0 and keeping evolution 1.4.2 fixes the problem. Setting mailer default charset from ISO-8859-2 to UTF-8 does not help. I am using cs_CZ.UTF-8 locale. Setting severity to Critical - international users cannot use IMAP.
Additional notes: Degrade to evolution-data-server 1.4.1 also fixes the problem.
same here with evo2.5.2. i cannot access an IMAP folder containing an "ü" currently. confirming.
Ok. This is *not* specific to IMAP. I have a feeling that this is a pango issue. Sigh!!!
And this is surely a duplicate of bug #317794
it's just the IMAP folder most of my mails are in, so i have to use the webmail interface instead. are you sure this is pango? hmm. me not. "CAMEL_DEBUG=all" output: Thread 43364bb0 > CamelStore('imap://andre@imap.example.com:993/;command=ssh%20-C%20-l%20%25u%20%25h%20exec%20/usr/sbin/imapd;use_ssl=always'):get_folder('Unerwünscht', 0) = 0x881c004 < 43364bb0 > sending : A00008 SELECT {12+} Unerwünscht received: A00008 NO Mailbox does not exist CamelException.setv(0x86bbee8, 301, 'IMAP command failed: Mailbox does not exist') Thread 43364bb0 > CamelStore('imap://andre@imap.example.com:993/;command=ssh%20-C%20-l%20%25u%20%25h%20exec%20/usr/sbin/imapd;use_ssl=always'):get_folder('Unbekannt', 0) = 0x881bfb8 < 43364bb0 > sending : A00009 SELECT Unbekannt received: * 0 EXISTS received: * OK [UNSEEN 0] received: * OK [PERMANENTFLAGS (\Deleted \Seen \Answered)] received: * OK [UIDVALIDITY 1] received: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) received: A00009 OK [READ-WRITE] SELECT Completed Thread 43364bb0 > CamelStore('imap://andre@imap.example.com:993/;command=ssh%20-C%20-l%20%25u%20%25h%20exec%20/usr/sbin/imapd;use_ssl=always'):get_folder('Unerwünscht', 0) = 0x881c004 < 43364bb0 > sending : A00010 SELECT {12+} Unerwünscht received: A00010 NO Mailbox does not exist CamelException.setv(0x86af9e0, 301, 'IMAP command failed: Mailbox does not exist')
I guess it is not duplicate of bug 317794. I am using plain IMAP, not IMAP4. If I try IMAP4, I see this bug too, and additionally I see also bug 272058.
Its not specific to IMAP since, GW/Exchange also has a similar issue. Bascaially, whenever a folder name contains a special character, Evolution does not display it properly, or does not display it at all in some cases.
"Unerwünscht" is displayed as "Unerwünscht" here, so it could be the old unicode character conversion problem. running cvs head version from yesterday, evolution crashes when i try to create an IMAP folder named "üüü".
There seems to be some confusion in the imap provider about when mailbox names are in MIME-UTF-7 and when in UTF-8. It seems that there is even double conversion happening from MIME-UTF-7 to UTF-8, and then again from what already is UTF-8 (as if it was MIME-UTF-7) to UTF-8... I think the cleanest way to handle it would be to keep everything in UTF-8 and do any conversion to/from MIME-UTF-8 as late/early as possible. The attached patch seems to fix the problem in HEAD.
Created attachment 55984 [details] [review] Minor patch that seems to help I verified that after this I could create folders with non-ASCII names, rename such folders, and also that such folders created with another client show up correct, and that folders created in Evo show up correct in the other client.
I really belive that this is *not* a provider specific issue. This is either an issue with camel or mailer
Am sorry please ignore the Comment #11. Tors patch is quite right. And as Tor pointed outs we are infact doing a double converstion. Thanks Tor. I think the patch should go in. Testing it right now. Should be in sooon.
Tor the patch is fine. Its ok to commit. Will you commit it?? Should be committed to both HEAD and stable branch.
Argh, I was writing garbage in my comment #9. When I said "MIME-UTF-7" what I meant was "IMAP-UTF-7"... sorry. Acronym overload. Anyway, yes, I can commit the fix to HEAD, but I don't have a checkout of the stable branch, could someone else commit it there please.
Patch committed to HEAD. 2005-12-14 Tor Lillqvist <tml@novell.com> Fix #323106: * camel-imap-store.c (camel_imap_store_summary_full_to_path, camel_imap_store_summary_path_to_full): Remove extraneous conversions from IMAP-UTF-7 to UTF-8 and back. * camel-imap-command.c (imap_command_strdup_vprintf): Do the conversion from UTF-8 to IMAP-UTF-7 only just before sending a request. Do it also for %S formats, as that is what the CREATE command uses. But %S is used also for other commands (like LOGIN and LIST), so maybe we really would need a new format that would be used *only* for mailbox (folder) names, assuming it's only mailbox names that use the IMAP-UTF-7 encoding. Or is that what %F is intended for? But why then does CREATE use %S?
patch committed to stable branch on behalf of Tor
not fixed. now i can access my mails in that folder, but the folder name is still incorrectly displayed (two chars instead of one) - i'm running 2.5.3 rpm 2005-12-19 here. also, i wonder myself what to tell all those people running 1.4.2.1? "wait until 1.4.3 gets out in february"? be aware of the fact that people running 1.4.2x are not able AT ALL to access mails in that particular folders for the next five weeks. ain't cool. :-/
*** Bug 325240 has been marked as a duplicate of this bug. ***
Any chance for a .2 release any soon? Being forced to use SquirrelMail is quite annoying ;)
Applied the patch to PLD Linux repositories. As reported by Andre, folders are now accessible but still displayed incorrectly. Anyway, this fixes our users' problems for a while. Looking forward to 1.4.3.
What are the folder names that are having problems?? Could you give a few names which i could try. Since the one "ü" and similar characters, render quite well and are accessible too.
partha: they do not render quite well at all here, "Unerwünscht" is still displayed as "Unerwünscht" - it's also "Unerwünscht" when i use the "rename" context menu entry. evo2.5.3 rpm 2005-12-19; suse9.3.
Created attachment 56720 [details] screen shot with the folder name Andre: So i created the folder name mentioned in the last comment, find the attached screen-shot. Hmm... Not sure what is wrong. And that too when the folder is being listed and can be renamed.
partha, i tried to delete that folder locally, but after syncing with my IMAP server its name was again wrong. if i try to rename that folder locally from "Unerwünscht" to "Unerwünscht", i get the error message: "A folder name "Unerwünscht" already exists".
Andre, can you try removing the .ev-store-summary file in your $HOME/mail/imap/<mail-account-name>/ directorey. (Back it up first) ;). And see if you still have the porblem??
have to say that my IMAP evolution-virtual-junk folder is also called "Unerwünscht" by evolution. (this worked pretty fine up to now.) well, i removed both folders, "Unerwünscht" and "Unerwünscht" (i had both). the "Unerwünscht" folder gets recreated, it's called "Unerwünscht" both in the UI and as folder name on the harddisk.
Adnre, can i have the camel traces export CAMEL_VERBOSE_DEBU=1 and launch evo from the terminal (You already know all this :D) and just attach the soap traces here. Also could you send me your .evolution/mail/imap/<account>/.ev-store-summary file to me personally. I wanted to check if the encoding is fine. TIA
as Tor noted in his comment with the patch, %S is used by other commands that are not meant to have their strings re-encoded into UTF-7, and so the patch is wrong. I would say that the correct fix is to fix the code that passes folder names as %S to: 1. use %F if possible (tho maybe there's a reason they use %S instead of %F?) or 2. do the charset conversion and THEN pass as %S in the calling function ok... looking at the code, looks like the correct solution would be #2, and to make it easier, probably make a new formatter option %N (for Namespace? or maybe %P for "path"? since that's what %S's are used for in the code where a folder path is passed in). This new formatter would then do the UTF-7 conversion same as the %F currently does, but we wouldn't call the path_to_full() on it, since it wouldn't necessarily be in the summary.
(the fact that the paths are not necessarily in the summary is why they are passed as %S's btw)
-> bug 326385 for this issue
*** Bug 326587 has been marked as a duplicate of this bug. ***
Fixed: The actual issue is solved. The problem why the fix is not being seen is because of an other bug. To see this fix just remove .ev-store-summary in your .evolution/mail/imap/<account-name>/.ev-store-summary and restart evolution. The folder should work.
Shreyas: what version fixed the problem?
the fix was committed two days ago on 2006-01-17, see http://cvs.gnome.org/viewcvs/evolution-data-server/camel/providers/imap/camel-imap-store.c?rev=1.340&view=log it will be included in evolution 2.4.3 (release date: 2006-02-08) and the unstable, experimental version 2.5.6.
In evolution-data-server-1.5.90cvs20060206 it seems to work, but after update it reports following warnings: (evolution:29186): camel-WARNING **: Trying to add folder 'INBOX/SuSE CR/balíky' to summary that already has it
I'm still able to reproduce this with evo 2.4.2.1 & eds 1.4.2.1 with patched camel-imap-command.c (1.71.2.3), camel-imap-store-summary.c (1.16.2.1) & camel-imap.store.c (1.330.2.3). Has anyone confirmed that this is actually fixed with versions < 2.5/1.5?
heikki: remove the folder in your harddisk cache at /.evolution/mail/imap/<account name>/folders/<folder name> and resync with the server, that worked for me
Andre: thanks, I will bring that back to the corresponding debian-bug-reports :)