After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 323106 - IMAP drawers with non-ASCII UTF-8 names do not work
IMAP drawers with non-ASCII UTF-8 names do not work
Status: RESOLVED FIXED
Product: evolution-data-server
Classification: Platform
Component: Mailer
1.4.x (obsolete)
Other Linux
: Normal critical
: ---
Assigned To: parthasarathi susarla
Evolution QA team
: 325240 326587 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2005-12-03 14:12 UTC by Stanislav Brabec
Modified: 2013-09-10 13:41 UTC
See Also:
GNOME target: ---
GNOME version: 2.11/2.12


Attachments
Minor patch that seems to help (1.85 KB, patch)
2005-12-14 14:16 UTC, Tor Lillqvist
committed Details | Review
screen shot with the folder name (75.56 KB, image/png)
2006-01-03 17:03 UTC, parthasarathi susarla
  Details

Description Stanislav Brabec 2005-12-03 14:12:25 UTC
Version details: evolution-data-server-1.4.2.1
Distribution/Version: SuSE Linux 10.0 + supplementary GNOME update

After update of evolution-data-server to 1.4.2.1 and evolution to 2.4.2, IMAP
drawers with non-ASCI UTF-8 names do not work any more.

These names are displayed mangled (seems equally like "iconv -f ISO-8859-1 -t
utf-8" done on UTF-8 string). Clicking to this mangled name, Evolution reports
folder does not exist error. Trying to subscribe also shows mangled names
without chance to subscribe.

How to reproduce:
1. Start evolution with evolution-data-server-1.4.0 and IMAP account.
2. Create folder containing international characters.
3. Upgrade to evolution-data-server-1.4.2.1
4. Try to read folder created in the step 2.

How often it happens:
Every time

Additional notes:
Degrade of evolution-data-server to 1.4.0 and keeping evolution 1.4.2 fixes the
problem.
Setting mailer default charset from ISO-8859-2 to UTF-8 does not help.
I am using cs_CZ.UTF-8 locale.
Setting severity to Critical - international users cannot use IMAP.
Comment 1 Stanislav Brabec 2005-12-05 11:33:09 UTC
Additional notes:
Degrade to evolution-data-server 1.4.1 also fixes the problem.
Comment 2 André Klapper 2005-12-06 00:20:46 UTC
same here with evo2.5.2. i cannot access an IMAP folder containing an "ü"
currently. confirming.
Comment 3 parthasarathi susarla 2005-12-06 07:43:58 UTC
Ok. This is *not* specific to IMAP.
I have a feeling that this is a pango issue.
Sigh!!!
Comment 4 parthasarathi susarla 2005-12-06 08:38:29 UTC
And this is surely a duplicate of bug #317794
Comment 5 André Klapper 2005-12-06 10:11:33 UTC
it's just the IMAP folder most of my mails are in, so i have to use the webmail
interface instead. are you sure this is pango? hmm. me not.


"CAMEL_DEBUG=all" output:

Thread 43364bb0 >
CamelStore('imap://andre@imap.example.com:993/;command=ssh%20-C%20-l%20%25u%20%25h%20exec%20/usr/sbin/imapd;use_ssl=always'):get_folder('Unerwünscht',
0) = 0x881c004
< 43364bb0 >
sending : A00008 SELECT {12+}
Unerwünscht
received: A00008 NO Mailbox does not exist
CamelException.setv(0x86bbee8, 301, 'IMAP command failed: Mailbox does not exist')
Thread 43364bb0 >
CamelStore('imap://andre@imap.example.com:993/;command=ssh%20-C%20-l%20%25u%20%25h%20exec%20/usr/sbin/imapd;use_ssl=always'):get_folder('Unbekannt',
0) = 0x881bfb8
< 43364bb0 >
sending : A00009 SELECT Unbekannt
received: * 0 EXISTS
received: * OK [UNSEEN 0]
received: * OK [PERMANENTFLAGS (\Deleted \Seen \Answered)]
received: * OK [UIDVALIDITY 1]
received: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft)
received: A00009 OK [READ-WRITE] SELECT Completed
Thread 43364bb0 >
CamelStore('imap://andre@imap.example.com:993/;command=ssh%20-C%20-l%20%25u%20%25h%20exec%20/usr/sbin/imapd;use_ssl=always'):get_folder('Unerwünscht',
0) = 0x881c004
< 43364bb0 >
sending : A00010 SELECT {12+}
Unerwünscht
received: A00010 NO Mailbox does not exist
CamelException.setv(0x86af9e0, 301, 'IMAP command failed: Mailbox does not exist')
Comment 6 Stanislav Brabec 2005-12-06 12:35:33 UTC
I guess it is not duplicate of bug 317794. I am using plain IMAP, not IMAP4. If
I try IMAP4, I see this bug too, and additionally I see also bug 272058.
Comment 7 parthasarathi susarla 2005-12-06 15:50:13 UTC
Its not specific to IMAP since, GW/Exchange also has a similar issue.

Bascaially, whenever a folder name contains a special character, Evolution does
not display it properly, or does not display it at all in some cases.
Comment 8 André Klapper 2005-12-13 13:49:14 UTC
"Unerwünscht" is displayed as "Unerwünscht" here, so it could be the old
unicode character conversion problem. running cvs head version from yesterday,
evolution crashes when i try to create an IMAP folder named "üüü".
Comment 9 Tor Lillqvist 2005-12-14 14:14:51 UTC
There seems to be some confusion in the imap provider about when mailbox names
are in MIME-UTF-7 and when in UTF-8. It seems that there is even double
conversion happening from MIME-UTF-7 to UTF-8, and then again from what already
is UTF-8 (as if it was MIME-UTF-7) to UTF-8... I think the cleanest way to
handle it would be to keep everything in UTF-8 and do any conversion to/from
MIME-UTF-8 as late/early as possible. The attached patch seems to fix the
problem in HEAD.
Comment 10 Tor Lillqvist 2005-12-14 14:16:43 UTC
Created attachment 55984 [details] [review]
Minor patch that seems to help

I verified that after this I could create folders with non-ASCII names, rename
such folders, and also that such folders created with another client show up
correct, and that folders created in Evo show up correct in the other client.
Comment 11 parthasarathi susarla 2005-12-14 14:46:33 UTC
I really belive that this is *not* a provider specific issue. This is either an
issue with camel or mailer
Comment 12 parthasarathi susarla 2005-12-14 15:15:58 UTC
Am sorry please ignore the Comment #11. Tors patch is quite right. 
And as Tor pointed outs we are infact doing a double converstion. Thanks Tor. I
think the patch should go in. Testing it right now. Should be in sooon.
Comment 13 parthasarathi susarla 2005-12-14 15:21:15 UTC
Tor the patch is fine. Its ok to commit. Will you commit it?? Should be
committed to both HEAD and stable branch. 
Comment 14 Tor Lillqvist 2005-12-14 20:49:16 UTC
Argh, I was writing garbage in my comment #9. When I said "MIME-UTF-7" what I
meant was "IMAP-UTF-7"... sorry. Acronym overload.

Anyway, yes, I can commit the fix to HEAD, but I don't have a checkout of the
stable branch, could someone else commit it there please.
Comment 15 Tor Lillqvist 2005-12-14 21:32:18 UTC
Patch committed to HEAD.

2005-12-14  Tor Lillqvist  <tml@novell.com>

	Fix #323106:
	
	* camel-imap-store.c (camel_imap_store_summary_full_to_path,
	camel_imap_store_summary_path_to_full): Remove extraneous
	conversions from IMAP-UTF-7 to UTF-8 and back.

	* camel-imap-command.c (imap_command_strdup_vprintf): Do the
	conversion from UTF-8 to IMAP-UTF-7 only just before sending a
	request. Do it also for %S formats, as that is what the CREATE
	command uses. But %S is used also for other commands (like LOGIN
	and LIST), so maybe we really would need a new format that would
	be used *only* for mailbox (folder) names, assuming it's only
	mailbox names that use the IMAP-UTF-7 encoding. Or is that what %F
	is intended for? But why then does CREATE use %S?
Comment 16 parthasarathi susarla 2005-12-15 05:10:00 UTC
patch committed to stable branch on behalf of Tor
Comment 17 André Klapper 2005-12-30 05:53:17 UTC
not fixed. now i can access my mails in that folder, but the folder name is still incorrectly displayed (two chars instead of one) - i'm running 2.5.3 rpm 2005-12-19 here.

also, i wonder myself what to tell all those people running 1.4.2.1?
"wait until 1.4.3 gets out in february"?

be aware of the fact that people running 1.4.2x are not able AT ALL to access mails in that particular folders for the next five weeks. ain't cool. :-/
Comment 18 André Klapper 2005-12-30 05:53:33 UTC
*** Bug 325240 has been marked as a duplicate of this bug. ***
Comment 19 Patryk Zawadzki 2005-12-30 12:26:17 UTC
Any chance for a .2 release any soon? Being forced to use SquirrelMail is quite annoying ;)
Comment 20 Patryk Zawadzki 2005-12-30 16:44:06 UTC
Applied the patch to PLD Linux repositories. As reported by Andre, folders are now accessible but still displayed incorrectly. Anyway, this fixes our users' problems for a while. Looking forward to 1.4.3.
Comment 21 parthasarathi susarla 2006-01-03 10:42:32 UTC
What are the folder names that are having problems?? Could you give a few names which i could try. Since the one "ü" and similar characters, render quite well and are accessible too.
Comment 22 André Klapper 2006-01-03 11:24:19 UTC
partha: they do not render quite well at all here, "Unerwünscht" is still displayed as "Unerwünscht" - it's also "Unerwünscht" when i use the "rename" context menu entry.
evo2.5.3 rpm 2005-12-19; suse9.3.
Comment 23 parthasarathi susarla 2006-01-03 17:03:13 UTC
Created attachment 56720 [details]
screen shot with the folder name

Andre:
So i created the folder name mentioned in the last comment, find the attached screen-shot. 
Hmm... Not sure what is wrong. And that too when the folder is being listed and can be renamed.
Comment 24 André Klapper 2006-01-03 18:01:00 UTC
partha, i tried to delete that folder locally, but after syncing with my IMAP server its name was again wrong.

if i try to rename that folder locally from "Unerwünscht" to "Unerwünscht", i get the error message: "A folder name "Unerwünscht" already exists".
Comment 25 parthasarathi susarla 2006-01-03 18:43:39 UTC
Andre, can you try removing the .ev-store-summary file in your $HOME/mail/imap/<mail-account-name>/ directorey. (Back it up first) ;). And see if you still have the porblem??
Comment 26 André Klapper 2006-01-04 09:54:13 UTC
have to say that my IMAP evolution-virtual-junk folder is also called "Unerwünscht" by evolution. (this worked pretty fine up to now.)

well, i removed both folders, "Unerwünscht" and "Unerwünscht" (i had both). the "Unerwünscht" folder gets recreated, it's called "Unerwünscht" both in the UI and as folder name on the harddisk.
Comment 27 parthasarathi susarla 2006-01-11 05:30:54 UTC
Adnre, can i have the camel traces
export CAMEL_VERBOSE_DEBU=1 and launch evo from the terminal (You already know all this :D) and just attach the soap traces here. Also could you send me your .evolution/mail/imap/<account>/.ev-store-summary file to me personally. I wanted to check if the encoding is fine. TIA
Comment 28 Jeffrey Stedfast 2006-01-11 15:51:34 UTC
as Tor noted in his comment with the patch, %S is used by other commands that are not meant to have their strings re-encoded into UTF-7, and so the patch is wrong.

I would say that the correct fix is to fix the code that passes folder names as %S to:

1. use %F if possible (tho maybe there's a reason they use %S instead of %F?)
or
2. do the charset conversion and THEN pass as %S in the calling function


ok... looking at the code, looks like the correct solution would be #2, and to make it easier, probably make a new formatter option %N (for Namespace? or maybe %P for "path"? since that's what %S's are used for in the code where a folder path is passed in). This new formatter would then do the UTF-7 conversion same as the %F currently does, but we wouldn't call the path_to_full() on it, since it wouldn't necessarily be in the summary.
Comment 29 Jeffrey Stedfast 2006-01-11 15:52:55 UTC
(the fact that the paths are not necessarily in the summary is why they are passed as %S's btw)
Comment 30 Ed Catmur 2006-01-11 16:18:36 UTC
-> bug 326385 for this issue
Comment 31 Shreyas Srinivasan 2006-01-12 09:22:18 UTC
*** Bug 326587 has been marked as a duplicate of this bug. ***
Comment 32 Shreyas Srinivasan 2006-01-18 10:44:17 UTC
Fixed: The actual issue is solved. The problem why the fix is not being seen is because of an other bug. To see this fix just remove .ev-store-summary in your 
.evolution/mail/imap/<account-name>/.ev-store-summary

and restart evolution. The folder should work. 
Comment 33 Mikael Nilsson 2006-01-18 12:47:10 UTC
Shreyas: what version fixed the problem?
Comment 34 André Klapper 2006-01-19 14:07:27 UTC
the fix was committed two days ago on 2006-01-17, see http://cvs.gnome.org/viewcvs/evolution-data-server/camel/providers/imap/camel-imap-store.c?rev=1.340&view=log

it will be included in evolution 2.4.3 (release date: 2006-02-08) and the unstable, experimental version 2.5.6.
Comment 35 Stanislav Brabec 2006-02-14 15:17:09 UTC
In evolution-data-server-1.5.90cvs20060206 it seems to work, but after update it reports following warnings:

(evolution:29186): camel-WARNING **: Trying to add folder 'INBOX/SuSE CR/balíky' to summary that already has it
Comment 36 Heikki Henriksen 2006-03-26 12:23:03 UTC
I'm still able to reproduce this with evo 2.4.2.1 & eds 1.4.2.1 with patched camel-imap-command.c (1.71.2.3), camel-imap-store-summary.c (1.16.2.1) & camel-imap.store.c (1.330.2.3).

Has anyone confirmed that this is actually fixed with versions < 2.5/1.5?
Comment 37 André Klapper 2006-04-25 13:21:29 UTC
heikki:
remove the folder in your harddisk cache at /.evolution/mail/imap/<account name>/folders/<folder name> and resync with the server, that worked for me
Comment 38 Heikki Henriksen 2006-04-25 13:42:44 UTC
Andre: 
thanks, I will bring that back to the corresponding debian-bug-reports :)