After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 400841 - Evolution 2.9.6 IMAP Headers broken with default settings
Evolution 2.9.6 IMAP Headers broken with default settings
Status: RESOLVED FIXED
Product: evolution
Classification: Applications
Component: Mailer
2.10.x (obsolete)
Other Linux
: High major
: ---
Assigned To: Sankar P
Evolution QA team
: 400929 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2007-01-25 23:40 UTC by Pascal Terjan
Modified: 2013-09-13 00:52 UTC
See Also:
GNOME target: ---
GNOME version: 2.17/2.18


Attachments
Drop extra spaces when using default or mailing list headers (1.20 KB, patch)
2007-01-26 10:53 UTC, Pascal Terjan
reviewed Details | Review
Move the space from the end to the beginning when using custom headers (450 bytes, patch)
2007-01-26 10:54 UTC, Pascal Terjan
reviewed Details | Review
Fix (EDS) (1.18 KB, patch)
2007-01-29 14:58 UTC, Sankar P
committed Details | Review
Fix (Evo) (1.07 KB, patch)
2007-01-29 14:58 UTC, Sankar P
committed Details | Review

Description Pascal Terjan 2007-01-25 23:40:54 UTC
Since I updated to 2.9.6 evolution did not see new mails anymore on any of my IMAP servers.

The issue was :

A00007 UID FETCH 19540:* (FLAGS RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM TO CC SUBJECT REFERENCES IN-REPLY-TO MESSAGE-ID MIME-VERSION CONTENT-TYPE X-MAILING-LIST X-LOOP LIST-ID LIST-POST MAILING-LIST ORIGINATOR X-LIST SENDER RETURN-PATH X-BEENTHERE  )])
A00007 BAD Invalid field-name in UID Fetch BODY.PEEK[HEADER.FIELDS

I found that there is now a plugin to select headers to fetch. I switched to 'all' and I get my mail again.
Comment 1 Pascal Terjan 2007-01-26 00:05:19 UTC
Hmm it looks like
http://mail.gnome.org/archives/tinymail-devel-list/2006-November/msg00040.html
Comment 2 Pascal Terjan 2007-01-26 00:08:53 UTC
I am using evolution 2.9.6 and e-d-s 1.9.6.1
Comment 3 Ruben Vermeersch 2007-01-26 09:08:22 UTC
This should probably be in evolution, as mail still happens in the client (not in eds, AFAIK)
Comment 4 Sebastien Bacher 2007-01-26 09:25:00 UTC
Ubuntu bug about that: https://launchpad.net/ubuntu/+source/evolution/+bug/81524
Comment 5 Pascal Terjan 2007-01-26 09:55:21 UTC
I think the faulty code is in evolution-data-server/camel/providers/imap/camel-imap-folder.c (in imap_update_summary) :

#define CAMEL_MESSAGE_INFO_HEADERS "DATE FROM TO CC SUBJECT REFERENCES IN-REPLY-TO MESSAGE-ID MIME-VERSION CONTENT-TYPE "
#define MAILING_LIST_HEADERS "X-MAILING-LIST X-LOOP LIST-ID LIST-POST MAILING-LIST ORIGINATOR X-LIST SENDER RETURN-PATH X-BEENTHERE "

                if (store->headers == IMAP_FETCH_ALL_HEADERS)
                        header_spec = g_string_new ("HEADER");
                else {
                        header_spec = g_string_new ("HEADER.FIELDS (");
                        header_spec = g_string_append (header_spec, CAMEL_MESSAGE_INFO_HEADERS);
                        if (store->headers == IMAP_FETCH_MAILING_LIST_HEADERS)
                                header_spec = g_string_append (header_spec, MAILING_LIST_HEADERS);
                        if (store->custom_headers)
                                header_spec = g_string_append (header_spec, store->custom_headers);

                        header_spec = g_string_append (header_spec, " )");
                }

For custom_headers the code (from evolution-2.9.6/plugins/imap-features/imap-headers.c) is :
do
{
  gtk_tree_model_get (model, &iter, 0, &header, -1);
  str = g_string_append (str, g_strstrip(header));
  str = g_string_append (str, " ");
} while (gtk_tree_model_iter_next(model, &iter));

As the 3 strings already contain a space at the end, I would just replace " )" with ")".
Comment 6 Pascal Terjan 2007-01-26 10:39:31 UTC
Actually dropping this space will left one space at the end, and there should be none according to my tests, I'm writing a better patch.
Comment 7 Pascal Terjan 2007-01-26 10:53:44 UTC
Created attachment 81262 [details] [review]
Drop extra spaces when using default or mailing list headers
Comment 8 Pascal Terjan 2007-01-26 10:54:34 UTC
Created attachment 81263 [details] [review]
Move the space from the end to the beginning when using custom headers
Comment 9 Pascal Terjan 2007-01-26 10:57:00 UTC
The 2 attached patches move the spaces to the beginning of appended strings to avoid having them when the string is not appended.
One applies to evolution-data-server for basic and mailing list headers, the other one is for evolution and does the same for custom headers.
Comment 10 Brian J. Murrell 2007-01-26 16:09:02 UTC
(In reply to comment #0)
> 
> I found that there is now a plugin to select headers to fetch. I switched to
> 'all' and I get my mail again.

Which plugin is this?  How did you select it?  I opened the Plugin Manager but don't see anything that looks related.

Using 2.9.6-0ubuntu1 here.

Cheers.
Comment 11 Pascal Terjan 2007-01-26 16:38:13 UTC
The plugin is called "IMAP Features" and adds a tab called "IMAP Headers" to the preference window of the accounts which are using IMAP.
Comment 12 Brian J. Murrell 2007-01-26 16:52:58 UTC
(In reply to comment #11)
> The plugin is called "IMAP Features" and adds a tab called "IMAP Headers" to
> the preference window of the accounts which are using IMAP.

Indeed!!

Yay.  Now I can see new mail.

Thanks Pascal.
Comment 13 Jeff Cai 2007-01-27 01:07:20 UTC
*** Bug 400929 has been marked as a duplicate of this bug. ***
Comment 14 Sankar P 2007-01-29 14:53:14 UTC
Against which IMAP server are all these tested? I tested against a GroupWise IMAP server and things seems to be working fine.

Based on the above comments, I understand that having multiple spaces is the source of the problem. 

Pascal: Thanks for your patches. However,

http://bugzilla.gnome.org/attachment.cgi?id=81262&action=view will not work if an user chooses to download basic + some custom headers.

And http://bugzilla.gnome.org/attachment.cgi?id=81263&action=view will not work since it merges all the custom headers without a space seperating them.

Patches with the right approach to follow.
Comment 15 Sankar P 2007-01-29 14:58:19 UTC
Created attachment 81437 [details] [review]
Fix (EDS)
Comment 16 Sankar P 2007-01-29 14:58:38 UTC
Created attachment 81438 [details] [review]
Fix (Evo)
Comment 17 Pascal Terjan 2007-01-29 14:59:19 UTC
(In reply to comment #14)
> Against which IMAP server are all these tested? I tested against a GroupWise
> IMAP server and things seems to be working fine.

On 2 different servers (one is cyrus the other one I don't know)

> Based on the above comments, I understand that having multiple spaces is the
> source of the problem. 
> 
> Pascal: Thanks for your patches. However,
> 
> http://bugzilla.gnome.org/attachment.cgi?id=81262&action=view will not work if
> an user chooses to download basic + some custom headers.

Why ?
The custom header will come with their spaces.

> And http://bugzilla.gnome.org/attachment.cgi?id=81263&action=view will not work
> since it merges all the custom headers without a space seperating them.

Hmm no, it just adds the space before each header instead of adding it after
Comment 18 Sankar P 2007-01-29 15:01:19 UTC
Can some one test these patches ? 

My IMAP Server returns proper headers even if there are multiple sucessive spaces are at the end of the HEADERS.

Comment 19 Sankar P 2007-01-29 15:09:08 UTC
Pascal: Sorry for the noise. I misread the evolution patch as a result, I mistankingly found that the EDS patch may also not work. (inter-dependant).

Your patch also addresses the scenario and looks good. 

My patch (evo.patch) fixes a memory leak in addition to the above problem. 
Comment 20 Sankar P 2007-01-29 15:19:37 UTC
One scenario where the problem could still exist even after the above patches are committed will be :

When the user enters clicks Add Headers and starts typing a lot of headers one after another seperated by multiple spaces. I need to look at ways to disable any keys other than alphabets and numbers on a GtkEntry. We should disable not just spaces, but any markup tags or scripts or any character that cannot be part of a Header name.

Comment 21 Pascal Terjan 2007-01-29 15:20:49 UTC
Yes you are right for the memory leak :)

And yes that would be nice to ensure that headers are correct 
Comment 22 Sankar P 2007-01-29 17:27:33 UTC
Pascal: There is one very little issue with your patch. It may not give the desired result if someone has altered the header-preferences using the old version (wihtout any of our patches, a fresh 2.9.6) 

Users may be forced to delete all headers and re-add them if they want to use it. 

Can someone test with the above patches (on a server that doesnt accept multiple spaces in request) and update the bug so that I can commit it ?
Comment 23 Claudio Saavedra 2007-01-30 00:26:36 UTC
I had the same problem with my email account in an IMAP Cyrus server. I tested Sankar' patches and now I can fetch new emails without any problem.

Comment 24 Sankar P 2007-01-30 12:22:19 UTC
I have committed my patches to the HEAD. I have added a g_print statement to print the HEADER string that will be sent to the server. I will keep this bug open until I remove the g_print statement that I have put in.  


Pascal: Thanks for your patches. I chose my patch as yours might break the behavior if some users have used the released 2.9.6 and configured some headers. Will love to some more patches from you :)
Comment 25 Nathaniel McCallum 2007-02-03 16:51:24 UTC
I'm still having a similar problem. CAMEL_VERBOSE_DEBUG gives me this:
sending : A00033 SELECT INBOX
received: * 1353 EXISTS
received: * 0 RECENT
received: * OK [UIDVALIDITY 1157468607] UID validity status
received: * OK [UIDNEXT 5098] Predicted next UID
received: * FLAGS (Junk $Label1 $Label2 $Label3 $Label4 $Label5 $Forwarded NonJunk $MDNSent \Answered \Flagged \Deleted \Draft \Seen)
received: * OK [PERMANENTFLAGS (Junk $Label1 $Label2 $Label3 $Label4 $Label5 $Forwarded NonJunk $MDNSent \* \Answered \Flagged \Deleted \Draft \Seen)] Permanent flags
received: * OK [UNSEEN 15] first unseen message in /var/spool/mail/num
received: A00033 OK [READ-WRITE] SELECT completed
sending : A00034 UID FETCH 1:* (FLAGS RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM TO CC SUBJECT REFERENCES IN-REPLY-TO MESSAGE-ID MIME-VERSION CONTENT-TYPE X-MAILING-LIST X-LOOP LIST-ID LIST-POST MAILING-LIST ORIGINATOR X-LIST SENDER RETURN-PATH X-BEENTHERE )])
received: A00034 BAD Syntax error in header fields

I don't know what the mail server is, but it does work in every other mail client.  Enabling all headers doesn't fix the problem, but this worked before I upgraded to 2.9.x.
Comment 26 Sankar P 2007-02-05 05:15:24 UTC
From Preferneces-> Account-editor, Set header-prefernces to "Full Headers" and restart Evolution. It will work. If it doesnt work, then paste the gconf-value, for /apps/evolution/mail/accounts.
Comment 27 Nathaniel McCallum 2007-02-05 14:26:17 UTC
I came into the office today, and it just worked, no errors at all.  I have no idea why.  Sorry for the spam.

(BTW, the above error messages occurred with "Full Headers" selected.)
Comment 28 Nathaniel McCallum 2007-02-06 14:59:28 UTC
OK, I was wrong.  The error messages *don't* occur with "Full Headers" selected, but do occur when "Basic + Mailing List" is selected.  Regardless, this breaks evolution out of the box, but with *no* warning to the user.  They think everything is working, but they just get no new mail.
Comment 29 Claudio Saavedra 2007-03-12 20:57:36 UTC
By the way... the beep in the printf(In reply to comment #24)
> I have committed my patches to the HEAD. I have added a g_print statement to
> print the HEADER string that will be sent to the server. I will keep this bug
> open until I remove the g_print statement that I have put in.  

The printf and the "\a" on it are still present. Will they be in the big .0 release? I don't think it's a good idea..
Comment 30 Harish Krishnaswamy 2007-03-14 07:11:14 UTC
\a needs to be removed and the entire statement should preferrably be executed iff the DEBUG flags are on.
Comment 31 Karsten Bräckelmann 2007-03-14 18:10:48 UTC
The beeps appear to have been fixed in SVN trunk already.
http://svn.gnome.org/viewcvs/evolution-data-server/trunk/camel/providers/imap/camel-imap-folder.c?r1=7628&r2=7654
Comment 32 Sankar P 2007-03-15 05:57:13 UTC
The patch is committed after the 2.10 tarballs are out. Fix will come in the next release.