Funnily enough, there are no messages with such numbers in my IMAP folder.

Do you have reason to believe these warnings concern actual messages you should have received (have messages gone missing)? Does this block receiving all or some messages (other than the messages that seem to be involved in this warning)? Other actions that apparently don't work?

Since this message arrived the first time, Evolution does not show newer mails (which exist, Thunderbird shows them), so it blocks them..

Looking into other folder works fine, also, Evolution does not crash.

I see.

Maybe you could try to generate some debug info. See: http://www.gnome.org/projects/evolution/bugs.shtml That might help the people that actually understand IMAP to see what is going on.

I'd guess closing evolution, "evolution --force-shutdown" and restarting evolution with "env CAMEL_DEBUG=imap" in a terminal might do the trick.

Do scan for personal info before posting.

Created attachment 117925 [details]
evolog debugging imap

Thanks for your help, this is a short cut of the things that could help fixing the problem..

Also I found this:

received: A00010 OK FETCH completed.

sending : A00011 UID FETCH 70071:* (FLAGS RFC822.SIZE INTERNALDATE BODY.PEEK[HEADER.FIELDS (DATE FROM TO CC SUBJECT REFERENCES IN-REPLY-TO MESSAGE-ID MIME-VERSION CONTENT-TYPE X-MAILING-LIST X-LOOP LIST-ID LIST-POST MAILING-LIST ORIGINATOR X-LIST SENDER RETURN-PATH X-BEENTHERE)])

received: * 12359 FETCH (UID 70071 FLAGS ()   )


(evolution:3780): camel-imap-provider-WARNING **: Unexpected FETCH response from server: (  )
received: * NO Cannot open message 12359

I now solved my problem (The IMAP server is a Courier server):

I copied a mail which was stored in someone's other mailbox to my mailbox. Because of this I was not mentioned in that mail.

Thunderbird seems to just have ignored that mail, at least the mail did not appear in it. 

Evolution in contrast did not ignore thatmail and just stopped working.

Here's some information on how I found which mail caused the issue: First I deleted the file courierimapuiddb in the mailbox. Then I started Evolution, looked at the number which is displayed besides of the error and searched for it in the file courierimapuiddb. It contains the filename of the mail in the cur folder so you know what mail you have to delete.

I can try to reproduce the problem with a seperate email account so if an Evolution developer is interested in getting the account data, just ask.

this is a courier imap bug, imap servers are not allowed to tell a client that a message exists if the server cannot open the message.

I've discussed this with the author of the imap specification and he has told me that courier imap is broken in this way and in a lot of other ways.

please complain loudly to the courier imap authors until they fix this long-lasting bug that has been going on for over 8 years.

Hi Jeff, and could we cope with this "properly" in Evolution, say at least as good as TB, thus not scream into the darkness and give up on the first problem, but keep going as long as we are able to?

not easily, no, because Evolution depends on the summary being accurate to properly handle EXPUNGE notifications, etc. If the summary from the server is incorrect, it becomes difficult to properly track these things.

The problem is that all you have for that message is the sequence id, but evolution tracks messages based on uid. since you can't get the uid, you're screwed.

Created attachment 117944 [details] [review]
proposed eds patch

for evolution-data-server;

I know what you mean, Jeff, the error will be shown "forever" now on, but there is no need to stop showing properly downloaded messages from the server in the UI only because something was wrong with some preceding message. It should be better to show user as much as we can and then report the error. Which is, in my opinion, better way than with TB, which just stay calm. I know there is no way to user to fix in from evo, but it's better than actual behavior.

If you look into the patch, then I hope you'll agree that the fix in this part should not hurt. Do you think it should?

If a user has this problem with your patch then his summary info will be out of sync with what the server's summary and so message indexes will not match up.

You can't ignore these bad responses from the server because these messages have uids on the server, it's just that the server won't tell us what the uids are.

the best you can do in this situation is to ignore all messages with a seq-id >= to the seq-id provided in the bad response (remember that the server need not respond in sequential order).

Fejj, Are there any issues that you foresee due to the mismatch of sequences?

As far as I can tell, I believe this could not hurt. I do not know what exactly Jeff means with the sequence numbers, but in the place where I made the change has this thing no meaning, there are only message infos and uids.

If Jeff means problems with ranges when calling get_matching function, then this one has been fixed recently, after db-summary, to behave correctly in bug #545103.

Fejj, what are your thoughts? Are there any issues  you forsee ?

Fejj, we don't add that to the system and we just handle it before it. But I get your point.

Milan, imap_rescan would actually delete rest of the message/summary on the db, beyond this uid/sequence.

Consider...

Server CLIENT
1       1
2       2
3[P]    4
4       5
5       6

'3' is the problematic thing that we have on the server. We compare always cache against server. Since 4 != 3, we dete 4, 5, 6.. etc trying to find '3 there. So, it might mean that we redownload all the summary for him. We problably look at a more optimized solution, or rewrite imap_rescan to deal such things.

(In reply to comment #16)
> ...snip...

I'm sorry, I didn't get it.

When I look into 'add_message_from_data' then the 'messages' array is always resized based on the sequence number, which means it can do NULL gaps, which is probably our case. In your example above, it tried to create with message 4 on the server (with sequence number 4, I guess) array of size 4, but before it had only size 2.

When looking around, I think near line 3391 in camel-imap-folder.c (in my revision 9821), if messages->pdata[seq - first] is NULL, then at the end is created a clone of some previous message info. We should assign this pmi to the messages->pdata[seq - first] probably, to have it accessible later, in the function which I touched in the patch.

But, it'll show definitely wrong mail in there, thus it's probably irrelevant.

Summarized, I do not think the patch is so wrong, neither I see any change in imap_rescan which would help here. I'm missing here something, obviously.

srag/jeff ping?

Milan, Ill try to recollect. 

THe point is not this iteration. In next start, we would endup in a different cache line up wrt uids. Which would end up with the uids to be deleted on the cache. 

Looking into place of the change in the above patch, I guess your presumption of client side is incorrect. The client has stored '3' in its cache, that's the reason why the request for a detailed information fails. Doing neither 'break' nor 'continue' removes anything from the cache in a place of the change.

I still feel like missing something here, though.

I'm not sure if I can ping srag again. This bug makes me feel silly.

bug 575923 is dupe ?

(In reply to comment #22)
> bug 575923 is dupe ?

Sounds different from me.

Hi Martin, 
if you have time, could you please check again whether this issue still happens in Evolution 3.4.2 or 3.2.3 and update this report by adding a comment and changing the "Version" field and provide information about your distribution? Thanks a lot.

Hi Martin, 
if you have time, could you please check again whether this issue still happens in Evolution 3.8.5 or 3.6.4 and update this report by adding a comment and changing the "Version" field and provide information about your distribution? Thanks a lot.

No reply by reporter & IMAP backend has been rewritten in the meantime. Please reopen if this is still a problem in 3.10 or newer.