GNOME Bugzilla – Bug 739610
[IMAPx] Prevent running FETCH and STORE at the same time
Last modified: 2015-04-07 09:25:14 UTC
I am using Evolution 3.12.7. I often notice that I get the same email two times. Since I made sure this is not on the server side (and doesn't depend on the sender) I think this is a bug in evolution. More specifically it seems like the filter does not delete the mail it copied from inbox so it will filter the mail again. I estimate that ~20% of all mails get duplicated (exact same message) and ~1% get even tripled. This affects different email accounts (all are imap) from different mail servers. Additionally I sometimes get completely empty messages from nowhere with sources like this: === X-Evolution-Source: [AccountID]@[Hostname] Date: Sat, 25 Oct 2014 03:51:13 +0200 Subject: No Subject Message-ID: <[AccountID].camel@[Hostname]> Mime-Version: 1.0 From: === where [AccoundID]@[Hostname] is some evolution account I can find in ~/.cache/evolution/mail. It seems to me like both are results of an email filter aborted while running.
Thanks for a bug report. If the filter is cancelled before its execution is finished, then it can do anything; what exactly it'll do depends on the time when the filter was cancelled. I do not expect you cancelled the filter on purpose, did you? Also, how do your filters look like, please? I guess it's like: Find items which match: [ All the following conditions ] Conditions [some header] [contains] [ X ] Actions [ Move to ] [ the same IMAP account/Inbox/sub/folder ] [ Delete ]
Hi I don't cancel the filter on purpose. My network connection seems quite stable (cable, no WLAN) so I don't think it causes the problem either. In fact it does look like the filter was cancelled. In some cases I still see messages in my inbox which should have been filtered (others have successfully). Running the filter manually (Ctrl+Y) works fine in every case I tried so far. All my ~30 filters look like this: Find items which match: [ One of the following conditions ] Conditions [ To | from or to | from] [contains] (email address or hostname) 1…20 conditions each Actions [ Move to ] [ localmailboxfolder/sub/folder ] My filters only move messages, they never delete them. All messages are moved from (different) IMAP accounts to my local mailbox folder's subfolders. I only filter by to/from/CC/subject headers, mostly from/to. All filters are "match one of the following conditions".
Thanks for the update. I found out the race in IMAPx flag save just today (the move is "copy & delete", where delete is "set flag 'deleted' on the message"), which can cause similar outcome as you see. I wouldn't be surprised if you face just that. It's an IMAPx race, if the folder is updating based on the server's content and during this update is issued a STORE command, then the server can still return "outdated" flags for the just-saved-message-flag-changes, which the IMAPx doesn't recognize (the server flags has a precedence from the locally stored) and it reverts the flag changes to those on the server before the STORE command was initiated.
I'm using this bug report as a reference for the race between FETCH and STORE IMAP commands as described in the previous comment. I'm pretty sure it's related to your issue, if not just it. I chose to add a lock to a mailbox for its simplicity, because chasing for already running jobs between multiple opened connections is much more complicated code than this. Created commit c61babe in eds master (3.13.8+) [1] Created commit 8558003 in eds evolution-data-server-3-12 (3.12.9+) [1] https://git.gnome.org/browse/evolution-data-server/commit/?id=c61babe
I still keep running into this bug every day with evolution 3.12.10 and eds 3.12.10. Many emails from different accounts get duplicated by evolution. Should I open a new bug report? Is there any information I can provide?
Hmm, that's weird. Could you disable multiple connections for that account, just to check whether it has anything with it, please? It's to set only 1 concurrent connection in the accounts properties. Better to restart evolution, to make sure it'll pick the change right.
I did that (only one connection per account) and it still happens after restarting evolution (and the whole OS).
So this problem is still happening and neither RESOLVED nor FIXED.
Maybe it's something else. It'll be good to figure that out. I suppose you have 3.12.10 of evolution-data-server and evolution installed. Please disable any other IMAP accounts you may have configured and keep enabled only the one where you can reproduce the issue. Then run evolution with IMAPx debugging on: $ CAMEL_DEBUG=imapx:io,imapx:conman evolution &>log.txt which will log connection changes and all the communication between the Evolution and your server. It may show whether the FETCH and STORE were run at the same time or not. As it logs everything, it'll contain message contents and folder listing and folder contents and so on, thus it's not suitable for a public sharing, unless sanitized. Feel free to sent it only to me, with a bug reference in subject (thus I'd not miss it in my spam folder), and mention in which folder the issue happened and how exactly it exhibited this time.
Thanks for the log you sent to me privately. I see there one message download, which is downloaded in two parts. First only headers are downloaded, then the message body. That is correct and expected. I do not see any other command in the log being involved during this time, thus it can be pretty much different issue than the original. I see that you've set only one connection for the account (which is perfectly fine), and you have most likely only one account enabled when you were capturing the log. The log shows that right after the message is downloaded an IDLE command is started, six times. It really shouldn't do this, it should run one IDLE per connection, not six same IDLE-s in one connection. I believe that's the reason, or at least a part of it, which also breaks your message delivery and duplicates newly received messages.
Yes, I am running evolution 3.12.11 and evolution-data-server 3.12.11. I didn't disable the other connections, I have 6 IMAP connections enabled. The bug happens on all of them so I didn't know which one to enable/disable. This might be the reason why you see six IDLE commands.
Is it "six connections" or "six IMAP accounts"? Nonetheless, in case this is related to the connections and the way they are used, then what you see can be addressed by bug #745545.
Six IMAP accounts. Preferences set to one connection. This is done for each IMAP account.