GNOME Bugzilla – Bug 489236
Evolution Inbox not displaying messages which have already been opened in another application using exchange - regression
Last modified: 2009-07-09 23:25:32 UTC
Please describe the problem: I have two computers. One runs Gutsy, the other XP. I use MS Exchange 2003 exclusively for email on both systems. If I check (i.e. open and read) an email on my XP computer, (or using OWA on the Gutsy computer) that email will not appear in my Evolution Inbox unless I reply to it or forward it. If I get the email while sitting at my XP system and do not reply to it or forward it, I will never be able to even see it in my Evolution Inbox. The only way to make the message appear in the inbox is to go to the other computer and forward it or reply to it. This is a new problem with Evolution 2.12. This problem never occured before. This is a critical problem because it renders Evolution essentially useless. I wanted to file this bug under "Evolution Exchange" but it does not list "2.12" there. Steps to reproduce: 1. YOU MUST BE USING EXCHANGE 2. Send yourself an email from gmail or something. 2. Open that email by clicking on it on OWA in Firefox or some other program. Do not reply or fwd it. 3. Open Evolution and look for that email in the inbox. Actual results: The email does not appear in Evolution if it has first been opened in OWA. Expected results: Normally, the email should appear in the evolution's inbox AND the OWA inbox. Does this happen every time? YES Other information: This is an exchange bug, and should be addressed by people using exchange 2003
Do you see this happening with 2.22 as well. Iirc, a lot of missing mail bugs were fixed during 2.22. Moving to the right product evolution-exchange.
Created attachment 114109 [details] [review] proposed eex patch for evolution-exchange; I'm not sure whether this belongs to this bug or not, but it sounds very similar, thus I will use this report. I had something similar, when opening Inbox folder, I didn't see all messages as in owa. I do not know what caused it exactly, but the change in refresh_folder_internal to not include check for high_article_num in the search for messages helped. Unfortunately, I get lost the old summary, so I do not know what number was stored there and why. Anyway, I didn't change anything in this function, I rather stepped a few functions above, to force full refresh when it seems like good idea.
Seems fine for me. Bharath what do you say?
This problem was dormant on my system for a long long time, until yesterday when things started going haywire again. The problem began when i rebooted an old computer using Microsoft outlook which I had not used for a very long time. As soon as I pressed send receive on that computer's outlook, all emails older than July 3 2008 were immediately gone from my evolution inbox. It could be just coincidence that these things happened at exactly the same time, but I think it is not. In the past, I was able to fix this problem of missing emails by doing: cd ~/.evolution evolution --force-shutdown rm -rf `find . -name *exch* -print` When I do this now, the missing emails show up while the inbox is resyncing, but once it finishes resyncing, they disappear, or they disappear if I press "send receive". From my research online, I'm pretty certain that this bug is related to the following bugs and is probably the same problem, so it could be a duplicate of these bugs: http://bugzilla.gnome.org/show_bug.cgi?id=435371 http://www.mail-archive.com/evolution-list@gnome.org/msg07847.html http://www.schnuckelig.eu/blog/evolution-exchange-connector-missing-mails-inbox I am failry new to linux and don't really know how to apply the proposed patch, so if anyone wanted to help me I'd totally love that, but more generally I'd love to see the patch applied somehow by someone so the next time I get an evolution update it is included. Thank you all so so much! Dooma
(In reply to comment #2) > Created an attachment (id=114109) [edit] > proposed eex patch > > for evolution-exchange; > > I'm not sure whether this belongs to this bug or not, but it sounds very > similar, thus I will use this report. I had something similar, when opening > Inbox folder, I didn't see all messages as in owa. I do not know what caused it > exactly, but the change in refresh_folder_internal to not include check for > high_article_num in the search for messages helped. Unfortunately, I get lost > the old summary, so I do not know what number was stored there and why. > Anyway, I didn't change anything in this function, I rather stepped a few > functions above, to force full refresh when it seems like good idea. > Missing mails is normally an issue w.r.t calculation of article-number. However, recalculating from the start is not a solution. In get_folder_contents_online, we fetch all the mails from server and compare/manipulate them against the ones that we have in cache and appropriately update our cache with the changes from the server. I do agree that something is triggering a mis-calculation of article number and refetching all the mails again will cost performance of the connector. In our labs, we have tried all possible scenarios that we know and are not able to reproduce the behavior as mentioned in various reports except for marking them as junk (messages were getting wrongly marked as junk by the spamassasin plugin). An alternative approach can be tried here: i) Store PR_CREATION_DATE and PR_MODIFICATION_TIME and use them as conditions to fetch records from the server in get_folder_contents_online ii) Iterate over these set of records and find the high_article_number and compare it with that of cached value. Whichever is smaller, use it as the marker for refreshing the folder for new messages. I am not able to recall as to what was the issue in using CREATION and MODIFICATION time, for now. I will update the bug once I get some data in hand. Also, change the way "seq" is used to keep track of deleted messages and use "uid" instead. (Marking patch to "needs-work").
varadhan, ping. How is it going with your patch?
Created attachment 115897 [details] [review] proposed eex patch ][ for evolution-exchange; I fixed totally 3 things in get_folder_contents_online: a) do not skip any message for decreasing b) call rm_idx_uid on all the left items c) if the high_article_num should be decreased, then it will be There is only one thing I'm not sure about, if there will be called 'refresh_folder_internal' before 'get_folder_contents_online', and eex will be forced to decrease, then it should ideally refresh folder again, any idea how or if really necessary? (it will be refreshed next time user selects the folder, but that can be "too late".) One note: this patch has been done on gnome-2-22 branch, because db-summary has its own "features" how to hide eex messages from user's view at the moment.
Tried to respond to email request ( answer is at bottom of text ).... This is the mail system at host relay.mail.twtelecom.net. I'm sorry to have to inform you that your message could not be delivered to one or more recipients. It's attached below. For further assistance, please send mail to postmaster. If you do so, please include this problem report. You can delete your own text from the attached returned message. The mail system <bugzilla-daemon@bugzilla.gnome.org>: host mail.gnome.org[209.132.176.177] said: 554 5.7.1 <bugzilla-daemon@bugzilla.gnome.org>: Recipient address rejected: Your dictionary attacks have tired us, buzz off (in reply to RCPT TO command) Reporting-MTA: dns; relay.mail.twtelecom.net X-Postfix-Queue-ID: C896B7981DE X-Postfix-Sender: rfc822; Reid.Thompson@ateb.com Arrival-Date: Wed, 6 Aug 2008 06:19:46 -0600 (MDT) Final-Recipient: rfc822; bugzilla-daemon@bugzilla.gnome.org Original-Recipient: rfc822;bugzilla-daemon@bugzilla.gnome.org Action: failed Status: 5.7.1 Remote-MTA: dns; mail.gnome.org Diagnostic-Code: smtp; 554 5.7.1 <bugzilla-daemon@bugzilla.gnome.org>: Recipient address rejected: Your dictionary attacks have tired us, buzz off Subject: Re: [Bug 478151] I lose the ability to see older messages in my Exchange inbox From: Reid Thompson <Reid.Thompson@ateb.com> Date: Wed, 06 Aug 2008 08:19:43 -0400 To: "Evolution Exchange (bugzilla.gnome.org)" <bugzilla-daemon@bugzilla.gnome.org> Evolution Exchange (bugzilla.gnome.org) wrote: > If you have any questions why you received this email, please see the text at > the end of this email. Replies to this email are NOT read, please see the text > at the end of this email. You can add comments to this bug at: > http://bugzilla.gnome.org/show_bug.cgi?id=478151 > > Evolution Exchange | Connector | Ver: 2.10.x > > > > > ------- Comment #20 from Bharath Acharya 2008-08-06 08:48 UTC ------- > @Paul,Reid Can anyone try the patch at > http://bugzilla.gnome.org/show_bug.cgi?id=489236#c7. Milan has fixed another > scenario in which the calculation of article numbers were being messed up. > Would be good to see if that fixes this or if we still have more issues. > > > applying patch to svn head and rebuilding now.
The issue is still present on my system. Path: . URL: http://svn.gnome.org/svn/glib/trunk Repository Root: http://svn.gnome.org/svn/glib Repository UUID: 5bbd4a9e-d125-0410-bf1d-f987e7eefc80 Revision: 7315 Node Kind: directory Schedule: normal Last Changed Author: djihed Last Changed Rev: 7315 Last Changed Date: 2008-08-06 04:30:32 -0400 (Wed, 06 Aug 2008) Path: . URL: http://svn.gnome.org/svn/libsoup/trunk Repository Root: http://svn.gnome.org/svn/libsoup Repository UUID: 03f96f1e-eb25-0410-be34-e7bec31b2f82 Revision: 1150 Node Kind: directory Schedule: normal Last Changed Author: danw Last Changed Rev: 1149 Last Changed Date: 2008-08-04 09:42:30 -0400 (Mon, 04 Aug 2008) Path: . URL: http://svn.gnome.org/svn/evolution-data-server/trunk Repository Root: http://svn.gnome.org/svn/evolution-data-server Repository UUID: d0434b6f-c725-0410-8785-fd8a057797ef Revision: 9281 Node Kind: directory Schedule: normal Last Changed Author: sragavan Last Changed Rev: 9281 Last Changed Date: 2008-08-06 07:34:44 -0400 (Wed, 06 Aug 2008) Path: . URL: http://svn.gnome.org/svn/gtkhtml/trunk Repository Root: http://svn.gnome.org/svn/gtkhtml Repository UUID: 6715bd1f-e025-0410-99cf-e594b4b1144d Revision: 8910 Node Kind: directory Schedule: normal Last Changed Author: djihed Last Changed Rev: 8910 Last Changed Date: 2008-08-06 04:31:51 -0400 (Wed, 06 Aug 2008) Path: . URL: http://svn.gnome.org/svn/evolution-webcal/trunk Repository Root: http://svn.gnome.org/svn/evolution-webcal Repository UUID: 30acfedd-e925-0410-bc1e-a2789d043b2b Revision: 437 Node Kind: directory Schedule: normal Last Changed Author: djihed Last Changed Rev: 437 Last Changed Date: 2008-08-06 04:30:05 -0400 (Wed, 06 Aug 2008) Path: . URL: http://svn.gnome.org/svn/evolution-exchange/trunk Repository Root: http://svn.gnome.org/svn/evolution-exchange Repository UUID: cbd800bf-df25-0410-b17b-96628719a9b0 Revision: 1730 Node Kind: directory Schedule: normal Last Changed Author: djihed Last Changed Rev: 1730 Last Changed Date: 2008-08-06 04:30:03 -0400 (Wed, 06 Aug 2008) Path: . URL: http://svn.gnome.org/svn/evolution/trunk Repository Root: http://svn.gnome.org/svn/evolution Repository UUID: 9792ae6b-c725-0410-a7bc-87ac6a6ec1ac Revision: 35915 Node Kind: directory Schedule: normal Last Changed Author: mmonreal Last Changed Rev: 35915 Last Changed Date: 2008-08-06 05:32:08 -0400 (Wed, 06 Aug 2008)
sorry -- i should have been more specific -- at this point, i've not seen any emails 'disappearing', but the folder counts for emails are still incorrect.
you scared me a bit :) the folder counts, can it be a part of db-summary? we have such bugs against it, even on imap. Is it a total-count or unread-count broken. In the tree and/or in the info-label (left top, just above the tree) Any dots ("...") there? (I know, this is foolish question, I'm sorry)
very sorry for the scare... Both numbers in the info-label would get into an invalid state. i.e. i've seen both with a negative number displayed. I'll have to pay more attention to the tree folder count to advise whether it becomes invalid or not.
the tree folder count can become invalid also. Just had a folder that was empty receive some new msgs. tree count remained at nothing displayed, i deleted the messages and info label counters went negative, tree folder count remained at nothing displayed
hmmm -- tree folder count has now 'updated' itself to list correct number of unread messages. info label counters remain invalidly negative, and decrement even more negatively as emails are deleted
Patch looks free of regressions. But I can't see this happening for me to test. I see positive responses at comment #10 I would prefer a review from our exchange expert on this one :) Varadhan?
(In reply to comment #7) > Created an attachment (id=115897) [edit] > proposed eex patch ][ > > for evolution-exchange; > > I fixed totally 3 things in get_folder_contents_online: > a) do not skip any message for decreasing > b) call rm_idx_uid on all the left items > c) if the high_article_num should be decreased, then it will be > > There is only one thing I'm not sure about, if there will be called > 'refresh_folder_internal' before 'get_folder_contents_online', and eex will be > forced to decrease, then it should ideally refresh folder again, any idea how > or if really necessary? (it will be refreshed next time user selects the > folder, but that can be "too late".) > > One note: this patch has been done on gnome-2-22 branch, because db-summary has > its own "features" how to hide eex messages from user's view at the moment. > Good work, Milan. However, my patch is quite simple. ;-) Your b) and c) fix is quite good. It unconditionally assigns the "re-calculated" high_article_num to mfld->high_article_num because "high_article_num" is the fresh and updated one. get_folder_contents_online is the one that gets called when get_folder is called for the first time. Also, if summary exists for a folder, refresh_folder_internal will not be called unless user refreshes the folder or hit Send/Recv.
Created attachment 116567 [details] [review] proposed eex patch ]I[ for evolution-exchange; as requested.
(In reply to comment #17) > Created an attachment (id=116567) [edit] > proposed eex patch ]I[ > > for evolution-exchange; > > as requested. > Patch is ready to rock the trunk. Good work, Milan.
Committed to trunk. Committed revision 1744. Thanks, I'm looking forward to see your improvements you mentioned on IRC.
built from svn head evolution --force-shutdown rm -rf .evolution/exchange/ .evolution/mail/exchange/ restart evo began rebuilding summaries, etc, etc evo finished rebuilding summaries etc etc Comments in two sections below LOCAL INBOX and EXCHANGE OWA. LOCAL INBOX: I have pop account going to 'local' folders. Filters either leave the mail in Inbox, or move the email to the appropriate local folder. start looking through folders, checking for issues... Several folder appear fine... Issue 1) folder has 8 msgs, 1 unread BUT list pane shows every email twice except for the Unread one and the info label shows 2 unread, 16 total. Leaving the unread email selected until it becomes 'read' causes Unread count to drop to 1 (still incorrect though). CTRL-A notes 15 selected, 16 total Issue 2) folder has same problem as Issue 1, info label lists 16 unread 1358 total. CTRL-A notes 1357 selected, 1358 total. If I tell list pane to show only unread messages and CTRL-A, info pane notes 16 unread and 14 selected Issue 3) folder same as 1 and 2, info label lists 288 drafts, tree notes 188 unread. All drafts are listed twice. CTRL-A notes 188 selected, 282 drafts. List pane shows NO messages as unread. EXCHANGE OWA: Issue1) Inbox tree view lists 315 unread info label lists -3 unread -24 total File/Quit evolution --force-shutdown restart let evo refresh and fetch summaries.... LOCAL INBOX: folder with Issue 1 above is now correct -- tree lists no unread messages, info label lists 8 total. folder with Issue 2 above appears correct -- tree lists 6 unread info label lists 6 unread 679 total. CTRL-A notes 679 selected 679 total folder with Issue 3 above tree notes 94 unread Info label lists 94 drafts. CTRL-A info label notes 94 selected 94 drafts EXCHANGE OWA: Inbox tree view lists 318 unread info label lists 315 unread 1539 total I select my spam folder - it has 6 msgs, all unread, tree view lists 6 unread info label lists 6 unread 6 total. I begin deleting msg by msg. after deleting 3 msgs and letting cursor selection turn two of the remaining 3 to 'read', tree notes 1 unread, info label notes 1 unread 6 total ( ??? 3 visible + 3 deleted but hidden msgs ??? ). File empty trash -> info label still lists 6 total. CTRL-E expunge, info label still lists 6 total. select another folder and return, still lists 6 total. Other folders have mis-matches between tree unread and info label unread Folder/Refresh Folder does not correct invalid information.
Reid, as I stated in comment #11, this mismatch is because of db-summary, let it be part of bug #543389 (some subbug is about that already, I think)
(In reply to comment #21) > Reid, as I stated in comment #11, this mismatch is because of db-summary, let > it be part of bug #543389 (some subbug is about that already, I think) > Sorry -- Again. I will post the data under comment 20 to #543389.
I'm using Ubuntu 9.04 (64 bit) Evolution version 2.26.1 Exchange 2003 Comment #4 after stopping evolution and rm'ing the exchange elements descibes exactly what I am experiencing. The trigger which initiated the problem was using the web-access to the server and Evolution's access simultaneously. I have not been able to repair the situation using the rm techniques suggested, before sync is complete the emails are listed but most disappear on completion. They are, of course, still present in Outlook or web-access. New mails may or may not show up and may or may not remain in the Evolution display of the inbox. This also affects my IMAP connexion but that may be a separate issue since the affected emails are not all the same! This is a killer for me trying to persuade the organisation to change! I get round it running Outlook through Crossover. Help?! -- Derek