After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 489236 - Evolution Inbox not displaying messages which have already been opened in another application using exchange - regression
Evolution Inbox not displaying messages which have already been opened in ano...
Status: RESOLVED FIXED
Product: Evolution Exchange
Classification: Deprecated
Component: Connector
2.12.x
Other All
: Normal critical
: ---
Assigned To: Connector Maintainer
Ximian Connector QA
Depends on:
Blocks:
 
 
Reported: 2007-10-23 02:10 UTC by Dooma
Modified: 2009-07-09 23:25 UTC
See Also:
GNOME target: ---
GNOME version: 2.19/2.20


Attachments
proposed eex patch (1.28 KB, patch)
2008-07-07 11:33 UTC, Milan Crha
needs-work Details | Review
proposed eex patch ][ (3.56 KB, patch)
2008-08-05 13:19 UTC, Milan Crha
none Details | Review
proposed eex patch ]I[ (2.30 KB, patch)
2008-08-14 12:04 UTC, Milan Crha
committed Details | Review

Description Dooma 2007-10-23 02:10:29 UTC
Please describe the problem:
I have two computers. One runs Gutsy, the other XP.

I use MS Exchange 2003 exclusively for email on both systems.

If I check (i.e. open and read) an email on my XP computer, (or using OWA on the Gutsy computer) that email will not appear in my Evolution Inbox unless I reply to it or forward it.

If I get the email while sitting at my XP system and do not reply to it or forward it, I will never be able to even see it in my Evolution Inbox. The only way to make the message appear in the inbox is to go to the other computer and forward it or reply to it.

This is a new problem with Evolution 2.12.  This problem never occured before. This is a critical problem because it renders Evolution essentially useless.

I wanted to file this bug under "Evolution Exchange" but it does not list "2.12" there. 

Steps to reproduce:
1. YOU MUST BE USING EXCHANGE 
2. Send yourself an email from gmail or something. 
2. Open that email by clicking on it on OWA in Firefox or some other program. Do not reply or fwd it.  
3. Open Evolution and look for that email in the inbox.


Actual results:
The email does not appear in Evolution if it has first been opened in OWA. 

Expected results:
Normally, the email should appear in the evolution's inbox AND the OWA inbox.

Does this happen every time?
YES

Other information:
This is an exchange bug, and should be addressed by people using exchange 2003
Comment 1 Sankar P 2008-05-18 17:25:39 UTC
Do you see this happening with 2.22 as well. Iirc, a lot of missing mail bugs were fixed during 2.22. Moving to the right product evolution-exchange.
Comment 2 Milan Crha 2008-07-07 11:33:14 UTC
Created attachment 114109 [details] [review]
proposed eex patch

for evolution-exchange;

I'm not sure whether this belongs to this bug or not, but it sounds very similar, thus I will use this report. I had something similar, when opening Inbox folder, I didn't see all messages as in owa. I do not know what caused it exactly, but the change in refresh_folder_internal to not include check for high_article_num in the search for messages helped. Unfortunately, I get lost the old summary, so I do not know what number was stored there and why.
Anyway, I didn't change anything in this function, I rather stepped a few functions above, to force full refresh when it seems like good idea.
Comment 3 Srinivasa Ragavan 2008-07-08 21:00:12 UTC
Seems fine for me. Bharath what do you say?
Comment 4 Dooma 2008-07-09 00:04:38 UTC
This problem was dormant on my system for a long long time, until yesterday when things started going haywire again. The problem began when i rebooted an old computer using Microsoft outlook which I had not used for a very long time. As soon as I pressed send receive on that computer's outlook, all emails older than July 3 2008 were immediately gone from my evolution inbox. It could be just coincidence that these things happened at exactly the same time, but I think it is not.   In the past, I was able to fix this problem of missing emails by doing: 

cd ~/.evolution
evolution --force-shutdown
rm -rf `find . -name *exch* -print` 

When I do this now, the missing emails show up while the inbox is resyncing, but once it finishes resyncing, they disappear, or they disappear if I press "send receive".  

From my research online, I'm pretty certain that this bug is related to the following bugs and is probably the same problem, so it could be a duplicate of these bugs: 
http://bugzilla.gnome.org/show_bug.cgi?id=435371
http://www.mail-archive.com/evolution-list@gnome.org/msg07847.html
http://www.schnuckelig.eu/blog/evolution-exchange-connector-missing-mails-inbox

I am failry new to linux and don't really know how to apply the proposed patch, so if anyone wanted to help me I'd totally love that, but more generally I'd love to see the patch applied somehow by someone so the next time I get an evolution update it is included. 

Thank you all so so much!

Dooma

Comment 5 Veerapuram Varadhan 2008-07-09 09:49:46 UTC
(In reply to comment #2)
> Created an attachment (id=114109) [edit]
> proposed eex patch
> 
> for evolution-exchange;
> 
> I'm not sure whether this belongs to this bug or not, but it sounds very
> similar, thus I will use this report. I had something similar, when opening
> Inbox folder, I didn't see all messages as in owa. I do not know what caused it
> exactly, but the change in refresh_folder_internal to not include check for
> high_article_num in the search for messages helped. Unfortunately, I get lost
> the old summary, so I do not know what number was stored there and why.
> Anyway, I didn't change anything in this function, I rather stepped a few
> functions above, to force full refresh when it seems like good idea.
> 
Missing mails is normally an issue w.r.t calculation of article-number.  However, recalculating from the start is not a solution.  In get_folder_contents_online, we fetch all the mails from server and compare/manipulate them against the ones that we have in cache and appropriately update our cache with the changes from the server.  I do agree that something is triggering a mis-calculation of article number and refetching all the mails again will cost performance of the connector. In our labs, we have tried all possible scenarios that we know and are not able to reproduce the behavior as mentioned in various reports except for marking them as junk (messages were getting wrongly marked as junk by the spamassasin plugin). 

An alternative approach can be tried here:

i) Store PR_CREATION_DATE and PR_MODIFICATION_TIME and use them as conditions to fetch records from the server in get_folder_contents_online
ii) Iterate over these set of records and find the high_article_number and compare it with that of cached value.  Whichever is smaller, use it as the marker for refreshing the folder for new messages.

I am not able to recall as to what was the issue in using CREATION and MODIFICATION time, for now.  I will update the bug once I get some data in hand.

Also, change the way "seq" is used to keep track of deleted messages and use "uid" instead. 

(Marking patch to "needs-work").
Comment 6 Milan Crha 2008-07-14 14:27:37 UTC
varadhan, ping. How is it going with your patch?
Comment 7 Milan Crha 2008-08-05 13:19:24 UTC
Created attachment 115897 [details] [review]
proposed eex patch ][

for evolution-exchange;

I fixed totally 3 things in get_folder_contents_online:
a) do not skip any message for decreasing
b) call rm_idx_uid on all the left items
c) if the high_article_num should be decreased, then it will be

There is only one thing I'm not sure about, if there will be called 'refresh_folder_internal' before 'get_folder_contents_online', and eex will be forced to decrease, then it should ideally refresh folder again, any idea how or if really necessary? (it will be refreshed next time user selects the folder, but that can be "too late".)

One note: this patch has been done on gnome-2-22 branch, because db-summary has its own "features" how to hide eex messages from user's view at the moment.
Comment 8 Reid Thompson 2008-08-06 12:23:07 UTC
Tried to respond to email request ( answer is at bottom of text )....

This is the mail system at host relay.mail.twtelecom.net.

I'm sorry to have to inform you that your message could not
be delivered to one or more recipients. It's attached below.

For further assistance, please send mail to postmaster.

If you do so, please include this problem report. You can
delete your own text from the attached returned message.

                   The mail system

<bugzilla-daemon@bugzilla.gnome.org>: host mail.gnome.org[209.132.176.177]
    said: 554 5.7.1 <bugzilla-daemon@bugzilla.gnome.org>: Recipient address
    rejected: Your dictionary attacks have tired us, buzz off (in reply to RCPT
    TO command)



Reporting-MTA: dns; relay.mail.twtelecom.net
X-Postfix-Queue-ID: C896B7981DE
X-Postfix-Sender: rfc822; Reid.Thompson@ateb.com
Arrival-Date: Wed,  6 Aug 2008 06:19:46 -0600 (MDT)

Final-Recipient: rfc822; bugzilla-daemon@bugzilla.gnome.org
Original-Recipient: rfc822;bugzilla-daemon@bugzilla.gnome.org
Action: failed
Status: 5.7.1
Remote-MTA: dns; mail.gnome.org
Diagnostic-Code: smtp; 554 5.7.1 <bugzilla-daemon@bugzilla.gnome.org>:
    Recipient address rejected: Your dictionary attacks have tired us, buzz off



Subject:
Re: [Bug 478151] I lose the ability to see older messages in my Exchange inbox
From:
Reid Thompson <Reid.Thompson@ateb.com>
Date:
Wed, 06 Aug 2008 08:19:43 -0400
To:
"Evolution Exchange (bugzilla.gnome.org)" <bugzilla-daemon@bugzilla.gnome.org>

Evolution Exchange (bugzilla.gnome.org) wrote:
> If you have any questions why you received this email, please see the text at
> the end of this email. Replies to this email are NOT read, please see the text
> at the end of this email. You can add comments to this bug at:
>   http://bugzilla.gnome.org/show_bug.cgi?id=478151
>
>   Evolution Exchange | Connector | Ver: 2.10.x
>
>
>
>
> ------- Comment #20 from Bharath Acharya  2008-08-06 08:48 UTC -------
> @Paul,Reid Can anyone try the patch at
> http://bugzilla.gnome.org/show_bug.cgi?id=489236#c7. Milan has fixed another
> scenario in which the calculation of article numbers were being messed up.
> Would be good to see if that fixes this or if we still have more issues.
>
>
>   
applying patch to svn head and rebuilding now.
Comment 9 Reid Thompson 2008-08-06 13:51:41 UTC
The issue is still present on my system.
Path: .
URL: http://svn.gnome.org/svn/glib/trunk
Repository Root: http://svn.gnome.org/svn/glib
Repository UUID: 5bbd4a9e-d125-0410-bf1d-f987e7eefc80
Revision: 7315
Node Kind: directory
Schedule: normal
Last Changed Author: djihed
Last Changed Rev: 7315
Last Changed Date: 2008-08-06 04:30:32 -0400 (Wed, 06 Aug 2008)

Path: .
URL: http://svn.gnome.org/svn/libsoup/trunk
Repository Root: http://svn.gnome.org/svn/libsoup
Repository UUID: 03f96f1e-eb25-0410-be34-e7bec31b2f82
Revision: 1150
Node Kind: directory
Schedule: normal
Last Changed Author: danw
Last Changed Rev: 1149
Last Changed Date: 2008-08-04 09:42:30 -0400 (Mon, 04 Aug 2008)

Path: .
URL: http://svn.gnome.org/svn/evolution-data-server/trunk
Repository Root: http://svn.gnome.org/svn/evolution-data-server
Repository UUID: d0434b6f-c725-0410-8785-fd8a057797ef
Revision: 9281
Node Kind: directory
Schedule: normal
Last Changed Author: sragavan
Last Changed Rev: 9281
Last Changed Date: 2008-08-06 07:34:44 -0400 (Wed, 06 Aug 2008)

Path: .
URL: http://svn.gnome.org/svn/gtkhtml/trunk
Repository Root: http://svn.gnome.org/svn/gtkhtml
Repository UUID: 6715bd1f-e025-0410-99cf-e594b4b1144d
Revision: 8910
Node Kind: directory
Schedule: normal
Last Changed Author: djihed
Last Changed Rev: 8910
Last Changed Date: 2008-08-06 04:31:51 -0400 (Wed, 06 Aug 2008)

Path: .
URL: http://svn.gnome.org/svn/evolution-webcal/trunk
Repository Root: http://svn.gnome.org/svn/evolution-webcal
Repository UUID: 30acfedd-e925-0410-bc1e-a2789d043b2b
Revision: 437
Node Kind: directory
Schedule: normal
Last Changed Author: djihed
Last Changed Rev: 437
Last Changed Date: 2008-08-06 04:30:05 -0400 (Wed, 06 Aug 2008)

Path: .
URL: http://svn.gnome.org/svn/evolution-exchange/trunk
Repository Root: http://svn.gnome.org/svn/evolution-exchange
Repository UUID: cbd800bf-df25-0410-b17b-96628719a9b0
Revision: 1730
Node Kind: directory
Schedule: normal
Last Changed Author: djihed
Last Changed Rev: 1730
Last Changed Date: 2008-08-06 04:30:03 -0400 (Wed, 06 Aug 2008)

Path: .
URL: http://svn.gnome.org/svn/evolution/trunk
Repository Root: http://svn.gnome.org/svn/evolution
Repository UUID: 9792ae6b-c725-0410-a7bc-87ac6a6ec1ac
Revision: 35915
Node Kind: directory
Schedule: normal
Last Changed Author: mmonreal
Last Changed Rev: 35915
Last Changed Date: 2008-08-06 05:32:08 -0400 (Wed, 06 Aug 2008)
Comment 10 Reid Thompson 2008-08-06 15:11:57 UTC
sorry -- i should have been more specific -- at this point, i've not seen any emails 'disappearing', but the folder counts for emails are still incorrect.
Comment 11 Milan Crha 2008-08-06 15:28:02 UTC
you scared me a bit :) the folder counts, can it be a part of db-summary? we have such bugs against it, even on imap. Is it a total-count or unread-count broken. In the tree and/or in the info-label (left top, just above the tree)
Any dots ("...") there? (I know, this is foolish question, I'm sorry)
Comment 12 Reid Thompson 2008-08-06 16:42:31 UTC
very sorry for the scare...

Both numbers in the info-label would get into an invalid state. i.e. i've seen both with a negative number displayed.

I'll have to pay more attention to the tree folder count to advise whether it becomes invalid or not.
Comment 13 Reid Thompson 2008-08-06 19:37:46 UTC
the tree folder count can become invalid also.  Just had a folder that was empty receive some new msgs.  tree count remained at nothing displayed, i deleted the messages and info label counters went negative, tree folder count remained at nothing displayed
Comment 14 Reid Thompson 2008-08-06 19:39:16 UTC
hmmm -- tree folder count has now 'updated' itself to list correct number of unread messages.  info label counters remain invalidly negative, and decrement even more negatively as emails are deleted
Comment 15 Bharath Acharya 2008-08-14 08:29:18 UTC
Patch looks free of regressions. But I can't see this happening for me to test. I see positive responses at comment #10 I would prefer a review from our exchange expert on this one :) Varadhan?
Comment 16 Veerapuram Varadhan 2008-08-14 09:41:12 UTC
(In reply to comment #7)
> Created an attachment (id=115897) [edit]
> proposed eex patch ][
> 
> for evolution-exchange;
> 
> I fixed totally 3 things in get_folder_contents_online:
> a) do not skip any message for decreasing
> b) call rm_idx_uid on all the left items
> c) if the high_article_num should be decreased, then it will be
> 
> There is only one thing I'm not sure about, if there will be called
> 'refresh_folder_internal' before 'get_folder_contents_online', and eex will be
> forced to decrease, then it should ideally refresh folder again, any idea how
> or if really necessary? (it will be refreshed next time user selects the
> folder, but that can be "too late".)
> 
> One note: this patch has been done on gnome-2-22 branch, because db-summary has
> its own "features" how to hide eex messages from user's view at the moment.
> 
Good work, Milan.  However, my patch is quite simple. ;-) Your b) and c) fix is quite good. 

It unconditionally assigns the "re-calculated" high_article_num to mfld->high_article_num because "high_article_num" is the fresh and updated one. 

get_folder_contents_online is the one that gets called when get_folder is called for the first time.  Also, if summary exists for a folder, refresh_folder_internal will not be called unless user refreshes the folder or hit Send/Recv. 

Comment 17 Milan Crha 2008-08-14 12:04:38 UTC
Created attachment 116567 [details] [review]
proposed eex patch ]I[

for evolution-exchange;

as requested.
Comment 18 Veerapuram Varadhan 2008-08-14 12:10:41 UTC
(In reply to comment #17)
> Created an attachment (id=116567) [edit]
> proposed eex patch ]I[
> 
> for evolution-exchange;
> 
> as requested.
> 
Patch is ready to rock the trunk. Good work, Milan. 
Comment 19 Milan Crha 2008-08-14 12:23:13 UTC
Committed to trunk. Committed revision 1744.

Thanks, I'm looking forward to see your improvements you mentioned on IRC.
Comment 20 Reid Thompson 2008-08-14 18:44:56 UTC
built from svn head
evolution --force-shutdown
rm  -rf  .evolution/exchange/ .evolution/mail/exchange/
restart
evo began rebuilding summaries, etc, etc
evo finished rebuilding summaries etc etc

Comments in two sections below LOCAL INBOX and EXCHANGE OWA.

LOCAL INBOX:
I have pop account going to 'local' folders.
Filters either leave the mail in Inbox, or move the email to the appropriate
local folder.
start looking through folders, checking for issues...
Several folder appear fine...
Issue 1)  folder has 8 msgs, 1 unread BUT list pane shows every email twice
except for the Unread one and the info label shows 2 unread, 16 total.  Leaving
the unread email selected until it becomes 'read' causes Unread count to drop to
1 (still incorrect though).  CTRL-A notes 15 selected, 16 total
Issue 2) folder has same problem as Issue 1, info label lists 16 unread 1358 
total.  CTRL-A notes 1357 selected, 1358 total.  If I tell list pane to show
only unread messages and CTRL-A, info pane notes 16 unread and 14 selected
Issue 3) folder same as 1 and 2, info label lists 288 drafts, tree notes 188
unread.  All drafts are listed twice.  CTRL-A notes 188 selected, 282 drafts.
List pane shows NO messages as unread.


EXCHANGE OWA:
Issue1) Inbox tree view lists 315 unread info label lists -3 unread -24 total

File/Quit
evolution --force-shutdown
restart

let evo refresh and fetch summaries....

LOCAL INBOX:
folder with Issue 1 above is now correct -- tree lists no unread messages, info
label lists 8 total. 
folder with Issue 2 above appears correct -- tree lists 6 unread info label lists
6 unread 679 total.  CTRL-A notes 679 selected 679 total
folder with Issue 3 above tree notes 94 unread Info label lists 94 drafts.
CTRL-A info label notes 94 selected 94 drafts

EXCHANGE OWA:
Inbox tree view lists 318 unread info label lists 315 unread 1539 total

I select my spam folder - it has 6 msgs, all unread, tree view lists 6 unread
info label lists 6 unread 6 total.  I begin deleting msg by msg. after deleting
3 msgs and letting cursor selection turn two of the remaining 3 to 'read', tree
notes 1 unread, info label notes 1 unread 6 total ( ??? 3 visible + 3 deleted but hidden msgs  ??? ). File empty trash -> info label still lists 6 total. CTRL-E expunge, info label still lists 6 total.  select another folder and return, still lists 6 total.

Other folders have mis-matches between tree unread and info label unread

Folder/Refresh Folder does not correct invalid information.
Comment 21 Milan Crha 2008-08-15 09:55:50 UTC
Reid, as I stated in comment #11, this mismatch is because of db-summary, let it be part of bug #543389 (some subbug is about that already, I think)
Comment 22 Reid Thompson 2008-08-15 11:26:22 UTC
(In reply to comment #21)
> Reid, as I stated in comment #11, this mismatch is because of db-summary, let
> it be part of bug #543389 (some subbug is about that already, I think)
> 

Sorry -- Again.  I will post the data under comment 20 to #543389.
Comment 23 Derek Harding 2009-07-09 23:25:32 UTC
I'm using Ubuntu 9.04 (64 bit) Evolution version 2.26.1 Exchange 2003

Comment #4 after stopping evolution and rm'ing the exchange elements descibes exactly what I am experiencing. The trigger which initiated the problem was using the web-access to the server and Evolution's access simultaneously.

I have not been able to repair the situation using the rm techniques suggested, before sync is complete the emails are listed but most disappear on completion. They are, of course, still present in Outlook or web-access.

New mails may or may not show up and may or may not remain in the Evolution display of the inbox.

This also affects my IMAP connexion but that may be a separate issue since the affected emails are not all the same!

This is a killer for me trying to persuade the organisation to change! I get round it running Outlook through Crossover.

Help?!

-- 
Derek