After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 478151 - I lose the ability to see older messages in my Exchange inbox
I lose the ability to see older messages in my Exchange inbox
Status: RESOLVED FIXED
Product: Evolution Exchange
Classification: Deprecated
Component: Connector
2.28.x
Other All
: High major
: ---
Assigned To: Connector Maintainer
Ximian Connector QA
Depends on:
Blocks:
 
 
Reported: 2007-09-18 19:43 UTC by Paul Smith
Modified: 2010-05-14 10:14 UTC
See Also:
GNOME target: ---
GNOME version: 2.27/2.28



Description Paul Smith 2007-09-18 19:43:55 UTC
Please describe the problem:
Every now and then (I have no idea what triggers this behavior) all the messages in my Exchange inbox older than, it appears, July 1 2007 magically disappear from view.  They are not shown in the summary window.  They are not counted in the "number of messages" count.

Stopping and restarting Evolution, even with --force-shutdown, does not help.  Letting it go for a while does not help.  The only way I've been able to see those messages through Evolution again is to "rm -rf ~/.evolution/exchange ~/.evolution/mail/exchange", then restart, and I can see them again.

Then after a while, they disappear again.

Note that if I use the webconnect interface through FireFox, they are still visible.

I've built Evo 2.12 from SVN as of Sep 17 2007 and I still see this behavior.  I also build libsoup, gtkhtml, e-d-s, evolution-exchange, and evo-webcal from the latest on the gnome-2-20 branch in SVN.  I'm running with all debugging enabled, etc. but there is nothing interesting in the log files when this happens (and I don't even notice every time it happens).

This is a regression, because I never saw this with Evo 2.10.  In fact, back when it first started I could undo the problem by running Evo 2.10 again, then exiting and coming back to Evo 2.11.x.  I haven't tried that lately since I've started just removing the ~/.evolution.../exchange directories.

Steps to reproduce:
1. Start evolution
2. Wait a while

I have no idea exactly what triggers this problem.


Actual results:
All of a sudden all mail prior to (it seems) Jul 1 2007 disappears out of my Exchange Inbox as displayed by Evolution.

Expected results:
All my mail continues to be visible.

Does this happen every time?
It's happened almost every time if I leave Evo running for long enough.  Sometimes it happens almost immediately; other times it takes a few days.  I suppose that there's some kind of message received from the Exchange server that's causing this, but I can't pin it down.

Other information:
I first reported this on the evolution-list mailing list, here:
http://mail.gnome.org/archives/evolution-list/2007-July/msg00080.html

I got a response from Veerapuram Varadhan <vvaradhan novell com>
http://mail.gnome.org/archives/evolution-list/2007-August/msg00145.html

with a patch to apply.  I did so (and I've been running with that patch applied ever since).  The problem seems to happen less frequently (sometimes it would happen multiple times a day before; now it seem to take a day or two on average).  But whether that means this change needs to be made in more than one place, or that the thing that causes the bug used to happen more often and now happens less often (and the patch has nothing to do with it) I can't tell.
Comment 2 Matthew Barnes 2008-01-29 13:28:36 UTC
Also reported on Red Hat's Bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=381351

Can someone confirm whether this problem still exists in 2.12.3?
Comment 3 Paul Smith 2008-01-30 22:10:10 UTC
I was seeing this in 2.12.0 and 2.12.1.  I've since moved to the SVN version of Evo (pre-2.22).  For a while I was seeing it there as well; all the way up to last month or so.

However, now that you mention it I haven't seen this happen for a few weeks now.  I can't say exactly how long, or even say with 100% certainty it's been fixed since it was never exactly clear what caused the problem in the first place... but it used to happen pretty quickly (within hours).  And, since it only affected fairly old email in my inbox, the only way I would normally notice it was that the summary showing the total# of messages would magically drop from 200 down to 60 or whatever but I'm not always paying attention.

But, (crossing fingers), the problem seems to be gone now.  Either someone fixed it, or else maybe something about the contents of my inbox changed so that the bug doesn't get tweaked any longer.

I'll keep a sharper eye for this on both my work and home systems and see if I see it happen again.
Comment 4 Paul Smith 2008-02-05 20:00:32 UTC
I spoke too soon: this bug still happens even in my Evo built fresh from SVN this morning (off the main branch, not the MAPI branch).
Comment 5 Bharath Acharya 2008-02-22 06:43:16 UTC
Hi Paul,

Can you please confirm if this is the same issue as http://bugzilla.gnome.org/show_bug.cgi?id=353715

TIA
Comment 6 Paul Smith 2008-02-22 12:50:09 UTC
Nope, this is _not_ the same issue.  The problem in bug # 353715 is that marking an email as junk makes it disappear.

This problem is that all email older than a specific date (or so it appears to me) spontaneously disappears.  Restarting Evo has no effect.  Then if I rm -rf ~/.evolution/exchange ~/.evolution/mail/exchange, the old mail will reappear.  Then after a while it will all disappear again.

It seems to me like some kind of local cache corruption, such that it can't see older email for some reason.  However, there are never any suspicious messages in the logs.


As you can see from the comments, I am NOT the only person experiencing this; there are lots of bugs of identical nature being reported to Red Hat, Ubuntu, and here in the Gnome bugzilla.  But, it's not the same behavior as the Junk problem.
Comment 7 Bharath Acharya 2008-02-26 04:14:43 UTC
Hi Paul,

The older email which disappears spontaneously for you, are those mails which are marked as junk by your system flags. So it looks like the same problem to me.

Can u try this http://bugzilla.gnome.org/show_bug.cgi?id=353715#c7

Let me jot down the steps in here:
1) Disable all the checks in Edit-> Preferences-> Mail Preferences-> Junk Tab.
2) Close Evolution.
3) Then try a rm -rf ~/.evolution/exchange ~/.evolution/mail/exchange.
4) Start Evolution.
5) All the mails are fetched again and the older mails as well.
6) The Older E-mails shouldn't disappear at any point in time.

After this you should not be experiencing any mails being missed out from your mailbox.

And i know a lot of users experienced this. It hit my Mailbox as well off late :) So it'd be great if you could try this out and post your comments in here. Thanks again.
Comment 8 Paul Smith 2008-02-26 04:59:22 UTC
No; I repeat these emails that disappear are NOT marked junk.  In fact, I mark very, very few emails as junk from my Exchange account because our corporate spam filters take care of almost all of it.  Also, since I know that the junk button doesn't do anything for my Exchange account, I never use it even on the few junk emails I do get.

The emails that disappear are every email in my inbox before July 1, 2007; that's about 11 months' worth of email (of course I delete a lot of email from my inbox and move them to other folders too so really it accounts for about 80-100 or so emails).

I don't see how this is related to the junk email bug.
Comment 9 Bharath Acharya 2008-02-26 05:32:17 UTC
The mails could be marked as Junk by Evo code, not explicitly by you or your corporate spam filters. If some older mails are not displayed by Evo but are visible in Web Access, that's the only explanation i got as of now. 

I faced the exact problem as you did. I debugged through the code and this was what i could see. SpamAssassin or the BogoFilter did play around a lot. They marked Older mails as Junk without the User's Knowledge. They marked some Older mails as Junk and since Exchange had no code to handle it, all it could do is not show them anywhere and that is what it did. So it appeared as if the local cache was corrupted, but it was not. 

Since it did fix the same problem for me, I thought i could try it with you and verify it out. I want to nail down this issue and need your help to get this fixed :)
Comment 10 Paul Smith 2008-02-29 23:42:53 UTC
OK.  On my home system I had this happen again.  I disabled all my junk filtering then restarted Evo; still have missing email.  I stopped Evo and removed my ~/.evolution/exchange and ~/.evolution/mail/exchange directories.  I restarted Evo and my missing mail (~100 messages, all before July 1 2007 as far as I can tell) was back again.

However, that was ALWAYS true; I could always get the mail to come back again by removing ~/.evolution/exchange and ~/.evolution/mail/exchange.  So, I'm not really sure what I'm supposed to do now.  I guess if the problem happens again we'll know it's not the junk filtering, but if not how do I prove a negative?  Lately the problem has been a lot less frequent than it used to be and I never knew exactly how to force it to happen.  Running without any junk filtering enabled is going to be a serious drag for mail to my ISP (IMAP) account so I'd rather not have to do it for weeks :-/.
Comment 11 Paul Smith 2008-03-04 02:43:08 UTC
I lived with no spam filtering for the entire weekend (ugh!) then tonight I got the same problem on my home system: my Exchange inbox went from ~510 email messages to 391.  Checking I can see that, as before, all messages older than Jan 02 2007 are missing.  If I go look with OWA or if I remove ~/.evolution/exchange and ~/.evolution/mail/exchange, they all come back.

I think (?) this should be pretty clear that it's not the spam filtering that's causing this bug, but rather something else.  I'm going to re-enable my spam filtering (phew!)

If there's anything I can do that will help you track down the problem please let me know; I build Evo from SVN myself with full debugging and I'm happy to add printfs or attach with the debugger, or if there are any utilities that will investigate/validate the Exchange cache (or even things I can look for with Emacs or whatever) I'm willing to do that as well.
Comment 12 Chris Dragan 2008-03-12 08:07:46 UTC
I have the exact same problem. I didn't have it with 2.10 though, it appeared first in 2.12 and it's still there in 2.21.91. I'm glad I found this bug, removing the two exchange folders helps. Before that, the new Evolution was unusable to me. Like Paul, I can't see any errors in the log. I would also be glad to help with resolving this issue.
Comment 13 Christopher Stokes 2008-04-25 15:42:55 UTC
I too have this issue. I have junk filtering disabled.

Removing the .evolution/exchange folder gets the mail back but eventually something triggers and the email no longer shows up.

Question: how big is everyone else's folder? Mine currently comes in at 327644.71 KB according to the folder properties box. My smaller folders do not seem to suffer from this problem.

If there is anything I can do to assist with this please let me know!

uname -a
Linux chris-desktop 2.6.24-16-generic #1 SMP Thu Apr 10 13:23:42 UTC 2008 i686 GNU/Linux

evolution --version
GNOME evolution 2.22.1
Comment 14 Paul Smith 2008-04-25 16:48:02 UTC
My inbox is ~49M.  It's the only folder of any size I have (other than Deleted Items, from which I always keep this month and last month's deleted email and delete everything before that, on the first of each month).  I've only ever noticed the problem in my Inbox but that doesn't mean much either way.  It could be size-related, or just related to the Inbox.

Do you ever see it in any other folder besides Inbox?  It might just be that that's where all your mail arrives, so that's where the summary is being updated on Send/Receive.  I don't have any filtering on my Exchange account.
Comment 15 Reid Thompson 2008-04-25 16:57:35 UTC
Randomly, folders with hundreds or thousands of emails will begin
displaying in Evolution with only a very small subset of the folder
content.  Ex:  I have a folder with over 5K emails in it.  Today, it
started displaying only 26 emails.
This is with Exchange OWA.
I quit evo, did a force-shutdown and cd'd into the folder's directory
and rm'd the summary file ( it obviously appeared to not have a summmary
of all of the emails ).  I then restarted evo.  Evo rebuilt the summary
file correctly and again displayed all ~5K emails.  

What causes the summary file to be modified?  

Is there a mechanism to force a re-generation of the summary file w/o
the steps above re rm'ing the file?


I recall a bug or bugs being posted for this, but I couldn't get the
right combination of search words to pull it up to add this info...
Comment 16 Jonathan Ryshpan 2008-07-01 18:46:05 UTC
I have a similar problem with locally stored mail; my machine doesn't have a connection with a remote evolution server or any other mail server.  In particular, the advice to clear ~/.evolution/.../exchange is not relevant, since no such folder exists.

Here's a brief description of the problem:

I see no messages displayed in Inbox earlier than Sept 11, 2007, though
I know there are earlier ones.  If I search Inbox for a message that I
know is there, received in 2005, it's not found.  But if I search the
*Account*, rather than Inbox, for the same message it is found.  

Can anyone explain this?

Inbox is rather large:
        3032 unread 8014 total
Comment 17 Jonathan Ryshpan 2008-07-04 19:58:51 UTC
I just deleted all of the files Inbox.ev-summary* and Inbox.ibex.index*.
When Evolution was restarted, it recreated them all, apparently
successfully.  However the original problem remains, namely nothing
shows up on the message summary window earlier than Sept 11 2007.  But
earlier messages can be found by searching the account, though not by
searching the folder.

If desired, I can post (rather long) log of the restart.
Comment 18 Dooma 2008-07-09 00:07:44 UTC
I believe this bug is the same or similar to one which I reported recently: 

http://bugzilla.gnome.org/show_bug.cgi?id=489236



Comment 19 jrobiso2 2008-07-23 00:23:06 UTC
Well, it's still going strong in 2.22.1.  I've lost all exchange emails since yesterday.  Nothing dated 07/21 or earlier shows up.  I've done a complete wireshark trace, including using "Track TCP Stream" (shows conversations between client and server in a readable format, like you were watching the http requests themselves in English).

It's absurd that this isn't getting more attention.  My email only had 330 emails in it.
Comment 20 Bharath Acharya 2008-08-06 08:48:05 UTC
@Paul,Reid 
Can anyone try the patch at http://bugzilla.gnome.org/show_bug.cgi?id=489236#c7. Milan has fixed another scenario in which the calculation of article numbers were being messed up. Would be good to see if that fixes this or if we still have more issues.
Comment 21 Gary Mansell 2008-11-20 11:09:20 UTC
I can confirm that the missing mail messages is still present with Evolution exchange 2.24 series that comes with Ubuntu 8.10.

I can see this listed on the Ubuntu Launchpad bug tracker, the redhat bug tracker and also in numerous bugs on the gnome bug tracker.

https://bugs.launchpad.net/ubuntu/+source/evolution-exchange/+bug/160142
https://bugzilla.redhat.com/show_bug.cgi?id=381351


Anyone got any suggestions out there?

Rgds

Comment 22 Reid Thompson 2008-12-04 18:01:54 UTC
This has reached the point that I have to --force-shutdown &&  rm  -rf  .evolution/exchange/ .evolution/mail/exchange/  just about daily now.  Not sure if this is an OWA only problem or not...
 
Comment 23 cott 2009-01-06 14:18:44 UTC
I'm still seeing this in Evolution 2.24.2. I too have to purge the ~/.evolution/exchange/mail/[account] directory every day or two in order to get my mail back, despite disabling all junk mail filtering as suggested above.
Comment 24 Robert Piasek 2009-07-10 08:35:59 UTC
evolution 2.26.2 still the same problem. This bug is getting pretty old now :)
Comment 25 mjs 2009-07-13 17:40:05 UTC
(In reply to comment #24)
> evolution 2.26.2 still the same problem. This bug is getting pretty old now :)
> 

Your smiley is upside down 8^(.

I think I have a fairly reliable way to reproduce this.  It happens to me this way, in any case.

1. Have OWA account set up with large inbox.
2. Access the account with a different OWA client (e.g., iPhone or Web browser).
3. Access the account with evolution.
4. Watch the fireworks as messages vanish in real time.

Eventually, some sort of refresh is done that restores the missing messages; or one can remove the local copies of the mailbox as above and wait for them to from scratch.

Makes me very averse to reading mail with my iPhone...
Comment 26 Akhil Laddha 2009-07-29 13:35:22 UTC
bug 581908 fix should help here.  
Comment 27 Paul Smith 2010-01-15 19:41:13 UTC
According to the comments in bug 581908 the patch there does not fix the problem.

My corporate exchange server was upgraded to Exchange 2007 so I can no longer use the evolution-exchange backend to determine if this bug still exists or to test patches, etc.
Comment 28 Endre C 2010-03-11 10:19:08 UTC
The bug still lives in 2.28.1.
I found that it should be related with filtering somehow.

In my mailbox settings I turned off 'Apply filters to new messages in Inbox on this server', deleted the folders as described above, and after the next synchronization I can see all of the missing messages.
I have no local mail filters set up, but when this box was checked, I can see my messages while the filtering process was running, but they disappeared immediately when this process finished, but still counted as unread in Inbox.
Comment 29 Milan Crha 2010-03-25 13:08:17 UTC
I thought this was fixed already, in some bug I'm unable to recall and find at the moment. Because 2.28 will not receive any further updates, could anyone of you try to reproduce with an upcoming 2.30.0 and update this bug report, please? 

Because I really know I was trying to fix something similar, one was just like this, and another similar involved Archive feature of an Outlook (after that called eex was pretty confused, if I recall correctly).

Thanks in advance.
Comment 30 Tobias Mueller 2010-05-14 10:14:46 UTC
Closing as FIXED as per comment 29. Please retest with 2.30.x and reopen if this is still an issue. Thanks in advance.