After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 550414 - Corruption of mailbox and can't expunge trash
Corruption of mailbox and can't expunge trash
Status: RESOLVED FIXED
Product: evolution
Classification: Applications
Component: Mailer
2.32.x (obsolete)
Other All
: Immediate blocker
: ---
Assigned To: Chenthill P
Evolution QA team
evolution[disk-summary]
: 544731 548973 551253 555427 558046 558735 559911 560717 561803 562350 562828 563352 563589 564369 rpaco 564623 564636 564971 565228 565235 565718 566099 566543 566554 566732 568004 568070 568411 568426 568665 568740 569629 570112 570274 570523 570926 572039 572165 572512 573299 575399 578734 579067 579102 579712 579745 580160 583232 583553 584643 584749 591568 593254 593994 594475 597501 598023 599856 601717 602683 602816 603023 603979 604140 604256 604654 606485 606600 607541 608964 608992 609295 609300 609412 611094 611338 613390 613391 614251 614871 614887 615414 617467 618092 624059 624627 624881 625429 626097 626295 626973 627316 627414 627560 627593 627697 628475 628592 630388 630389 630605 630772 631741 635046 635622 636386 637928 639297 642430 642924 645319 645561 647593 648843 649819 650472 651148 665183 (view as bug list)
Depends on:
Blocks: 543389 617261
 
 
Reported: 2008-09-02 08:50 UTC by Robert Fox
Modified: 2011-11-30 10:35 UTC
See Also:
GNOME target: 2.30.x
GNOME version: 2.27/2.28


Attachments
Snapshot of corrupted Inbox with duplicates (156.85 KB, image/jpeg)
2008-09-05 11:27 UTC, Robert Fox
  Details
Example of error when I try to expunge the Trash box (17.07 KB, image/jpeg)
2008-09-05 13:32 UTC, Robert Fox
  Details
Test patch (496 bytes, patch)
2009-01-22 05:24 UTC, Srinivasa Ragavan
reviewed Details | Review
Capture - Unread mail vfolder (97.46 KB, image/png)
2009-01-22 18:25 UTC, toystorie69
  Details
EvoLog POP 3 (58.28 KB, application/octet-stream)
2009-06-10 13:21 UTC, H.Habighorst
  Details
EvoLog POP 3 (58.28 KB, application/octet-stream)
2009-06-10 13:22 UTC, H.Habighorst
  Details
filters.xml with the relavant rules described in comment #102 (15.30 KB, application/xml)
2009-08-31 17:51 UTC, Christoph Wickert
  Details
proposed eds patch (2.01 KB, patch)
2010-05-26 19:36 UTC, Milan Crha
committed Details | Review
patch for gnome-2-28 (1.90 KB, patch)
2010-05-27 02:08 UTC, Thomas
rejected Details | Review
proposed eds patch (for 2.30) (8.21 KB, patch)
2010-05-27 12:26 UTC, Milan Crha
committed Details | Review
patch for gnome-2-28 (7.69 KB, patch)
2010-05-27 17:12 UTC, Thomas
none Details | Review
patch for gnome-2-28 (corrected (6.88 KB, patch)
2010-05-27 19:49 UTC, Thomas
reviewed Details | Review
Screenot of the error message in 2.32.1 (124.54 KB, image/png)
2011-02-25 18:24 UTC, Christoph Wickert
  Details
proposed eds patch ][ (6.69 KB, patch)
2011-04-29 08:00 UTC, Milan Crha
committed Details | Review
eds patch ][ for 2.32.2 (6.60 KB, patch)
2011-05-02 06:58 UTC, Milan Crha
reviewed Details | Review
Adopted a patch for evolution 2.32 to 2.28, https://bugzilla.gnome.org/attachment.cgi?id=187010 (6.70 KB, patch)
2011-05-17 22:27 UTC, Thomas
reviewed Details | Review

Description Robert Fox 2008-09-02 08:50:56 UTC
Please describe the problem:
Since latest releases of Evolution (evolution-2.23.91)- I get corrupted mailboxes and can't expunge the trash folder - constantly getting errors like: "Summary and
folder mismatch, even after a sync"

Steps to reproduce:
1. Open Evolution
2. try and expunge Trash folder
3. get lots of errors

Actual results:
My inbox has many duplicates and grows by itself - when trying to clean it it gets more screwed up.  When I try and expunge the Trash folder, it doesn't work - I get several errors "Summary and
folder mismatch, even after a sync"

Expected results:
When Trash is expunged, the contents of trash disappear - cleaned out.  Inbox also should stay stable (seems corrupted)

Does this happen every time?
YES

Other information:
I am a long time Evolution user (many years) and have never experienced such behavior, and I have lost some mails due to these problems.  Not Good.  This has been since the 2.23 series
Comment 1 Srinivasa Ragavan 2008-09-03 10:55:03 UTC
Robert, sorry for this. Evolution has the new on-disk summary branch merged. We have fixed the summary folder mismatch on trunk (2.23.91), but if your mbox/summary is old/corrupted it may stay.

delete folders.db in .evolution/mail/accounts?
find . -name folders.db | xargs rm $1 or something like this in that directory?

It cleans up your summary and restart would let you rebuilt it fresh. Trash expunge may still fails saying 'no such table' which Im still working on. There are wrong count issue also. It should be already by the 2.24 release.

Duplicates? Can you explain more on it? First time or persistent? Can you delete the summary and retry? Im trying to collect more data to fix it well.
Comment 2 Robert Fox 2008-09-05 11:27:46 UTC
Created attachment 118098 [details]
Snapshot of corrupted Inbox with duplicates
Comment 3 Robert Fox 2008-09-05 11:28:12 UTC
I did the removal of the "folders.db" trick and it seemed to have helped - still awaiting the results if things are OK.

I have had this problem on several machines as well.  I am attaching an example of how the Inbox get corrupted and the duplicate entries (screenshot) - after the "folder.db" removal - it seems to be OK

BTW - Why can there be a function to "re-index" or a "summary rebuild" command or tool like the "killev" tool does??  This would be useful . . .
Comment 4 Robert Fox 2008-09-05 13:32:36 UTC
Created attachment 118104 [details]
Example of error when I try to expunge the Trash box
Comment 5 Srinivasa Ragavan 2008-09-06 19:45:33 UTC
POP provider? I heard similar issues. I know the error on expunge, where it says no such table. I think I fixed it yesterday on trunk.. Fixing too many things make me forget. I will check up anyways.
Comment 6 Robert Fox 2008-09-07 08:15:49 UTC
YES - all accounts are POP3 accounts - 
Comment 7 Srinivasa Ragavan 2008-09-08 03:12:42 UTC
Im giving pop a better focus this week. I have more POP issues reported recently with the disk summary backend.
Comment 8 Akhil Laddha 2008-09-08 05:20:45 UTC
*** Bug 551253 has been marked as a duplicate of this bug. ***
Comment 9 Srinivasa Ragavan 2008-09-12 14:01:02 UTC
Im hardly able to reproduce the pop3 dupe mails bug :( Any sequence I need to follow?
Comment 10 Robert Fox 2008-09-12 16:20:16 UTC
I don have a specific sequence - it just happened during the corruption problems - once I removed the folders.db - and since updated to latest version provided from Mandriva Cooker - evolution-2.23.92-1mdv2009.0 - things seem to be better.
Still have a problem with junk filtering and errors with spamassassin (Pipe to SpamAssassin failed, error code: 5) - but thatś something different.

The duplication problem has not returned yet.
Comment 11 Akhil Laddha 2008-10-08 03:15:06 UTC
*** Bug 555427 has been marked as a duplicate of this bug. ***
Comment 12 chewearn 2008-10-13 05:40:45 UTC
I am pointed to this bug from bug 555979.

The trash is not emptied on exit.  I got this error meessage:

Error in SQL SELECT statement: SELECT COUNT (*) FROM 'Outbox' WHERE deleted = 1 [no such table: Outbox]
Comment 13 Srinivasa Ragavan 2008-10-14 18:43:34 UTC
Robert, did you had the corruption any time after this?

Chewearn, this is a differnt bug, and I will look at bug #555979.
Comment 14 Milan Crha 2008-11-05 12:46:56 UTC
I saw duplicated emails after importing MBOX file with two messages in it, somewhere to non-inbox local folder, the second mail was shown twice, until I restarted evo, but I cannot reproduce it at the moment.
Comment 15 Akhil Laddha 2008-11-14 03:43:26 UTC
*** Bug 560717 has been marked as a duplicate of this bug. ***
Comment 16 Susana 2008-12-01 14:13:31 UTC
*** Bug 562350 has been marked as a duplicate of this bug. ***
Comment 17 Akhil Laddha 2008-12-31 02:59:28 UTC
*** Bug 566099 has been marked as a duplicate of this bug. ***
Comment 18 Akhil Laddha 2008-12-31 04:14:50 UTC
*** Bug 562828 has been marked as a duplicate of this bug. ***
Comment 19 Akhil Laddha 2008-12-31 05:04:08 UTC
*** Bug 563589 has been marked as a duplicate of this bug. ***
Comment 20 Akhil Laddha 2008-12-31 05:35:38 UTC
*** Bug 564453 has been marked as a duplicate of this bug. ***
Comment 21 Akhil Laddha 2008-12-31 05:36:56 UTC
so many dupes pointed out problem of expunge in trash so raising priority 
Comment 22 Akhil Laddha 2008-12-31 06:18:29 UTC
*** Bug 564971 has been marked as a duplicate of this bug. ***
Comment 23 Akhil Laddha 2008-12-31 06:18:40 UTC
*** Bug 561803 has been marked as a duplicate of this bug. ***
Comment 24 Akhil Laddha 2008-12-31 06:18:51 UTC
*** Bug 564623 has been marked as a duplicate of this bug. ***
Comment 25 Akhil Laddha 2008-12-31 11:15:52 UTC
*** Bug 565228 has been marked as a duplicate of this bug. ***
Comment 26 Akhil Laddha 2008-12-31 11:17:57 UTC
*** Bug 565235 has been marked as a duplicate of this bug. ***
Comment 27 rpa 2008-12-31 14:45:01 UTC
A lot of duplicates here including my own =564453.

This is I believe, because when I searched on the word "expunge" only one old thread/bug showed up and this itself was noted as a duplication of another which had been removed completely, thus it looked like mine was the only instance currently filed. 

Maybe I cant use the search system properly but if not then it's too difficult.
Comment 28 Akhil Laddha 2009-01-02 06:18:40 UTC
*** Bug 565718 has been marked as a duplicate of this bug. ***
Comment 29 Srinivasa Ragavan 2009-01-04 19:24:01 UTC
So this is purely the local provider. 'On this Computer' ? Can't empty trash ?
Comment 30 rpa 2009-01-04 20:40:44 UTC
(In reply to comment #29)
> So this is purely the local provider. 'On this Computer' ? Can't empty trash ?
> 

By local provider do you mean ISP?
Because this had been reported by many people now who must have various different IPs

What more information do you need?

 
Comment 31 lbeck4 2009-01-04 21:03:20 UTC
(In reply to comment #30)
> (In reply to comment #29)
> > So this is purely the local provider. 'On this Computer' ? Can't empty trash ?
> > 
> 
> By local provider do you mean ISP?
> Because this had been reported by many people now who must have various
> different IPs
> 
> What more information do you need?
> 
> 
> 
from my experience the "trash" folder has become corrupted. I deleted this folder/file and have been able to once more expunge trash. 

I would suggest an option to delete folders/files individually so saved emails etc don't get lost allowing the application to once again work as designed.
Comment 32 Akhil Laddha 2009-01-05 03:25:27 UTC
see bug 566543 also
Comment 33 Srinivasa Ragavan 2009-01-05 06:22:55 UTC
Do you see this on the 'Trash' on 'On This Computer' account or imap/Exchange/* accounts?
Comment 34 rpa 2009-01-05 12:56:52 UTC
It is  POP mail so the trash file is "on this computer"

I have now found that if one expunges the trash or "deleted items" as it is actually called in the application. This causes an "Error on expunge" message. The items remain in the deleted items folder. However if the Application is then closed and re-opened the Deleted items folder has been cleared.
Comment 35 Srinivasa Ragavan 2009-01-05 16:51:08 UTC
Can you see, if you get anything on the console?

Also, in a terminal set
export CAMEL_DEBUG=all
evolution 

THis dumps lots of logs. Just clear the console log, and empty trash and capture the log for last operation/empty trash and attach it here. Thanks.
Comment 36 Akhil Laddha 2009-01-19 05:05:24 UTC
*** Bug 568070 has been marked as a duplicate of this bug. ***
Comment 37 Akhil Laddha 2009-01-21 03:38:54 UTC
*** Bug 568409 has been marked as a duplicate of this bug. ***
Comment 38 Akhil Laddha 2009-01-21 03:39:32 UTC
bug 568409 has camel_debug logs 
Comment 39 Akhil Laddha 2009-01-21 04:31:24 UTC
*** Bug 568411 has been marked as a duplicate of this bug. ***
Comment 40 Robert Fox 2009-01-21 08:36:56 UTC
This problem went away for some time - but now has reappeared.  When I expunge the trash, it deletes only a portion of the mails and an error pops up about the mismatch.  If I shutdown Evolution and restart, the remainder of the mails in the trash disappear.

Here is an error in the console:

(evolution:8805): camel-local-provider-WARNING **: Didn't get the next message where I expected (5707473) got 5709086 instead                                                                                                 

(evolution:8805): camel-WARNING **: Error storing '~/.evolution/mail/local/potjunk (mbox)': Summary and folder mismatch, even after a sync                    

and here is the version I am using:

evolution-data-server-2.25.4-1mdv2009.1
evolution-2.25.4-1mdv2009.1


Thx,
Robert
Comment 41 rpa 2009-01-21 09:19:50 UTC
Sorry but I have finally given up and installed thunderbird.
I have removed myself from the cc list.
Comment 42 Milan Bouchet-Valat 2009-01-21 09:48:45 UTC
Akhil Laddha: Not sure mine is a duplicate. I'm using IMAP, which can make a big difference I guess. And I never get any error message when emptying the trash. Hope this helps.
Comment 43 Akhil Laddha 2009-01-21 10:14:50 UTC
(In reply to comment #42)
> Akhil Laddha: Not sure mine is a duplicate. I'm using IMAP, which can make a
> big difference I guess. And I never get any error message when emptying the
> trash. Hope this helps.
> 

Yeah, i noticed your bug was against IMAP provider but problem was in trash only so closed as dupe, you can see comment#38 where i mentioned that your bug has camel debug logs. You / Developer can reopen original report if he feels both are independent issues, thanks.    
Comment 44 Akhil Laddha 2009-01-22 03:41:17 UTC
*** Bug 558046 has been marked as a duplicate of this bug. ***
Comment 45 Srinivasa Ragavan 2009-01-22 05:06:32 UTC
(In reply to comment #42)
> Akhil Laddha: Not sure mine is a duplicate. I'm using IMAP, which can make a
> big difference I guess. And I never get any error message when emptying the
> trash. Hope this helps.
> 


Its different issue and Im reopening your issue again. 
Comment 46 Srinivasa Ragavan 2009-01-22 05:23:12 UTC
Guys, the main issue is this..

When you empty trash [which is a vfolder]

- It empty's one  by one all folders (expunges).
- if One folder expunge fails, it preserves the error
- continues to the next folder and goes on.

At the end, it returns the preserved error. I can compress the error, its a minor issue/fix, but the bigger issue is that why the expunge of a folder failed.

As I see its the reason 'Folder/Summary mismatch'. The reason why a folder summary mismatch could happen was fixed. [I'm sure it is fixed, since I have fixed it so many times] But the left-over-print of a previous folder-summary mismatch couldn't be removed unless you edit the mbox to remove that corrupted mail due to the previous/last mismatch.  

Lemme see, if I can write a small tool, that can read mbox files and rewrite a new one, removing the corrupted one. Infact Fejj had written one, which I can reuse for this purpose.
Comment 47 Srinivasa Ragavan 2009-01-22 05:24:49 UTC
Created attachment 126966 [details] [review]
Test patch

This is the hack, I was saying about suppressing the error. Actually since the folder is emptied, it might make sense to have this, but for now.. just try with it, mean while, lemme create that mbox tool. Thanks for your patience guys.
Comment 48 Robert Fox 2009-01-22 12:33:34 UTC
I performed the following (as mentioned earlier in this bug report):

delete folders.db in .evolution/mail/accounts?
find . -name folders.db | xargs rm $1 or something like this in that directory?

and restarted - each folder gave an error as I opened it - but after that, everything has gone back to "normal" - and I can expunge the trash without problems again.

There must be some sort of corruption which occurs in the folders.db somehow (over time)

Cheers,
Robert
Comment 49 toystorie69 2009-01-22 18:24:27 UTC
I did apply this workaround too and don't have any SQL error anymore and I am now able to delete messages and expunge Trash without problems.

But... I still have a count mismatch somewhere because the "Unread mail" vfolder still indicate two unread mails even if they are all read.
When opening this folder nothing is shown...

See snapshot.
Comment 50 toystorie69 2009-01-22 18:25:56 UTC
Created attachment 127019 [details]
Capture - Unread mail vfolder
Comment 51 Srinivasa Ragavan 2009-01-22 18:32:52 UTC
(In reply to comment #49)
> I did apply this workaround too and don't have any SQL error anymore and I am
> now able to delete messages and expunge Trash without problems.
> 
> But... I still have a count mismatch somewhere because the "Unread mail"
> vfolder still indicate two unread mails even if they are all read.
> When opening this folder nothing is shown...
> 

Is that a vfolder that shows your unread mails in n folders? if so, it is a different bug. Its just that the count is wrong and nothing else. 

Comment 52 Srinivasa Ragavan 2009-01-22 18:35:24 UTC
(In reply to comment #48)
> I performed the following (as mentioned earlier in this bug report):
> 
> delete folders.db in .evolution/mail/accounts?
> find . -name folders.db | xargs rm $1 or something like this in that directory?
> 

So, it wasn't about folders.db corrupted, but about the mbox folder being corrupted. So, it was fixed in early 2.24.x, but you were carrying the old corrupted mbox, which will create problems over a period of time. Im thinking of making a tool to rewrite such broken mboxes and ignore the corrupted mail.

-Srini
Comment 53 Robert Fox 2009-01-22 19:13:16 UTC
>So, it wasn't about folders.db corrupted, but about the mbox folder being
>corrupted. So, it was fixed in early 2.24.x, but you were carrying the old
>corrupted mbox, which will create problems over a period of time. Im thinking
>of making a tool to rewrite such broken mboxes and ignore the corrupted mail.
>
>-Srini


Sounds good - but instead of ignoring the corruption, a tool to rebuild (reindex?) the mboxes would be nicer.  Kinda like a "chkdisk" command for filesystem but for mbox - that would be cool . . .

Robert
Comment 54 toystorie69 2009-01-22 19:41:20 UTC
(In reply to comment #51)
> (In reply to comment #49)
> > I did apply this workaround too and don't have any SQL error anymore and I am
> > now able to delete messages and expunge Trash without problems.
> > 
> > But... I still have a count mismatch somewhere because the "Unread mail"
> > vfolder still indicate two unread mails even if they are all read.
> > When opening this folder nothing is shown...
> > 
> 
> Is that a vfolder that shows your unread mails in n folders? if so, it is a
> different bug. Its just that the count is wrong and nothing else. 
> 

Yes, this is the standard "Unread mail" folder that comes with evolution and I didn't modified the filter rule which is still "state / is not / read on all local folders"
Comment 55 André Klapper 2009-01-22 23:29:45 UTC
*** Bug 568665 has been marked as a duplicate of this bug. ***
Comment 56 André Klapper 2009-01-22 23:30:13 UTC
*** Bug 563352 has been marked as a duplicate of this bug. ***
Comment 57 André Klapper 2009-01-22 23:30:19 UTC
*** Bug 564636 has been marked as a duplicate of this bug. ***
Comment 58 André Klapper 2009-01-22 23:30:27 UTC
*** Bug 568740 has been marked as a duplicate of this bug. ***
Comment 59 André Klapper 2009-01-22 23:31:50 UTC
*** Bug 568004 has been marked as a duplicate of this bug. ***
Comment 60 André Klapper 2009-01-22 23:32:01 UTC
*** Bug 566543 has been marked as a duplicate of this bug. ***
Comment 61 André Klapper 2009-01-22 23:32:33 UTC
*** Bug 548973 has been marked as a duplicate of this bug. ***
Comment 62 André Klapper 2009-01-22 23:33:56 UTC
(In reply to comment #46)
> As I see its the reason 'Folder/Summary mismatch'. The reason why a folder
> summary mismatch could happen was fixed. [I'm sure it is fixed, since I have
> fixed it so many times]

When was this fixed? I expected it to be fixed in 2.22.3.1 and 2.23.5 but obviously that's wrong after getting all those 2.24 dups.
Comment 63 Srinivasa Ragavan 2009-01-23 03:55:33 UTC
Andre, the folder/summary mismatch was mostly seen on spool/* accounts and POP3. Im 200% sure that it won't happen in spool/unix-mailbox etc account. Are all these bugs from Pop3? Could be still from pop, my code review shows some loops, but I want to be sure.
Comment 64 Srinivasa Ragavan 2009-01-27 10:38:49 UTC
http://bugzilla.gnome.org/show_bug.cgi?id=548826 has a patch, which should fix it. As I said 2.22.3.1, you should never face it. It was a typo that re-introduced in 2.24.x with disk summary. I just approved. But I think I must watch for pop, since that still looks vulnerable for a corruption on a first look. I would analyse deep to be sure of it.

That patch would fix the expunge/trash work well for 90% cases. Rest 10% I'm seeing if there are any problems. I hope not.
Comment 65 Akhil Laddha 2009-01-29 13:27:43 UTC
*** Bug 569629 has been marked as a duplicate of this bug. ***
Comment 66 Srinivasa Ragavan 2009-01-29 16:28:13 UTC
Given that patch at #548826 is committed, this bug should be gone again. Please confirm it in 2.24.4 which would be out tomorrow. THanks. I would keep it open though.
Comment 67 André Klapper 2009-02-01 19:46:26 UTC
*** Bug 570112 has been marked as a duplicate of this bug. ***
Comment 68 André Klapper 2009-02-02 22:56:41 UTC
*** Bug 570274 has been marked as a duplicate of this bug. ***
Comment 69 tony 2009-02-02 23:24:22 UTC
(In reply to comment #48)
> I performed the following (as mentioned earlier in this bug report):
> 
> delete folders.db in .evolution/mail/accounts?
> find . -name folders.db | xargs rm $1 or something like this in that directory?
> 
> and restarted - each folder gave an error as I opened it - but after that,
> everything has gone back to "normal" - and I can expunge the trash without
> problems again.
> 
> There must be some sort of corruption which occurs in the folders.db somehow
> (over time)
> 
> Cheers,
> Robert
> 

Same thing worked for me too. I only deleted .evolution/mail/local/folders.db on Fedora. It took a while before the SearchFolders sorted themselves but all is working normally now.
Comment 70 B.J. McClure 2009-02-04 02:34:41 UTC
Deleting ~/.evolution/mail/local/folders.db also worked for me.
Comment 71 Akhil Laddha 2009-02-05 04:26:23 UTC
*** Bug 570523 has been marked as a duplicate of this bug. ***
Comment 72 Akhil Laddha 2009-02-06 11:13:14 UTC
I just got folder and summary mismatch error in 2.25.90 while trying to empty local trash.
Comment 73 André Klapper 2009-02-09 00:09:43 UTC
*** Bug 570926 has been marked as a duplicate of this bug. ***
Comment 74 Srinivasa Ragavan 2009-02-09 04:13:16 UTC
Looks like some more cases, open, which Im debugging atm. Will keep you all updated. Thanks.
Comment 75 André Klapper 2009-02-16 23:46:31 UTC
*** Bug 572039 has been marked as a duplicate of this bug. ***
Comment 76 Akhil Laddha 2009-02-20 03:46:36 UTC
*** Bug 572512 has been marked as a duplicate of this bug. ***
Comment 77 jsm 2009-02-22 19:43:34 UTC
With 2.24.4 I am getting this problem. I'm not sure when my system updated to this version, but the "Summary and Folder mismatch even after a sync" suddenly re-appeared in the last week or so. I had previously been plagued by it for a year or so, then it was fixed (hurrah!) but now it has come back.

I use a local unix mbox : /var/spool/mail/jsm and a POP3 account. The problem appears to be in the local file.

Previously I was able to tediously work around it by:
cp /var/spool/mail/jsm ~/mbox.safe.XXXX
cat /dev/null > /var/spool/mail/jsm
<start evolution>
cat ~/mbox.safe.XXXX >> /var/spool/mail/jsm

and it worked. However, with the current version of evolution that isn't working any more. The mailbox doesn't get re-indexed and it claims there are zero messages. I found that deleting all the files in
~/.evolution/mail/spool/var/spool/mail/jsm
will make it work somewhat.

I'm happy to provide any more debug information that may help resolve this problem.

James
Comment 78 Akhil Laddha 2009-02-24 11:36:01 UTC
*** Bug 568426 has been marked as a duplicate of this bug. ***
Comment 79 André Klapper 2009-02-26 19:11:25 UTC
*** Bug 573299 has been marked as a duplicate of this bug. ***
Comment 80 André Klapper 2009-02-26 19:11:51 UTC
2.24.5 is out.
Feedback highly welcome.
Comment 81 jsm 2009-03-03 11:11:38 UTC
I installed 2.24.5-1.fc10 and my mailbox opens normally without the mismatch errors.

I am able to open the print dialogue without a crash.

thanks
Comment 82 André Klapper 2009-03-11 13:46:20 UTC
I can still reproduce this in 2.24.5 (just happened to a folder that had no problems before).
Comment 83 André Klapper 2009-03-15 14:57:14 UTC
*** Bug 575399 has been marked as a duplicate of this bug. ***
Comment 84 Akhil Laddha 2009-04-13 05:54:11 UTC
*** Bug 578734 has been marked as a duplicate of this bug. ***
Comment 85 Akhil Laddha 2009-04-16 03:49:27 UTC
*** Bug 579102 has been marked as a duplicate of this bug. ***
Comment 86 André Klapper 2009-04-16 09:25:46 UTC
*** Bug 579067 has been marked as a duplicate of this bug. ***
Comment 87 Olivier Berger 2009-04-20 13:22:25 UTC
Should add that I experienced it too with 2.24.5 on Debian (http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=524363)
Comment 88 André Klapper 2009-04-21 22:12:37 UTC
*** Bug 579745 has been marked as a duplicate of this bug. ***
Comment 89 André Klapper 2009-04-21 22:12:44 UTC
*** Bug 579712 has been marked as a duplicate of this bug. ***
Comment 90 Dave Jewett 2009-04-22 02:53:19 UTC
I have read through this thread.  So far as I can tell, there seems to be no fix for this, so far.

As I am running Evolution 2.24.3, what actions do I need to take to correct this?
Comment 91 André Klapper 2009-04-25 10:41:35 UTC
*** Bug 580160 has been marked as a duplicate of this bug. ***
Comment 92 André Klapper 2009-05-19 22:21:09 UTC
*** Bug 583232 has been marked as a duplicate of this bug. ***
Comment 93 tremaine 2009-05-25 12:02:52 UTC
*** Bug 583553 has been marked as a duplicate of this bug. ***
Comment 94 André Klapper 2009-06-04 15:07:55 UTC
*** Bug 584749 has been marked as a duplicate of this bug. ***
Comment 95 H.Habighorst 2009-06-10 13:21:54 UTC
Created attachment 136272 [details]
EvoLog POP 3

Hi - I can reproduce this on 2.27 ( newest git ).

I have added an log where I cut out all the places where grep found the string "error". If you need more, I would save my local preferences, just start up with the 2 pop stores and then create an log ( have got 5 email accounts and the log file hits the 60000 border -.- Don't wanna filter out the private stuff ;) :-) )

I did the following : I've got two POP 3 Boxes, struggleutmost / hhevot, and I deleted randomly around 100 - 200 mails. Then expunged. Again message delete. Then expunge. Again and again till it's empty.

Evolution was started with CAMEL_DEBUG=all .
Comment 96 H.Habighorst 2009-06-10 13:22:27 UTC
Created attachment 136273 [details]
EvoLog POP 3

Hi - I can reproduce this on 2.27 ( newest git ).

I have added an log where I cut out all the places where grep found the string "error". If you need more, I would save my local preferences, just start up with the 2 pop stores and then create an log ( have got 5 email accounts and the log file hits the 60000 border -.- Don't wanna filter out the private stuff ;) :-) )

I did the following : I've got two POP 3 Boxes, struggleutmost / hhevot, and I deleted randomly around 100 - 200 mails. Then expunged. Again message delete. Then expunge. Again and again till it's empty.

Evolution was started with CAMEL_DEBUG=all .
Comment 97 Walt Armour 2009-07-02 18:19:08 UTC
Removing the folders.db has resolved this for me.

rm .evolution/mail/local/folders.db

However, I fully expect it to return again.  If it does I'll try to capture some logs with CAMEL_DEBUG=all .

More data points for anyone:

Fedora 11
Evolution 2.26.2
POP3
Comment 98 krzysztof pijarski 2009-07-03 06:58:04 UTC
removing folders.db deletes ALL your metadata. If you don't tag your messages at all this is no problem, but for most this won't be a solution!
Comment 99 André Klapper 2009-08-13 13:41:53 UTC
*** Bug 591568 has been marked as a duplicate of this bug. ***
Comment 100 Akhil Laddha 2009-08-27 07:37:26 UTC
*** Bug 584643 has been marked as a duplicate of this bug. ***
Comment 101 Akhil Laddha 2009-08-28 04:00:05 UTC
*** Bug 593254 has been marked as a duplicate of this bug. ***
Comment 102 Christoph Wickert 2009-08-31 17:49:24 UTC
Is anybody of you running filters on the affected mailboxes?

For me this is almost certain caused by by some filters, because...
1. Computer/Fedora/Commits was corrupted, so I removed 
~/.evolution/mail/local/Computer.sbd/Fedora.sbd/Commits.(cmeta,ibex.*)
--> error remains
2. Removed all cmeta and ibxe* files
--> error remains
3. Removed all mail from Computer/Fedora/Commits and empty trash 
--> error gone
4. Receive mail
--> still no error
5. Receive mail, this time a filter gets a applied to a mail and moves it into the offending folder
--> folder corrupted again

Every time I delete the content of the folder (no need to remove any metadata) everything is fine until I receive new mail from this particular mailing list. The offending folder has several sub-folders that contain mail from the same list but a few certain senders.

Attaching the relevant parts of filters.xml (rest removed for privacy reasons).
Comment 103 Christoph Wickert 2009-08-31 17:51:08 UTC
Created attachment 142147 [details]
filters.xml with the relavant rules described in comment #102
Comment 104 krzysztof pijarski 2009-08-31 18:12:09 UTC
I do have filters on that one mailbox that the error occurs in, yes. If if remove cmeta and ibex files, the error goes away for a short time only. I'll have a look if really reappers when evolution applies fiters.

(Just to clarify: nobody talked about the need to remove metadata - I was just saying that if you delete your folders.db (one of the fixes stated above), all your metadata will be lost; also removing your cmeta file makes you lose your tags, which made me stop using tags on that particular directory!)
Comment 105 André Klapper 2009-09-03 10:21:26 UTC
*** Bug 593994 has been marked as a duplicate of this bug. ***
Comment 106 krzysztof pijarski 2009-09-03 16:20:43 UTC
I have checked and realized that this error appeared only in my Inbox (I mean this particular folder named "Inbox"). I had filters for different accounts to be transferred to different folders anyway but have left one for the Inbox. So now I tried to transfer my whole Inbox to a folder and establish another filter for that last account. 
Guess what - everything works! Now I can expunge every folder, even my Inbox. However, during the transfer process I lost about 20-30 emails that were in a subfolder of the Inbox folder - ist it possible that it was those e-mails that were causing trouble?
Comment 107 Han Pilmeyer 2009-09-06 15:20:13 UTC
I'm on Ubuntu 9.04 / Evolution 2.26.1. I had this issue frequently in the passed, but I had not seen it for quite some time until this weekend. This time the problem was very persistent and deleting the folders.db files and the "meta" files for the offending folder did not help. In the end I had to delete the offending folder from the command line by copying /dev/null to it. I lost around 20000 mail messages this way, so I could have really used the tool that was mentioned in comment 46.

Oh, and I know for sure that the offending folder had no filters applied to it.
Comment 108 tony 2009-09-06 21:30:19 UTC
Just had it happen again on 2.24.5. I have lots of filters and search folders.
Once again deleting .evolution/mail/local/folders.db followed by stop / start Evolution fixed it. It worked better this time than last in that there was no delay in the search folders sorting themselves out.
Comment 109 Akhil Laddha 2009-09-08 11:58:16 UTC
*** Bug 594475 has been marked as a duplicate of this bug. ***
Comment 110 Akhil Laddha 2009-10-06 04:02:11 UTC
*** Bug 597501 has been marked as a duplicate of this bug. ***
Comment 111 benh 2009-10-06 05:10:42 UTC
Ok, I missed the duplicate, got mislead by the fact that I didn't experience
anything that looked like mailbox corruption.
Comment 112 benh 2009-10-06 05:19:23 UTC
Interesting. So all my folders seem to have a summary mismatch error in my log, and all my folders also fail to expunge. Do you think those two problems are related ?

Note that while I did move that db over from a previous evolution (.26 on ubuntu jaunty), I'm pretty sure I used the export/import feature to do so, does it make a difference ?

Is there some kind of "corruption" in some metadata that is creeping along ? In this case, is there a way to rebuild those metadata ? 

In January, Srinivasa Ragavan was talking about writing a utility to rewrite
the folder to fix it ?
Comment 113 Akhil Laddha 2009-10-08 06:56:39 UTC
*** Bug 544731 has been marked as a duplicate of this bug. ***
Comment 114 Akhil Laddha 2009-10-08 07:20:24 UTC
*** Bug 566732 has been marked as a duplicate of this bug. ***
Comment 115 Akhil Laddha 2009-10-12 04:16:53 UTC
*** Bug 598023 has been marked as a duplicate of this bug. ***
Comment 116 benh 2009-10-25 07:10:42 UTC
Hoy there ! Any news on that one ? 2.28.1 has the same problem. Should "expunge" be more robust and skip over the bad entries in the metadata (if that is indeed the problem) and still expunge -something- ? Or is the answer in that metadata-cleaning utility that Srinivasa Ragavan proposed back in January ?

My new SSD is filling up quickly, it would be nice to get rid of all those old email :-)

Anything I can do to help short of sending you the email files proper, just let me know.
Comment 117 Akhil Laddha 2009-10-28 04:09:28 UTC
*** Bug 599856 has been marked as a duplicate of this bug. ***
Comment 118 Akhil Laddha 2009-11-23 03:41:47 UTC
*** Bug 602683 has been marked as a duplicate of this bug. ***
Comment 119 Akhil Laddha 2009-11-24 10:24:58 UTC
*** Bug 602816 has been marked as a duplicate of this bug. ***
Comment 120 Akhil Laddha 2009-11-26 11:21:58 UTC
*** Bug 603023 has been marked as a duplicate of this bug. ***
Comment 121 Akhil Laddha 2009-11-26 11:57:49 UTC
Please read faq [1] to work around this problem 

[1] http://www.go-evolution.org/FAQ#Why_do_I_get_an_error_.22Summary_and_folder_mismatch.2C_even_after_a_sync.22.3F
Comment 122 Akhil Laddha 2009-12-04 12:40:44 UTC
*** Bug 558735 has been marked as a duplicate of this bug. ***
Comment 123 Akhil Laddha 2009-12-04 12:42:45 UTC
*** Bug 559911 has been marked as a duplicate of this bug. ***
Comment 124 Akhil Laddha 2009-12-09 09:34:03 UTC
*** Bug 604140 has been marked as a duplicate of this bug. ***
Comment 125 André Klapper 2009-12-10 18:42:57 UTC
*** Bug 604256 has been marked as a duplicate of this bug. ***
Comment 126 André Klapper 2009-12-10 22:14:22 UTC
*** Bug 603979 has been marked as a duplicate of this bug. ***
Comment 127 Akhil Laddha 2009-12-16 04:13:06 UTC
*** Bug 604654 has been marked as a duplicate of this bug. ***
Comment 128 Akhil Laddha 2009-12-29 10:08:46 UTC
*** Bug 564369 has been marked as a duplicate of this bug. ***
Comment 129 Akhil Laddha 2009-12-29 10:40:52 UTC
*** Bug 566554 has been marked as a duplicate of this bug. ***
Comment 130 Akhil Laddha 2009-12-29 11:06:24 UTC
*** Bug 572165 has been marked as a duplicate of this bug. ***
Comment 131 John Mellor 2010-01-05 03:26:05 UTC
This bug just bit me again in latest Fedora 12 release, after not seeing it for a year.  I assume that someone has re-introduced the defect in evolution-2.28.2-1.fc12.i686 somehow.

After an unrelated crash, my Inbox count of unread mails mismatched the visible number of messages.  In spite of having Evolution configured to immediately remove deleted messages, I was also unable to delete a couple of (but not all) messages, which it marked as deleted instead of moving them to Trash.

In order to perform an apparent recovery, I did:
1. evolution --force-shutdown
2. rm /.evolution/mail/local/folders.db
3. Start evolution and enter each folder.  It does a couple of seconds of work in each one before showing the contents again.

Interestingly, a half-dozen previously-hidden messages suddenly became visible in my Inbox after doing this, and I was able to delete them normally.

I've had this problem and minor variants of it hit me about a half-dozen times over the years.  In hindsight, putting a binary DB underneath mail seems like a poor decision that needs revisiting.  The related code seems to be excessively fragile.  We URGENTLY need a tool in the Evolution deliverables to verify correctness and to fix corrupt SQLite files and get Evolution back to sanity when this occurs.  In my opinion, since this is actively driving Evolution users to other solutions, this is the highest priority "feature" to add to Evolution at this point.  Even Lotus Notes has this capability, via the ZapNotes tool.
Comment 132 Akhil Laddha 2010-01-11 03:22:15 UTC
*** Bug 606600 has been marked as a duplicate of this bug. ***
Comment 133 Akhil Laddha 2010-01-11 03:59:44 UTC
*** Bug 606485 has been marked as a duplicate of this bug. ***
Comment 134 Akhil Laddha 2010-01-20 12:14:00 UTC
*** Bug 607541 has been marked as a duplicate of this bug. ***
Comment 135 Akhil Laddha 2010-02-04 10:00:51 UTC
*** Bug 608964 has been marked as a duplicate of this bug. ***
Comment 136 Akhil Laddha 2010-02-08 09:58:20 UTC
*** Bug 609295 has been marked as a duplicate of this bug. ***
Comment 137 Akhil Laddha 2010-02-08 11:18:08 UTC
*** Bug 609300 has been marked as a duplicate of this bug. ***
Comment 138 torbenfriis 2010-02-08 12:17:45 UTC
Re. Cant delete trash
I am utterly confused. Is there a fix for this?
Best regards
Torben
Comment 139 Akhil Laddha 2010-02-09 03:46:05 UTC
(In reply to comment #138)
> Re. Cant delete trash
> I am utterly confused. Is there a fix for this?

Did you try work around mentioned here
https://bugzilla.gnome.org/show_bug.cgi?id=550414#c121 ?
Comment 140 torbenfriis 2010-02-09 09:01:13 UTC
Hi Adhil Laddha,
Thanks. On the advice of a guy called Martin I used the advice in
comment 131 with succes.
regards
torben
Comment 141 Akhil Laddha 2010-02-09 12:45:01 UTC
*** Bug 609412 has been marked as a duplicate of this bug. ***
Comment 142 Gilboa Davara 2010-02-10 06:51:06 UTC
Any chance of resolving this issue?
Or at least, any advise on how to help you debug this issue?

I'm being hammered by the "mismatch" bug on a weekly basis now. (Forcing me to delete the ibex files)

- Gilboa
Comment 143 André Klapper 2010-02-26 09:29:35 UTC
*** Bug 611094 has been marked as a duplicate of this bug. ***
Comment 144 Akhil Laddha 2010-03-01 03:49:29 UTC
*** Bug 611338 has been marked as a duplicate of this bug. ***
Comment 145 storm66 2010-03-15 10:12:47 UTC
Hello,

The solution with the delete of all "folders.db" files works for me, I can now expunge the bin without any error message.

Regards
Comment 146 Akhil Laddha 2010-03-21 07:05:19 UTC
*** Bug 613391 has been marked as a duplicate of this bug. ***
Comment 147 Akhil Laddha 2010-03-22 03:54:49 UTC
*** Bug 613390 has been marked as a duplicate of this bug. ***
Comment 148 Matthew Barnes 2010-03-24 10:38:22 UTC
*** Bug 601717 has been marked as a duplicate of this bug. ***
Comment 149 gordon 2010-03-25 09:36:25 UTC
I ran this script on my mailbox & since then everything has run perfectly.
#!/bin/bash
export LANG=C
set -e
evolution --force-shutdown
tar -zcf dot_evolution_$$.tgz ~/.evolution/
for each in index data cmeta; do
 find ~/.evolution/mail/local -type f -iname "*.$each" | xargs rm
done
rm ~/.evolution/mail/local/folders.db

(In reply to comment #0)
> Please describe the problem:
> Since latest releases of Evolution (evolution-2.23.91)- I get corrupted
> mailboxes and can't expunge the trash folder - constantly getting errors like:
> "Summary and
> folder mismatch, even after a sync"
> 
> Steps to reproduce:
> 1. Open Evolution
> 2. try and expunge Trash folder
> 3. get lots of errors
> 
> Actual results:
> My inbox has many duplicates and grows by itself - when trying to clean it it
> gets more screwed up.  When I try and expunge the Trash folder, it doesn't work
> - I get several errors "Summary and
> folder mismatch, even after a sync"
> 
> Expected results:
> When Trash is expunged, the contents of trash disappear - cleaned out.  Inbox
> also should stay stable (seems corrupted)
> 
> Does this happen every time?
> YES
> 
> Other information:
> I am a long time Evolution user (many years) and have never experienced such
> behavior, and I have lost some mails due to these problems.  Not Good.  This
> has been since the 2.23 series
Comment 150 gordon 2010-03-25 09:41:08 UTC
I ran this script below & since then (1month ago) I have had no problems & everything has worked perfectly.

#!/bin/bash
export LANG=C
set -e
evolution --force-shutdown
tar -zcf dot_evolution_$$.tgz ~/.evolution/
for each in index data cmeta; do
 find ~/.evolution/mail/local -type f -iname "*.$each" | xargs rm
done
rm ~/.evolution/mail/local/folders.db
Comment 151 Akhil Laddha 2010-03-29 05:08:12 UTC
*** Bug 608992 has been marked as a duplicate of this bug. ***
Comment 152 Fabio Durán Verdugo 2010-03-29 13:53:23 UTC
*** Bug 614251 has been marked as a duplicate of this bug. ***
Comment 153 Fabio Durán Verdugo 2010-04-05 18:53:40 UTC
*** Bug 614887 has been marked as a duplicate of this bug. ***
Comment 154 Thomas 2010-04-09 01:04:36 UTC
(In reply to comment #150)
> I ran this script below & since then (1month ago) I have had no problems &
> everything has worked perfectly.
> 
> #!/bin/bash
> export LANG=C
> set -e
> evolution --force-shutdown
> tar -zcf dot_evolution_$$.tgz ~/.evolution/
> for each in index data cmeta; do
>  find ~/.evolution/mail/local -type f -iname "*.$each" | xargs rm
> done
> rm ~/.evolution/mail/local/folders.db

Just had the error as well with version 2.28.4. Doing the above, loses the
labels, of course.
So, as an alternative, I copied the folder to a new location, made sure
that the expunge worked on the copied folder, removed/renamed the original one,
and moved the copy to the original place.
Should work for reasonably sized local mailboxes.
Comment 155 Thomas 2010-04-09 01:11:32 UTC
(In reply to comment #154)
> (In reply to comment #150)
>
> So, as an alternative, I copied the folder to a new location, made sure
> that the expunge worked on the copied folder, 

Did the copy with in the application via menu or popup context menu, of course!!
Not via file system copy.
Comment 156 André Klapper 2010-04-11 19:20:27 UTC
*** Bug 615414 has been marked as a duplicate of this bug. ***
Comment 157 benh 2010-04-27 01:39:01 UTC
Note that the corruption still happens with 2.28.1. ie. I did the whole metadata removal thing a few month ago, managed to expunge a whole bunch of crap I had there, and forgot about it until I tried to expunge again today and -all- of my folders fail to expunge with what looks like the same old error
Comment 158 Gilboa Davara 2010-04-27 06:31:56 UTC
Count your self lucky.

I've got a script that deletes the metadata and I'm forced to use once every 2-3 days.

- Gilboa
Comment 159 Akhil Laddha 2010-05-03 03:44:19 UTC
*** Bug 617467 has been marked as a duplicate of this bug. ***
Comment 160 Akhil Laddha 2010-05-08 12:17:35 UTC
*** Bug 618092 has been marked as a duplicate of this bug. ***
Comment 161 bwallum 2010-05-10 09:40:26 UTC
This bug is killing my email. It appears to randomly deny access to email folders and then when I re-enter Evolution the folder is no longer shown.

I cannot 'expunge' the Deleted Items (trash) folder. I get the previously reported 

Error storing '~/.evolution/mail/local/Sent (mbox)': Summary and folder mismatch, even after a sync

type of error. Each time I shut down and reload Evolution, then try to expunge I get a different set of folder names reported in the error message.

This is a major regression.
Comment 162 Thomas 2010-05-11 01:00:01 UTC
(In reply to comment #161)
> This bug is killing my email. It appears to randomly deny access to email
> folders and then when I re-enter Evolution the folder is no longer shown.
> 
> I cannot 'expunge' the Deleted Items (trash) folder. I get the previously
> reported 
> 
> Error storing '~/.evolution/mail/local/Sent (mbox)': Summary and folder
> mismatch, even after a sync
> 
> type of error. Each time I shut down and reload Evolution, then try to expunge
> I get a different set of folder names reported in the error message.
> 
> This is a major regression.

Have you tried the method, I suggested in comment 154 and comment 155?
That worked for me to also preserve the labels.
Comment 163 bwallum 2010-05-21 08:56:03 UTC
I deleted .evolution/mail/local/folders.db and upon restarting Evolution I found it took some extra time to load (regenerating an index?). Once loaded all my folders and mail were intact and the Deleted Items folder was empty.

This fix should be a 'janitor' option within Evolution to allow a corrupt folders.db file to be regenerated. It wasted a fair bit of time for me and by the look of it also wasted a lot of other's time.

Thanks Thomas! You saved me wasting more time.
Comment 164 Milan Crha 2010-05-26 19:36:59 UTC
Created attachment 162044 [details] [review]
proposed eds patch

for evolution-data-server;

It seems that this is caused when two threads are working with one folder summary, making a mismatch as a result. I only guess this, when I realized that the sync function is called when the summary is not locked, thus any other thread can touch it meanwhile it is under change). Maybe more locking will be needed, I do not know yet. This patch will not fix already broken summaries, but should guard it to not happen again. I'm not able to reproduce this myself, unfortunately.

I would like to ask anyone of you luckier with reproducing to give this patch a try, though it has one issue, the patch is for actual master (~2.31.2), because of other related code changes, like making accessible CamelFolderSummary locks from outside, and making them recursive (note that rewriting this for 2.30.x would need also some changes in evolution-mapi, due to its camel-private.h, which is gone on actual git master).

The best when you try with a fresh new summary (no folders.db, .ibex* or .cmeta files in ~/.evolution/mail/local/ - see more detailed descriptions above or in Evolution's FAQ page), and just use your Evolution as before.

As the patch would be otherwise harmless, on the first look, then maybe it'll be committed just before 2.31.3, if there will not be found any negative side effects of it. But no promises.
Comment 165 Thomas 2010-05-26 20:49:03 UTC
(In reply to comment #164)
> Created an attachment (id=162044) [details] [review]
> proposed eds patch
> 
> for evolution-data-server;
> 
> It seems that this is caused when two threads are working with one folder
> summary, making a mismatch as a result. I only guess this, when I realized that
> the sync function is called when the summary is not locked, thus any other
> thread can touch it meanwhile it is under change). Maybe more locking will be
> needed, I do not know yet. This patch will not fix already broken summaries,
> but should guard it to not happen again. I'm not able to reproduce this myself,
> unfortunately.
> 
> I would like to ask anyone of you luckier with reproducing to give this patch a
> try, though it has one issue, the patch is for actual master (~2.31.2), because
> of other related code changes, like making accessible CamelFolderSummary locks
> from outside, and making them recursive (note that rewriting this for 2.30.x
> would need also some changes in evolution-mapi, due to its camel-private.h,
> which is gone on actual git master).
> 

I just tried to quickly adapt your patch to 2.28. Did not build cleanly, of course. Can you give me some short hints about what I do have to do, to adopt
this to 2.28? Is there a way to do it via git merge?

thomas
Comment 166 André Klapper 2010-05-26 21:13:09 UTC
(Please avoid quoting the complete former comment.)

Manually edit the file(s) this patch covers and insert the changes.
Comment 167 Thomas 2010-05-27 02:08:21 UTC
Created attachment 162066 [details] [review]
patch for gnome-2-28

My adaptation of patch https://bugzilla.gnome.org/attachment.cgi?id=162044, which is for git master, to gnome-2-28.
Comment 168 Thomas 2010-05-27 02:34:23 UTC
Review of attachment 162066 [details] [review]:

Sorry, did not work. Guess, I need to adapt those recursive locks in some way.
Comment 169 Milan Crha 2010-05-27 10:13:59 UTC
Yup, the patch is using new API in camel. I'll attach a patch for 2.30, and you can then backport it to your ancient 2.28, as there is not much difference between 2.30 and 2.28 in this area, together with a mapi patch.
Comment 170 Milan Crha 2010-05-27 12:26:22 UTC
Created attachment 162092 [details] [review]
proposed eds patch (for 2.30)

for evolution-data-server 2.30.x;

When applying to 2.28.x, then you shouldn't need chunks involving 'frompos' changes, those are also specific to 2.30, and adding them to 2.28 may break things, so skip them, please. Namely:
 - camel/providers/local/camel-mbox-summary.c - first three chunks
 - camel/providers/local/camel-mbox-summary.h - whole

With respect of evolution-mapi, with this patch applied also part in
camel/camel-private.h should be applied to evolution-mapi as well. The file is located in src/camel/camel-private.h there.
Comment 171 Thomas 2010-05-27 17:12:04 UTC
Created attachment 162124 [details] [review]
patch for gnome-2-28

Next try for bug 550414. This time with patch
    https://bugzilla.gnome.org/attachment.cgi?id=162092 for evolution
    2.30, kindly provided by Milan Crha. See also: comment 170 in the
    bug.

Expunging a single folder worked without problems. So far, so good.
Comment 172 Thomas 2010-05-27 19:49:57 UTC
Created attachment 162143 [details] [review]
patch for gnome-2-28 (corrected

Supersedes the earlier one.
Comment 173 Milan Crha 2010-05-28 09:11:05 UTC
Review of attachment 162143 [details] [review]:

Thanks, patch looks good for 2.28, but as there is not planned any update for it then I'm just marking this patch as reviewed, to not confuse other people. Anyone using 2.28 can take it and apply to evolution-data-server (and as mentioned earlier also to evolution-mapi, the part for camel-private.h).
Comment 174 Hussam Al-Tayeb 2010-06-02 23:09:49 UTC
Just a note: 
I had this problem tonight for the first time and patch for gnome-2-30 branch https://bugzilla.gnome.org/attachment.cgi?id=162092 fixes it.
I would recommend checking it into git. I will report any regressions if I stumble onto any.
Comment 175 Milan Crha 2010-06-03 08:33:10 UTC
Thanks for the update. Thomas, how about on your side, please? Any other testers involved too?
Comment 176 Thomas 2010-06-03 19:47:07 UTC
(In reply to comment #175)
> Thanks for the update. Thomas, how about on your side, please? 

I generally use expunge only on individual folders and this works
cleanly for some time now on gnome-2-28 and also with the patch
applied. 

> Any other
> testers involved too?

Not that I know off.
Comment 177 Thomas 2010-06-03 19:47:20 UTC
(In reply to comment #175)
> Thanks for the update. Thomas, how about on your side, please? 

I generally use expunge only on individual folders and this works
cleanly for some time now on gnome-2-28 and also with the patch
applied. 

> Any other
> testers involved too?

Not that I know of.
Comment 178 Milan Crha 2010-06-07 08:26:23 UTC
OK, thanks for the update.

Created commit 9da96c2 in eds master (2.31.3+)
Created commit 354aa2e in eds gnome-2-30 (2.30.2+)
Created commit 2aeb3cd in ema gnome-2-30 (0.30.2+)
Comment 179 Gilboa Davara 2010-06-29 17:16:18 UTC
Been testing a fixed F13 build for a couple of weeks now.
Bug fully resolved.

Thanks!!!
Comment 180 Akhil Laddha 2010-07-10 04:26:58 UTC
*** Bug 614871 has been marked as a duplicate of this bug. ***
Comment 181 Akhil Laddha 2010-07-11 07:35:36 UTC
*** Bug 624059 has been marked as a duplicate of this bug. ***
Comment 182 André Klapper 2010-07-18 09:07:54 UTC
*** Bug 624627 has been marked as a duplicate of this bug. ***
Comment 183 Akhil Laddha 2010-07-21 03:57:35 UTC
*** Bug 624881 has been marked as a duplicate of this bug. ***
Comment 184 Akhil Laddha 2010-07-28 05:43:53 UTC
*** Bug 625429 has been marked as a duplicate of this bug. ***
Comment 185 Akhil Laddha 2010-08-06 06:37:05 UTC
*** Bug 626097 has been marked as a duplicate of this bug. ***
Comment 186 André Klapper 2010-08-10 14:46:49 UTC
*** Bug 626295 has been marked as a duplicate of this bug. ***
Comment 187 bwallum 2010-08-10 15:47:16 UTC
This problem is a regression. It may be repeated by creating a new mail folder, lets say Inbox>Test then dragging another Inbox mail folder to it. If a message filter directed mail to the moved folder then the moved items appear in the Deleted Items folder and won't delete.

The index gets screwed up. 

Deleting the folders.db file in /home/user/.evolution folder fixes the problem. Allow time for a new index to regerate when you next start Evolution.

...and it's back in 2.28.3 having been ok for a while.
Comment 188 André Klapper 2010-08-10 15:49:16 UTC
bwallum: It is fixed in 2.30 and later. See comment 178. It's not fixed in 2.28 as 2.28 is old and not maintained anymore, but feel free to ask your distribution to backport it.
Comment 189 Akhil Laddha 2010-08-14 12:43:01 UTC
*** Bug 626889 has been marked as a duplicate of this bug. ***
Comment 190 Christoph Anton Mitterer 2010-08-14 17:43:53 UTC
I still severely suffer from this in 2.30 (having all indexes/etc deleted as described in 626889)
Comment 191 Russ Mannex 2010-08-14 17:50:22 UTC
Christoph, are you using 2.30.2? I backed up my mail, upgraded to that version, then renamed the ~/.evolution/mail/local/folders.db file, and that seemed to fix the problem. Of course, this should all be done with Evolution closed.
Comment 192 Akhil Laddha 2010-08-16 00:08:42 UTC
*** Bug 626973 has been marked as a duplicate of this bug. ***
Comment 193 Christoph Anton Mitterer 2010-08-16 19:24:20 UTC
Russ, I'm sure that I did this procedure (deleting all meta data as I've described in #626889) in one of the 2.30.x versions,... but it might have been also .0 or .1.


If that bug was really fixed in 2.30.2 I could do it again (which is really a pain in the arse for me, see https://bugzilla.gnome.org/show_bug.cgi?id=626889#c5).

But I had the impression that this should have been fixed before 2.30?

And also that really a corruption in the mboxes themselves had been introduced, which would probably not be fixed by just recreating the metadata.
And even if you've added code to "work around" such corruptions, any other MUAs (or tools like mailarchive) would be still likely to fail over such corruptions, right?
So I guess it would be better to _really_ clean such corruptions, should they really exist.

Cheers,
Chris.
Comment 194 Christoph Anton Mitterer 2010-08-16 19:26:28 UTC
(btw: just the number of duplicates which still came up here show, that there has to be still something done I guess... ;) )
Comment 195 Russ Mannex 2010-08-16 19:36:00 UTC
I would agree, though for me the problem seemed to be that only the meta data had been corrupted. Thus, forcing Evolution to recreate that file fixed it. I am not sure which version it was fixed in, but the procedure I outlined worked in my case. Unfortunately, I did not try it before 2.30.2. Hopefully, only your meta data is corrupted, and not your actual mail.

I know it's a pain. I have a huge amount of mail as well. It was quite a relief to be able to finally delete things, though :).
Comment 196 Christoph Anton Mitterer 2010-08-16 19:39:11 UTC
On Mon, 2010-08-16 at 19:36 +0000, Evolution wrote:
I know it's a pain. I have a huge amount of mail as well. It was quite a relief
> to be able to finally delete things, though :).
> Well perhaps evolution helps me with that ;)
Comment 197 Fabio Durán Verdugo 2010-08-18 23:14:40 UTC
*** Bug 627316 has been marked as a duplicate of this bug. ***
Comment 198 Akhil Laddha 2010-08-20 03:38:56 UTC
*** Bug 627414 has been marked as a duplicate of this bug. ***
Comment 199 Akhil Laddha 2010-08-21 06:12:34 UTC
*** Bug 627560 has been marked as a duplicate of this bug. ***
Comment 200 Fabio Durán Verdugo 2010-08-21 21:34:02 UTC
*** Bug 627593 has been marked as a duplicate of this bug. ***
Comment 201 Akhil Laddha 2010-08-23 09:27:01 UTC
*** Bug 627697 has been marked as a duplicate of this bug. ***
Comment 202 Akhil Laddha 2010-09-01 10:04:36 UTC
*** Bug 628475 has been marked as a duplicate of this bug. ***
Comment 203 André Klapper 2010-09-02 10:59:20 UTC
*** Bug 628592 has been marked as a duplicate of this bug. ***
Comment 204 Akhil Laddha 2010-09-23 10:32:45 UTC
*** Bug 630389 has been marked as a duplicate of this bug. ***
Comment 205 Akhil Laddha 2010-09-23 10:32:51 UTC
*** Bug 630388 has been marked as a duplicate of this bug. ***
Comment 206 Akhil Laddha 2010-09-28 04:16:14 UTC
*** Bug 630772 has been marked as a duplicate of this bug. ***
Comment 207 André Klapper 2010-10-09 19:13:27 UTC
*** Bug 631741 has been marked as a duplicate of this bug. ***
Comment 208 Christoph Wickert 2010-10-22 08:05:24 UTC
Can somebody please reopen this bug? It was fixed in 2.30.something, but it's back after I updated to evolution-2.32.0-2.fc14.x86_64
Comment 209 Milan Crha 2010-10-22 11:19:20 UTC
(In reply to comment #208)
> Can somebody please reopen this bug? It was fixed in 2.30.something, but it's
> back after I updated to evolution-2.32.0-2.fc14.x86_64

It would be quite surprising seeing this under 2.32.0, unless the folder summary being broken (just?) before the update, though even there, if under 2.30.2+ it may not happen. I just checked and I see the changes with locking in the evolution-data-server 2.32.0.
Comment 210 Michael Sanders 2010-10-22 12:03:10 UTC
I am using Evolution 2.28.3 and have had this issue for several versions.  To fix, I use the script listed in previous comment to clean messages.  Otherwise, my trash folder would be full of orphaned messages.  I have never had this problem with Thunderbird or other email programs.  Please address this, because it clouds an otherwise good application.  I am actively seeking a suitable replacement as a backup.
Comment 211 André Klapper 2010-10-22 12:20:19 UTC
(In reply to comment #210)
> I am using Evolution 2.28.3

Of course this bug happens in ancient versions. See the version field of this report. No need for confirmations...
Comment 212 Thomas 2010-10-22 20:08:29 UTC
(In reply to comment #210)
> I am using Evolution 2.28.3 and have had this issue for several versions.  To
> fix, I use the script listed in previous comment to clean messages.  

Have you tried my adaptation for 2.28.xx of Milan's patch, https://bugzilla.gnome.org/attachment.cgi?id=162143? See comment 170 and 172.
Comment 213 Akhil Laddha 2010-11-17 06:04:34 UTC
*** Bug 635046 has been marked as a duplicate of this bug. ***
Comment 214 karlrt 2010-12-03 12:17:59 UTC
Seeing all the duplicates coming in, this bug is clearly not fixed! Please can someone reopen it?

I ran into the same problem with 2.30.3-1ubuntu7.1 a fresh install and a fresh configuration. It downloaded a lot of files from my pop account, as i keep them there for backup, trying to delete gives the same error as described many times.

Also, removing all index files and folder.db while evolution shut-down doesnt resolve the issue for me. I also tried renaming the inbox file but it stayed the same!
Comment 215 André Klapper 2010-12-04 01:06:17 UTC
*** Bug 636386 has been marked as a duplicate of this bug. ***
Comment 216 Akhil Laddha 2010-12-07 08:45:23 UTC
*** Bug 635622 has been marked as a duplicate of this bug. ***
Comment 217 Akhil Laddha 2010-12-24 11:56:54 UTC
*** Bug 637928 has been marked as a duplicate of this bug. ***
Comment 218 Akhil Laddha 2011-01-12 14:26:54 UTC
*** Bug 639297 has been marked as a duplicate of this bug. ***
Comment 219 Akhil Laddha 2011-01-17 04:45:53 UTC
*** Bug 630605 has been marked as a duplicate of this bug. ***
Comment 220 Christoph Wickert 2011-02-10 21:44:48 UTC
Believe it or not, I am still seeing this bug in 2.32.1. evolution-2.32.1-1.fc14.x86_64 to be precise. This is definitely NOT fixed.
Comment 221 Akhil Laddha 2011-02-16 04:46:48 UTC
*** Bug 642430 has been marked as a duplicate of this bug. ***
Comment 222 Akhil Laddha 2011-02-22 03:33:29 UTC
*** Bug 642924 has been marked as a duplicate of this bug. ***
Comment 223 Christoph Wickert 2011-02-25 18:24:29 UTC
Created attachment 181936 [details]
Screenot of the error message in 2.32.1

Now will you please believe me that this bug still occurs in evolution 2.32.1 and kindly reopen this bug?
Comment 224 karlrt 2011-02-26 14:41:07 UTC
+1
Comment 225 Akhil Laddha 2011-03-21 03:37:21 UTC
*** Bug 645319 has been marked as a duplicate of this bug. ***
Comment 226 Akhil Laddha 2011-03-23 16:34:52 UTC
*** Bug 645561 has been marked as a duplicate of this bug. ***
Comment 227 Akhil Laddha 2011-04-13 03:58:40 UTC
*** Bug 647593 has been marked as a duplicate of this bug. ***
Comment 228 André Klapper 2011-04-28 11:09:08 UTC
*** Bug 648843 has been marked as a duplicate of this bug. ***
Comment 229 Christoph Anton Mitterer 2011-04-28 21:49:41 UTC
Btw: I can confirm that this bug MUST still somehow exist.

I started from scratch with a 2.32 installation of evolution.
Removed ALL of evolutions index files, databases, etc.. Just the whole evolution directory.

Then I reimported (which took me about 8 hours) all my emails (mbox files), and in the meantime I can see all the symptoms of this issue again.

Starting from "error while storing folder", "mismatch even after sync", etc. etc.

Cheers,
Chris.
Comment 230 Christoph Wickert 2011-04-28 22:15:08 UTC
What folder? For me it is always and only the Sent folder. Evolution 2.32.2 btw.
Comment 231 Christoph Anton Mitterer 2011-04-28 22:54:11 UTC
It definitely happened/s with Sent, but I think I can remember that I've already seen it with others, too.

I haven't followed it closely anymore recently, as this bug seems to be largely ignored/denied anyway, but I'll watch out again.
Comment 232 Christoph Anton Mitterer 2011-04-28 22:55:58 UTC
Oh and also the issue, that folders are printed bold (as having unread mail), although they haven't, appears again.

e.g. I do a "Mark all mails read", but it's still displayed bold,... this goes away then when I open all the (sub)folders where evolution pretends that emails would be unread.
Comment 233 Christoph Wickert 2011-04-28 23:05:56 UTC
(In reply to comment #232)
> Oh and also the issue, that folders are printed bold (as having unread mail),
> although they haven't, appears again.

This is bug 577542 which is supposed to be fixed in 2.91.0.
Comment 234 Christoph Anton Mitterer 2011-04-28 23:26:42 UTC
Ah, I see,... thought these were / could be related.

Nevertheless, the corruption issue remains.

Actually (and in all doing respect) I think that there might be general problem with the way how meta-data/indexing and all that is handled (though I must admit that I've never found the time to look at the code so far):

- It seems to happen quite easy, that the meta-data becomes inconsistent (and crashes simply do happen, even if Evolution itself would never crash).

- The whole thing seems to be pretty slow. I really have many emails, in total (~26GiB currently) and per folder (take mailing lists like lkml, or commit lists from big projects like gnome or some BSDs).
When for some reason the indexes are recreated or I get into the "storing folder" thingy,... then this can easily take up to an hour per folder (for the biggest ones). And the funny thing is, that when I look at iotop, evolution basically reads only a few KB/s. I'd expect it to read/parse the mbox at full speed.

Not sure if I'm to optimistic that this could be made faster in general (independently of this bug),...
Comment 235 Milan Crha 2011-04-29 05:19:03 UTC
(In reply to comment #234)
> - It seems to happen quite easy, that the meta-data becomes inconsistent (and
> crashes simply do happen, even if Evolution itself would never crash).

Being it easy to reproduce on all machines then it would be fixed (properly) already. Reproducibility is usually main point in fixing bugs. At least for me.

> - The whole thing seems to be pretty slow. I really have many emails, in total
> (~26GiB currently) and per folder (take mailing lists like lkml, or commit
> lists from big projects like gnome or some BSDs).
> When for some reason the indexes are recreated or I get into the "storing
> folder" thingy,... then this can easily take up to an hour per folder (for the
> biggest ones). And the funny thing is, that when I look at iotop, evolution
> basically reads only a few KB/s. I'd expect it to read/parse the mbox at full
> speed.

This might be a clue. The initial patch added some locking to common operations to prevent parallel access to indexes, so it's not written corrupted. If there was some place overlooked, then it may mean you found (and face) it. Might be a good time to revisit the bug.

> Not sure if I'm to optimistic that this could be made faster in general
> (independently of this bug),...

The 3.0.0+ is using maildir instead of mbox as its primary storage, which doesn't suffer of this "rewrite whole 2GB mbox file on change", though it might have other issues with large folders. No clues here, though.
Comment 236 Milan Crha 2011-04-29 08:00:25 UTC
Created attachment 186857 [details] [review]
proposed eds patch ][

for evolution-data-server;

This patch doesn't do anything inventive, it only locks more aggressively, and in more functions (I missed some functions from locking in the previous patch, and this one locks much earlier, before opening underlying mbox file - all these things together might exhibit the issue, even less often for some people).

It is for git master, but I can adapt it to 2.32.2 too, and even build a test package for Fedora 14, if there will be someone being able to reproduce this reliably and willing to give it a try. Please let me know if I might do it. Thanks in advance.
Comment 237 Olivier Berger 2011-05-01 17:34:21 UTC
FYI, the problem still exists for me in Debian testing version (2.32.2), as reported in :
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=524363#74

I guess the patch against 2.32.2 could indeed be helpful, even though I can't promise I'll be able to rebuild a patched package and test.

Thanks in advance.
Comment 238 Milan Crha 2011-05-02 06:58:41 UTC
Created attachment 187010 [details] [review]
eds patch ][ for 2.32.2

for evolution-data-server 2.32.2;

This is the same patch as the previous one, only adapted to 2.32.2 eds sources. Here [1] is a scratch build for Fedora 14 with this patch included (it will disappear in a week or two). Any testing from users experiencing this issue appreciated.

[1] http://koji.fedoraproject.org/koji/taskinfo?taskID=3043942
Comment 239 Olivier Berger 2011-05-02 09:11:15 UTC
(In reply to comment #238)
> Created an attachment (id=187010) [details] [review]
> eds patch ][ for 2.32.2
> 
> for evolution-data-server 2.32.2;
> 
> This is the same patch as the previous one, only adapted to 2.32.2 eds sources.
> Here [1] is a scratch build for Fedora 14 with this patch included (it will
> disappear in a week or two). Any testing from users experiencing this issue
> appreciated.
> 
> [1] http://koji.fedoraproject.org/koji/taskinfo?taskID=3043942

I've rebuilt a Debian package including this patch, and it seems I can no longer see the trash expunging problems.

Looks like it works, at least for me.

Hope this helps.
Comment 240 Olivier Berger 2011-05-03 13:59:11 UTC
(In reply to comment #239)

> I've rebuilt a Debian package including this patch, and it seems I can no
> longer see the trash expunging problems.
> 
> Looks like it works, at least for me.
> 
> Hope this helps.

Well... actually, it seems to have improved, but I noticed corrupted/non expungeable mail again :-( There are probably other problems.
Comment 241 Milan Crha 2011-05-04 14:55:58 UTC
(In reply to comment #240)
> Well... actually, it seems to have improved, but I noticed corrupted/non
> expungeable mail again :-( There are probably other problems.

Is it possible it was kept from the previous run? The patch isn't fixing already broken state, it only tries to avoid such situation in the future, thus if it was broken already, then only using newer binary will not solve the issue.
Comment 242 Olivier Berger 2011-05-04 17:22:35 UTC
(In reply to comment #241)

> 
> Is it possible it was kept from the previous run? The patch isn't fixing
> already broken state, it only tries to avoid such situation in the future, thus
> if it was broken already, then only using newer binary will not solve the
> issue.

That's very much likely, yes, but in between trials to reproduce, I erase all index files and recreate them, with something like :

EVOLUTION_STORAGE=~/.local/share/evolution/mail
RM_COMMAND="rm -f"
LANG=C evolution --force-shutdown
find $EVOLUTION_STORAGE/local -name '*ev-summary*' -exec $RM_COMMAND {} \;
find $EVOLUTION_STORAGE/local -name '*.ibex*' -exec $RM_COMMAND {} \;
find $EVOLUTION_STORAGE/local -name '*.cmeta*' -exec $RM_COMMAND {} \;
find $EVOLUTION_STORAGE/local -name '*index.data' -exec $RM_COMMAND {} \;
find $EVOLUTION_STORAGE/local -name '*index' -exec $RM_COMMAND {} \;

cd $EVOLUTION_STORAGE/
for i in `find . -name folders.db`
        do
        echo "Rebuilding Table $i"
        sqlite3 $i "vacuum;"
done

I've now added a 'formail -ds' cleanup of all the mailboxes too, but there are still not expungeable messages... There may be other breakages inside these mboxes than what formail -ds can fix.

I definitely believe the patch brought some improvements though, as the trash now contains much less un-expungable messages.
Comment 243 Christoph Anton Mitterer 2011-05-04 17:46:24 UTC
Milan,
Your right the reproducibility is an important point, but for me this is quite difficult, as (as I've already said) it takes many hours until Evolution has recreated all index files.
And I basically have to attend this and do it one by one (via clicking at a folder while being in offline mode); if I let it recreate them automatically (when needed) it always used to go just crazy and not work at all.
Unfortunately there's still nothing like what I've asked for in #626890.

Also:
Not sure whether maildir is really the solution for everything. I mean especially if one has very large folders with many 10.000s of mails, you may sooner or later get problems on your filesystem (well at least on some) and you also waste a lot of space.

This should be either configurable per folder AND (as a real solution) something like #627220 should be implemented.
That way one could have the archived folders mbox-ed, and the "current" folders maildir-ed.


Regarding your patch:
As Oliver has probably found that it is not enough, I won't test it now (given the extreme effort this takes for me); maybe when a new iteration of it is out.
Comment 244 Christoph Anton Mitterer 2011-05-04 17:48:30 UTC
Oliver has implied that there is indeed some corruption within the mbox itself, which would actually be a catastrophe.

Is this investigated yet? Any definite steps to save as much as possible?
Does formail really correctly split the mails within the mbox, or could I end up in even more currupted (or concatenated) mails?
Comment 245 Olivier Berger 2011-05-04 17:59:36 UTC
(In reply to comment #244)

> Is this investigated yet? Any definite steps to save as much as possible?
> Does formail really correctly split the mails within the mbox, or could I end
> up in even more currupted (or concatenated) mails?

Being text files, the mboxes can be compared (diff) before and after processing them with formail. Man formail for more details.
Comment 246 Christoph Anton Mitterer 2011-05-04 18:07:49 UTC
Well... as previously mentioned... ~32GB of mail... ;)
Comment 247 Christoph Wickert 2011-05-04 18:09:44 UTC
(In reply to comment #244)
> Oliver has implied that there is indeed some corruption within the mbox itself,
> which would actually be a catastrophe.
> 
> Is this investigated yet? 

Yes, read this bug report.

> Any definite steps to save as much as possible?

No mail is getting lost, just the meta information like flags.
Comment 248 Milan Crha 2011-05-05 07:07:43 UTC
(In reply to comment #242)
> cd $EVOLUTION_STORAGE/
> for i in `find . -name folders.db`
>         do
>         echo "Rebuilding Table $i"
>         sqlite3 $i "vacuum;"
> done

The thing is that the folders.db contains the actual cached data, like "pointers" into the mbox file where each message ought to begin, and this error message, about which is this bug report, is just about the value in folders.db file not in sync with actual mbox file. Vacuum is for something else. Just move away the folders.db file and let evolution recreate it based on the actual content of your mbox file(s). See comment #1 and comment #3.
Comment 249 Olivier Berger 2011-05-05 10:28:53 UTC
(In reply to comment #248)
> (In reply to comment #242)
> > cd $EVOLUTION_STORAGE/
> > for i in `find . -name folders.db`
> >         do
> >         echo "Rebuilding Table $i"
> >         sqlite3 $i "vacuum;"
> > done
> 
> The thing is that the folders.db contains the actual cached data, like
> "pointers" into the mbox file where each message ought to begin, and this error
> message, about which is this bug report, is just about the value in folders.db
> file not in sync with actual mbox file. Vacuum is for something else. Just move
> away the folders.db file and let evolution recreate it based on the actual
> content of your mbox file(s). See comment #1 and comment #3.

You're right, I removed them, and let evo rebuild them, and now it seems that expunging works OK.

I believe that I've now workarounded the issue :-)

So, to summarize, get rid of all indexes and stuff, cleanup mboxes with formail, and reload evo with locking patch... at last we may be there.
Comment 250 Milan Crha 2011-05-05 12:14:41 UTC
(In reply to comment #249)
> So, to summarize, get rid of all indexes and stuff, cleanup mboxes with
> formail, and reload evo with locking patch... at last we may be there.

"cleanup mboxes with formail" is unnecessary from my point of view, I do not recall seeing anyone doing that. Though I agree that if one wants to be really sure, then yes, can try even with that.
Comment 251 Thomas 2011-05-17 22:27:22 UTC
Created attachment 187997 [details] [review]
Adopted a patch for evolution 2.32 to 2.28, https://bugzilla.gnome.org/attachment.cgi?id=187010

Had another case of a folder not being expunged, so I adopted that patch. Let's see. I haven't seen this in a long time, I might add.
Comment 252 Thomas 2011-05-17 23:58:37 UTC
(In reply to comment #251) 
> Had another case of a folder not being expunged, 

Could that be connected to the fact that the day before I upgraded sqlite3
from 3.6.16 to 3.7.6.2 without recompiling evolution and evo eds?
Comment 253 Milan Crha 2011-05-18 05:47:53 UTC
(In reply to comment #252)
> (In reply to comment #251) 
> > Had another case of a folder not being expunged, 
> 
> Could that be connected to the fact that the day before I upgraded sqlite3
> from 3.6.16 to 3.7.6.2 without recompiling evolution and evo eds?

If they didn't change API then it might not be a problem, I guess, but one never know.
Comment 254 Christoph Anton Mitterer 2011-05-19 10:39:11 UTC
--- Comment #247 from Christoph Wickert <cwickert@fedoraproject.org> 2011-05-04 18:09:44 UTC ---
> > Is this investigated yet? 
> Yes, read this bug report.
> Which I did,... what comments specifically do you mean?

> Any definite steps to save as much as possible?
> No mail is getting lost, just the meta information like flags.
> Ah,.. ok,.. so that wouldn't be too disastrous.
But these damaged headers for the flags,... are they going to disturb Evolution over and over again? So will I have to remove them or what?
Comment 255 Christoph Anton Mitterer 2011-05-19 10:39:46 UTC
--- Comment #249 from Olivier Berger <olivier.berger@it-sudparis.eu> 2011-05-05 10:28:53 UTC ---
> So, to summarize, get rid of all indexes and stuff, cleanup mboxes with
> formail, and reload evo with locking patch... at last we may be there.
> I did the very same _except_ formail and the new patch that was added later here.

So is this then the procedure?
1) Apply patch.
2) Remove all indices, DBs, etc. (everything except the mbox files themselves)
3) Optionally, formail
4) Recreate all of them.


Quite a few number of patches are attached now.
I'd like to open a bug to the Debian Evolution maintainers.
Which of the patches should they apply?


Cheers,
Chris.
Comment 256 André Klapper 2011-05-22 04:26:40 UTC
*** Bug 650472 has been marked as a duplicate of this bug. ***
Comment 257 Milan Crha 2011-05-23 14:42:31 UTC
(In reply to comment #255)
> So is this then the procedure?
> 1) Apply patch.
> 2) Remove all indices, DBs, etc. (everything except the mbox files themselves)
> 3) Optionally, formail
> 4) Recreate all of them.

Yes, basically

> Quite a few number of patches are attached now.
> I'd like to open a bug to the Debian Evolution maintainers.
> Which of the patches should they apply?

It depends on their version. Basically anything committed from the list of attachments, but because there were API changes between eds versions then it's little more complicated.
Comment 258 Milan Crha 2011-05-23 14:56:45 UTC
Created commit cd34676 in eds master (3.1.2+)
Created commit c9f7743 in eds gnome-3-0 (3.0.2.1+)
Comment 259 Akhil Laddha 2011-05-26 15:03:43 UTC
*** Bug 651148 has been marked as a duplicate of this bug. ***
Comment 260 Milan Crha 2011-06-30 17:05:31 UTC
*** Bug 649819 has been marked as a duplicate of this bug. ***
Comment 261 Akhil Laddha 2011-11-30 10:35:22 UTC
*** Bug 665183 has been marked as a duplicate of this bug. ***