GNOME Bugzilla – Bug 655263
mail lost in MailDir migration for folders with corrupted summary
Last modified: 2021-05-19 12:15:04 UTC
I was just digging in my (migrated 3.0 'sent' folder), and was -appalled- to discover that mail had not been migrated. The old folder has 79 messages that are not in the new folder - these were the last ~80 messages in the mbox it seems, and just didn't make it to the new maildir world. My (new) maildir Sent box looks like: mail foo ----------- 2011-07-20 mail baa ----------- 2011-07-15 And I'm most certain I sent mail between those dates :-) better it even made it to the mbox folder. Of course - this being Evolution, I am resigned to my 'Sent' mbox folder displaying this wonderful, affirming and encouraging message: "Error while Storing folder 'Sent'." "Summary and folder mismatch, even after a sync" when I switch to and from it, that is par for the course: but I -don't- expect it to go loosing data. Of course, my 'Sent' folder summary had for a while been showing some miraculous duplicatin, or even triplication of messages in the message list [ this seems to have gone in 3.0 too ] - with bogus mails you couldn't click on and so on. More encouragingly the 26k messages in my Sent-to-2010 and the 12k messages in Sent-to-2005 have (at least) identical folder counts between maildir and mbox. I had -hoped- that the migration would use the actual mbox itself, parsing it accurately and migrating what it could, rather than foolishly believing whatever compound brokenness exists at the upper levels ;-) any thoughts ? Clearly I can't really attach my Sent mail box (sorry).
Folder with corrupted summary doesn't get migrated to maildir format properly. Workaround is to first make summary sane by removing folder.db in existing mbox format and then manually move folder content to maildir.
I did not consider the case of summary being corrupted. I can prolly remove the old folder summary before migration the folder which should solve this case.
glad there is a potentially easy fix :-) Lots of my folders (for unknown reasons) complain(ed) about summary corruption.
Hmm. removing the folders.db would delete all the user_flags which has been set in mbox messages. Am looking around to see if i can detect a folder_summary mismatch and then remove the folders.db.
am now planning to fix this in the following way, copy messages until an error is encountered due to the summary_mismatch if summary_mismatch rebuild the summary, preserving the user_flags (at-least my assumption is that it should just update the file offsets, need to check if its correct) continue to copy all the left messages. <Add progress information>.
Created attachment 192739 [details] [review] Fallback method if migration fails due to an error. First we try to migrate the folder contents along with the summary. Migrate mails one by one to detect errors. It should be as fast as previous code as the mbox is indexed. If an error is detected, use the fallback mechanism which directly imports the mbox file into the target folder.Note that the flags will be lost with the fallback mechanism. Will make a separate patch for showing the progress information.
committed to gnome-3-0 branch. will need to modify a bit to master as some api's have been removed..
*** Bug 657008 has been marked as a duplicate of this bug. ***
(In reply to comment #7) > committed to gnome-3-0 branch. will need to modify a bit to master as some > api's have been removed.. Chen: So this is only fixed when updating from 2.3x to 3.0, but not from 2.3x to 3.2/3.4?
Comment on attachment 192739 [details] [review] Fallback method if migration fails due to an error. $:andre\> git apply --check 655263.diff error: patch failed: mail/e-mail-migrate.c:797 error: mail/e-mail-migrate.c: patch does not apply error: patch failed: mail/em-utils.c:2446 error: mail/em-utils.c: patch does not apply error: patch failed: mail/em-utils.h:35 error: mail/em-utils.h: patch does not apply error: patch failed: mail/importers/mail-importer.c:40 error: mail/importers/mail-importer.c: patch does not apply
Removing blocker status since this bug isn't blocking anything.
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/Community/GettingInTouch/BugReportingGuidelines and create a new bug report ticket at https://gitlab.gnome.org/GNOME/evolution/-/issues/ Thank you for your understanding and your help.