GNOME Bugzilla – Bug 559153
Unable to convert mailboxes from Hardy to Intrepid versions
Last modified: 2011-04-28 14:55:49 UTC
I have about 1.5 gigabytes in mail folders. Everything worked fine under Ubuntu 8.04. I backed up my mail through Evolution's backup function from the main menu of Evolution while still under 8.04. I then installed Ubuntu 8.10, brought up the new Evolution and attempted to restore from backup when prompted to do so under the new Evolution. The app would unpack the backup file and then proceed to attempt to convert the mailbox files from version 22 to 24 (sqlite). The conversion dialog would show a progress bar which constantly stayed at 0% although the name of the mailbox file being processed did change from time to time so there appeared to be some activity (i.e. it did not "freeze"). However it took an extraordinary amount of time. I let it run for 15 or more hours. Finally I checked it and the dialog box was gone. I attempted to bring up evolution again and this time the conversion dialog box was displayed immediately and it appeared to try to go through the conversion process yet again. I let if complete again (another 12-15 hours) and attempted to bring up evolution a third time. Again, the conversion dialog came up and the process began to repeat a third time. I killed evolution at that point and am filing this bug report. Again, note that there was no indication of error and that progress bar constantly remained at 0% throughout. I tried to run gconf and set the version number of the app to 2.24.0 as I read this was a work around that some folks had luck with but that did not help. I am now without the use of my primary mail application and have had to resort to using the web interface to my mail account at my ISP. Any assistance would be appreciated! Best regards, Peter Pavlovich pavlovich@gmail.com
Sankar, can you give your patch here for the migration fix? Seb can give out a release for Ubuntu users to test?
Created attachment 121937 [details] [review] Fix
Created attachment 121938 [details] [review] Fix
Greetings, I see that Sankar has added what I presume to be a "fix" to this comment trail. Will there be directions forthcoming which will describe how to apply the patch? Again, any help would be very much appreciated! Thanks again!
(In reply to comment #1) > Sankar, can you give your patch here for the migration fix? Seb can give out a > release for Ubuntu users to test? > Can you tell me when "Seb" will be releasing a patch? Can I grab a copy of it as a test case? Thanks!
Seb, can you get a test package with this fix? I may not commit it to stable, as it ahs string/ui changes. but Im pusing this to OpenSUSE.
For the record, I have the same issue that Peter Pavlovich describes above. I'm using: - Evolution 2.24.1 - GNOME 2.24.1 - Ubuntu Linux 8.10 (64-bit) - Kernel 2.6.27-7-generic running on: - AMD Athlon 64 X2 4400+ - Asus A8R32-MVP Deluxe motherboard - ATI Radeon X1800 graphics card I copied my 46 MB Evolution backup (made on Hardy with all the latest updates applied) to a machine running a clean install of Intrepid and experienced similar events to those Peter has reported. When attempting to start Evolution for the first time under Intrepid I used the application's initialization wizard to locate my backup and restore it. The result was a brief display of the following dialog: http://farm4.static.flickr.com/3003/3003590641_551257e899_o.png followed by the silent fail Peter describes above. All subsequent attempts to start Evolution have failed in the same way. The dialog box visible in the image above flashes (very) briefly, and Evolution dies silently. This issue has persisted across system restarts and I'm currently unable to start the Evolution client. I have the latest Intrepid updates installed as of 11:24, 5th Nov 2008. Regards
Created attachment 122111 [details] [review] Fix that applies again 2.24.1 evo
Sankar, anything apart from the string change that stops us to push to stable?
For the Ubuntu users: Ubuntu Intrepid test packages are available at https://launchpad.net/~hggdh2/+archive.
Sankar commit to trunk anyways.
Greetings, I had tried the patches listed over the weekend but it did not seem to fix the problem. In order to gain access to my mail archives and basic functionality I had to take drastic measures. I found a work-around to the problem by deleting all ev-summary and ev-summary-meta files from the .evolution/.../mail subfolders before trying to restart evolution. The indicies seemed to be rebuilt correctly and Evolution is now functioning. There appears to be a few little problems during startup (when the program is scanning the folders) but the errors flash by so quickly I can't read them and there is nothing in any log file that I can find so I guess I will just live with it for now. Once I see another update to evolution through the main update mechanism, I'll back up my current files, restore the old backup and try again ... just to verify that whatever issue there was with the index conversion is fixed by whatever update is committed. If you have further questions or if I can help further diagnose this issue, please let me know at ppavlovich"at"gmail"dot"com.
Hmm, that sounds interesting. Evolution would do a automigration on demand. But this is a premigration during startup. Deleting those files would make it auto-migrate when the folders are loaded, but that might be slow and won't load the folder, that you haven't clicked yet. Peter, can you try us to get some gdb trace of 'thread apply all bt' during the migration at every hour interval? some 5-10 samples? I can see whatz happening at that instant. Thanks. I think that is just a work around. I need this to be fixed. Sankar, can you put some attention to this bug?
New comment from the Ubuntu bug: "I had the same problem. (0 % progress bar) I tried the test packages that hggdh proposed, I updated Evolution without errors, but when I tried to open it, the process stoped at 16%. I Waited for 1 hour, I restarted the computer, tried again and stoped at 15%. I don't know if is something to do with my accounts, I had IMAP and POP account as well."
hggdh, the main thing we want is that, when the user says its hung, I want some gdb samples to see what is going on 'thread apply all bt' sort of traces would do.
I have asked for a backtrace. Peter -- would you be able to get it?
*** Bug 561021 has been marked as a duplicate of this bug. ***
Posting requested in bug 561021 bactrace here, since the problem is very similar. The only difference with described here behavior I see is that 0% is the only problem I've seen during transition. And after 15 minutes transition finished. That's said during transition evolution is always in the same place: (gdb) thread apply all bt
+ Trace 209915
Thread 1 (Thread 0xb731d700 (LWP 28819))
At the same time in console there was: peter@camobap ~ $ LC_ALL=C /usr/libexec/evolution-data-server-2.24 evolution-data-server-Message: Starting server e-data-server-Message: adding type `ECalBackendCalDAVEventsFactory' e-data-server-Message: adding type `EBookBackendGoogleFactory' e-data-server-Message: adding type `ECalBackendContactsEventsFactory' e-data-server-Message: adding type `ECalBackendGoogleTodosFactory' e-data-server-Message: adding type `ECalBackendGoogleEventsFactory' e-data-server-Message: adding type `EBookBackendGroupwiseFactory' e-data-server-Message: adding type `ECalBackendWeatherEventsFactory' e-data-server-Message: adding type `EBookBackendFileFactory' e-data-server-Message: adding type `ECalBackendGroupwiseTodosFactory' e-data-server-Message: adding type `ECalBackendGroupwiseEventsFactory' e-data-server-Message: adding type `ECalBackendGroupwiseJournalFactory' e-data-server-Message: adding type `EBookBackendWebdavFactory' e-data-server-Message: adding type `ECalBackendFileTodosFactory' e-data-server-Message: adding type `ECalBackendFileEventsFactory' e-data-server-Message: adding type `ECalBackendFileJournalFactory' e-data-server-Message: adding type `EBookBackendVCFFactory' e-data-server-Message: adding type `ECalBackendHttpTodosFactory' e-data-server-Message: adding type `ECalBackendHttpEventsFactory' e-data-server-Message: adding type `ECalBackendHttpMemosFactory' in server_log_handler evolution-data-server-Message: Server up and running impl_GNOME_Evolution_Addressbook_BookFactory_getBook + file:///home/peter/.evolution/addressbook/local/system => 0x8086600 impl_GNOME_Evolution_Addressbook_Book_open (0x8086600) (evolution-data-server-2.24:29189): libedata-book-WARNING **: impl_GNOME_Evolution_Addressbook_Book_getBookView ((contains "x-evolution-any-field" "")) e_data_book_respond_get_book_view impl_GNOME_Evolution_Addressbook_BookFactory_getBook + file:///home/peter/.evolution/addressbook/local/1179227797.8433.0@camobap => 0x8086690 impl_GNOME_Evolution_Addressbook_Book_open (0x8086690) (evolution-data-server-2.24:29189): libedata-book-WARNING **: impl_GNOME_Evolution_Addressbook_Book_getBookView ((contains "x-evolution-any-field" "")) e_data_book_respond_get_book_view impl_GNOME_Evolution_Addressbook_BookFactory_getBook + file:///home/peter/.evolution/addressbook/local/1185002325.8404.10@camobap => 0x8086720 impl_GNOME_Evolution_Addressbook_Book_open (0x8086720) (evolution-data-server-2.24:29189): libedata-book-WARNING **: impl_GNOME_Evolution_Addressbook_Book_getBookView ((contains "x-evolution-any-field" "")) e_data_book_respond_get_book_view book_view file uref book_view file uref in server_log_handler (evolution-data-server-2.24:29189): libebookbackend-WARNING **: libdb error: page 71: illegal page type or format in server_log_handler (evolution-data-server-2.24:29189): libebookbackend-WARNING **: libdb error: PANIC: Invalid argument in server_log_handler (evolution-data-server-2.24:29189): libebookbackend-WARNING **: libdb error: /home/peter/.evolution/addressbook/local/system/addressbook.db: pgin failed for page 71 in server_log_handler (evolution-data-server-2.24:29189): libebookbackend-WARNING **: libdb error: PANIC: fatal region error detected; run recovery in server_log_handler (evolution-data-server-2.24:29189): libebookbackend-WARNING **: e_book_backend_file_search: error building list: DB_RUNRECOVERY: Fatal error, run database recovery book_view file uref ^C peter@camobap ~ $ LC_ALL=C evolution (evolution:29208): camel-WARNING **: camel_exception_get_id called with NULL parameter. ^C peter@camobap ~ $ Do you need any other tests?
Thanks a lot , but I don't need EDS traces, its Ok :-) YOu need to install debug info packages. [Evo/EDS/Gtk/Glib/Sqlite] and generate the traces? Do you mean that, in 15 mins, the migration is done?
Created attachment 122837 [details] evolution backtraces during convertation. I'm a bit confused with the last comment, but I supposed "YOu need to install debug info packages. [Evo/EDS/Gtk/Glib/Sqlite] and generate the traces?" is not a question, but a statement. So in attachment you'll find traces with sqlite and bonobo debugging symbols installed. And yes, actually migration of all my mailboxes (1.5G of mails) took ~7 minutes. (15 was a very rough estimate).
And forgot to note... There really exist a bug that after conversion you'll see two copies of each mail. I had to remove all .cmeta, .summary and etc files to get rid of them.
Created attachment 123157 [details] [review] Fix
Created attachment 123158 [details] [review] Fix
If anyone with source build + large data, can give feedback about the above two patches, it will be nice.
I am building test packages for Ubuntu Intrepid; they should be available for tests very soon.
two issues foound (please note I am building this on 2.24.1.1, since this the current stable that needs the patch): (1) there is a string case mismatch on your patch: 2.24.1.1 (mail/em-migrate.c) has: em_migrate_setup_progress_dialog (_("The summary format of the Evolution mailbox " "folders has been moved to sqlite since Evolution 2.24.\n\nPlease be " "patient while Evolution migrates your folders...")); While your patch has: em_migrate_setup_progress_dialog (_("The summary format of the Evolution mailbox " "folders has been moved to SQLite since Evolution 2.24.\n\nPlease be " "patient while Evolution migrates your folders...")); Notice the case mismatch on SQLite. Trunk matches 2.24.1.1. (2) when building I get a compile failure: em-migrate.c: In function 'em_migrate_imap_cmeta_1_4': em-migrate.c:2491: warning: ignoring return value of 'fwrite', declared with attribute warn_unused_result em-migrate.c: In function 'migrate_folders': em-migrate.c:2870: error: 'CAMEL_STORE_FOLDER_INFO_DISABLE_SUMMARY_LOAD' undeclared (first use in this function) em-migrate.c:2870: error: (Each undeclared identifier is reported only once em-migrate.c:2870: error: for each function it appears in.) make[5]: *** [em-migrate.lo] Error 1 make[5]: Leaving directory `/build/buildd/evolution-2.24.1.1/mail' make[4]: *** [all-recursive] Error 1 make[4]: Leaving directory `/build/buildd/evolution-2.24.1.1/mail' make[3]: *** [all] Error 2 The constant CAMEL_STORE_FOLDER_INFO_DISABLE_SUMMARY_LOAD does not seem to be defined anywhere (both Evo and E-D-S, both 2.24.1.1 and trunk).
recheck shows that trunk already has the SQLite string. Still, this patch needs to be applied to 2.24 stable: 2.24 is the current stable, and this is where most people will migrate to.
The build error of : undeclared enum : is probably because you missed my eds patch at comment #22 I will get the patch applied to 2.24 once it is tested to be usable with large datasets.
Sankar, thanks. I am not sure how, but I was using an older version of the patch indeed on E-D-S. First two reports from the Ubuntu bug state it is still failing to convert. One of the reports states it hangs on 15% (as before). I have asked the reporters for a stacktrace.
hggdh, when you say its failing and hanging, we would need good traces with debug symbols. Sankar's thing would fix up the issue of slowness and some stupidity in migration.
In Peter's case, I dont see a hang/block in the thread. Its going fine. It might take ~7 minutes before and with Sankar's patch, it should take pretty less. Still need to see the duplicate mails issue, not siure, if Sankar's thing fixes it.
One of the subscribers to the Ubuntu bugs says: "OK, Its working now. This is what I did: I made a new backup of my evolution emails in Hardy. Then I tried to import them in Fedora 10. With Fedora 10 I had exactly the same behaviour (stopped at 0%) but it gave a bug report. I decided to delete .evolution from my Home. I uninstalled and reinstalled evolution. After of that I tried to open Evolution and worked perfectly. That was with fedora 10. So I did the same with ubuntu 8.10 but before of that I did the "GDB stacktrace" that hggdh asked. I have attached the result here. After of that, I deleted .evolution from my Home, I uninstalled and reinstalled evolution. Then the migration was successfully, it took a few seconds only. Now is working good." The stacktrace is (from before removing ~/.evolution): (gdb) thread apply all bt
+ Trace 210293
I noticed the Ubuntu bug is not cross-referenced here. Here it is: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/293207 And, for completeness, this is the Fedora error report: Distribution: Fedora release 10 (Cambridge) Gnome Release: 2.24.1 2008-11-06 (Red Hat, Inc) BugBuddy Version: 2.24.1 System: Linux 2.6.27.5-117.fc10.i686 #1 SMP Tue Nov 18 12:19:59 EST 2008 i686 X Vendor: The X.Org Foundation X Vendor Release: 10503000 Selinux: Enforcing Accessibility: Disabled GTK+ Theme: Nodoka Icon Theme: Nou Memory status: size: 0 vsize: 0 resident: 0 share: 0 rss: 0 rss_rlim: 0 CPU usage: start_time: 0 rtime: 0 utime: 0 stime: 0 cutime:0 cstime: 0 timeout: 0 it_real_value: 0 frequency: 0 ----------- .xsession-errors --------------------- ** Message: rm -rf /home/jerson/.evolution-old/ ** Message: rm -rf /home/jerson/.camel_certs_old ** Message: rm /home/jerson/.evolution/.running rm: no se puede borrar «/home/jerson/.evolution/.running»: No existe el fichero o el directorio ** Message: evolution ** (nm-applet:2761): CRITICAL **: nm_utils_ssid_to_utf8: assertion `ssid != NULL' failed (evolution:3045): camel-WARNING **: camel_exception_get_id called with NULL parameter. (evolution:3045): camel-WARNING **: Cannot load summary file: '/home/jerson/.evolution/mail/local/Inbox.ev-summary': No existe el fichero o el directorio (evolution:3045): camel-WARNING **: Cannot load summary file: '/home/jerson/.evolution/mail/local/Outbox.ev-summary': No existe el fichero o el directorio ** (nm-applet:2761): CRITICAL **: nm_utils_ssid_to_utf8: assertion `ssid != NULL' failed --------------------------------------------------
Another backtrace, migration hanging:
+ Trace 210303
Thread 1 (Thread 0xb6452940 (LWP 11875))
Srinivasa I've tried your patch with evolution/eds-2.24.2 (yea, I had to modify patches a bit, but it's trivial). Now XX% at least starts to move but then in some circumstances it hangs. Also it hangs on the second run (if you just modify evolution version number in gconf-editor without moving back old mails). In logs I see this messages: ###0 records migrated for Gentoo/gentoo-dev/-securityOne chunk written ###0 records migrated for Gentoo/gentoo-dev/-catalystOne chunk written ###0 records migrated for Gentoo/gentoo-dev/-webappOne chunk written Thanks for your time anyway and if you need more tests, just ask.
Created attachment 123781 [details] [review] eds patch for 2.24 branch
Created attachment 123783 [details] [review] evo patch for 2.24 These patches address some more performance issues. The ui blocking issue is nto fixed yet. Jsut performance improvements alone. Can someone with a source build give feedback on this ? Thanks.
Well I've tried. On first start conversion took less then minute (for 1.5G of mails) but then I saw no mails in mailboxes and in the preview area I either saw nothing or Unable to retrieve message Cannot get message: в? l from folder /home/peter/.evolution/mail/local/Bugs.sbd/udev No such message messagges. At the same time in console I saw: ======================================================================= peter@camobap ~ $ LC_ALL=C evolution (evolution:6069): camel-WARNING **: camel_exception_get_id called with NULL parameter. em-migrate.c:3011:migrate_to_db: failed to get folder infos em-migrate.c:3011:migrate_to_db: failed to get folder infos em-migrate.c:3011:migrate_to_db: failed to get folder infos em-migrate.c:3011:migrate_to_db: failed to get folder infos em-migrate.c:3011:migrate_to_db: failed to get folder infos em-migrate.c:3011:migrate_to_db: failed to get folder infos addressbook_migrate (2.22.0) (evolution:6069): e-data-server-DEBUG: Loading categories from "/home/peter/.evolution/categories.xml" (evolution:6069): e-data-server-DEBUG: Loaded 37 categories ** (evolution:6069): DEBUG: mailto URL command: evolution %s ** (evolution:6069): DEBUG: mailto URL program: evolution libnm_glib_nm_state_cb: dbus returned an error. (org.freedesktop.DBus.Error.ServiceUnknown) The name org.freedesktop.NetworkManager was not provided by any .service files (evolution:6069): evolution-mail-WARNING **: VFolder of VFolders not supporting. Ignoring loading this vfolder as a subfolder (evolution:6069): evolution-mail-WARNING **: VFolder of VFolders not supporting. Ignoring loading this vfolder as a subfolder (evolution:6069): Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_text() (evolution:6069): Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_text() (evolution:6069): Pango-WARNING **: Invalid UTF-8 string passed to pango_layout_set_text() ======================================================================= After I closed and opened evolution again I saw mails and lots of following messages in console: ======================================================================= (evolution:6196): camel-local-provider-WARNING **: Summary doesn't match the folder contents! eek! expecting offset 294849 got -1, state = 8 =======================================================================
Can you remove all folders.db and folders.db.journal under your ~/.evolution ? and ensure that /apps/evolution/version and /apps/evolution/last_version both say 2.22 before starting evo with this patch ?
Sankar I've used .evolution folder which I keep just for tests from the times I've used evolution-2.22. I still have it as .evolution.bak and every time I test conversion I do rm -rf .evolution && cp -a .evolution.bak .evolution. Thus I'm sure (and actually I've double checked now) that there is no folders.db files. Also to trigger conversion of course before start evolution I set last_version and version to 2.22 in gconf-editor. Also this bug is reproducible as actually I did this tests twice. Also may be important: during tests I've kept evolution in off-line mode as I don't want new mails to mix with old backup...
tests packages are being built for Ubuntu Intrepid.
(In reply to comment #40) > Sankar I've used .evolution folder which I keep just for tests from the times > I've used evolution-2.22. I still have it as .evolution.bak and every time I > test conversion I do rm -rf .evolution && cp -a .evolution.bak .evolution. Thus > I'm sure (and actually I've double checked now) that there is no folders.db > files. Also to trigger conversion of course before start evolution I set > last_version and version to 2.22 in gconf-editor. > > Also this bug is reproducible as actually I did this tests twice. Also may be > important: during tests I've kept evolution in off-line mode as I don't want > new mails to mix with old backup... > Peter, 'Folder Summary mismatch' was a Evo-famous state of cache, which was caused by Evolution pre 2.24.x. In 2.24.x that issue is fixed. But your cache comes from 2.22.x and the only way to get rid of that is by removing the summary file and recreating with 2.22.x. This is what Sankar was trying to say, I guess
(In reply to comment #42) > Peter, 'Folder Summary mismatch' was a Evo-famous state of cache, which was > caused by Evolution pre 2.24.x. In 2.24.x that issue is fixed. But your cache > comes from 2.22.x and the only way to get rid of that is by removing the > summary file and recreating with 2.22.x. This is what Sankar was trying to say, > I guess > Yes. If the old summary is infected with "summary-folder-mismatch", migration will fail :( Are there any feedbacks on performance by anyone else ? I guess there is no way other than re-creating .ev-summary file with 2.22 for broken summaries (before migration) :(
Sankar for your patch. * Have a proper table name. something like .#evolution/temp or something like that * Check for warnings * Use proper naming for structures * Remove unwanted comments/dead code. Otherwise, commit it to trunk. remove UI portions and commit to stable.
Stable branch: http://svn.gnome.org/viewvc/evolution-data-server?view=revision&revision=9843 Trunk: http://svn.gnome.org/viewvc/evolution-data-server?view=revision&revision=9842 Needs more verifications and test runs. After which the bug can be closed.
Evne though the operations happen in a thread, I dont know why the ui is blocked. should identify and fix that as well.
I ran into the same problem when migrating from Fedora 9 to Fedora 10, here is the bug report that I filed on the Fedora Bugzilla. I have 2.5G in my .evolution directory. Description of problem: The F10 version of Evolution crashes when it tries migrate folders from the earlier version of Evolution. It looks like it's using an initialized size value when it attempts to allocate memory. Version-Release number of selected component (if applicable): How reproducible:100% Steps to Reproduce: 1. Start Evolution on F10 2. 3. Actual results: (evolution:22123): camel-WARNING **: Cannot load summary file: '/home/bjrosen/.evolution/mail/local/Chassis.sbd/Agilent.ev-summary': No such file or directory (evolution:22123): camel-WARNING **: Cannot load summary file: '/home/bjrosen/.evolution/mail/local/Chassis.sbd/HiQ.ev-summary': No such file or directory GLib-ERROR **: gmem.c:136: failed to allocate 32026589200 bytes aborting...
I still get the same issue with Evolution 2.24.4. evolution-shell-Message: Killing old version of evolution-data-server... (evolution:1219): camel-WARNING **: camel_exception_get_id called with NULL parameter. (evolution:1219): camel-WARNING **: Cannot load summary file: '/home/remm/.evolution/mail/local/Outbox.ev-summary': No such file or directory GLib-ERROR **: gmem.c:136: failed to allocate 31400991760 bytes aborting...
This bug is blocking many people because of this issue. I could not move my 8+ GB mbox to Fedora-10 yet, just because of this. What I propose is: (If possible), let's have an option to disable sqlite usage. I'm fairly happy with on-disk summary file actually ;) It just work. Regards, Devrim
I agree that there should be a way to disable SqLite. The F10 version of Evolution is seriously broken in other ways also almost certainly due to the SqLite change. I was able to do the conversion on a different machine (I'm guessing that there is an uninitialized pointer somewhere which causes this problem, and depending on the luck of the draw it can sometimes do the conversion). However Virtual Folders is a mess on the F10 version Evolution. The Folder summaries don't get updated when you delete mail unless you exit and reenter Evolution. The performance is also just awful.
What is the status of this bug? Are you aware that people cannot switch to Gnome 2.24, right? I had to skip Fedora 10 just for this -- if you don't care, probably evo users won't care, too -- and move to other clients.
Devrim, the main issue was that if you are migrating from 32bit OS to 64bit OS this crash happens and its not fixable through code, the old code had a issue. But as a work around, drop the summary, and let it regenerate as you start Evolution, there won't be data loss. It might take few more minutes.
Drop the summary? I'm talking about ~50 of them. It is a pity that Evolution 2.22 bug has *not* been fixed yet, and this means all Evo users must find another software. What I'm thinking is that Evo developers do not care about this issue. When are you planning to fix this? When noone will be using Evo?
Srinivasa, This does not fix the problem. I deleted all inbox files except "inbox", let all the other files regenerate, backed up, and still could not transfer to a fresh Jaunty, fully updated. I'll be stuck on Hardy forever or check out other mail clients unless this is fixed.
Dervim, If the bug is with the new software, we can fix it. But unfortunately, the bug is with the old softwared that was used, which wrote the partial data making it impossible for migration. search for file .summary or .ev-summary. These are the migration files. Just remove these files, it would regenerate the data. There shouldn't be any data loss.
Oh, do you mean I should back-up, and then edit the archive itself to delete the .summary and .ev-summary files? If this is the case, it might make sense to delete (or ignore and auto-regenerate) the .summary and .ev-summary files automatically for all migrations in the latest version of Evolution. This would effectively solve the problem, and you wouldn't need to change the old code.
On your 2.26.x or 2.24.x machine, just move these files (from your .evolution/*) to a temp location and start evo. Evo won't generate these again. These files are being represented in the DB now.
Ah, gotcha, thanks! However, unless someone checks out this specific bug report and reads to the bottom, they'll only know their mail doesn't work in a new version of Evo. Could we add a patch that simply ignores or deletes the .summary and .ev-summary files on transition to sqlite? This would make a lot of people happy, as they can move to the new evo and have it "just work." I'm no programmer, but if someone could do this it would make a lot of non-tech people happy I'm sure! Thanks!
Its only for this specific case. Otherwise, it should be normal. The only disadvantage would be that, this way of migration would be very slow, compared to the current default way. Lemme see what can be done.
This issue has created considerable problems for myself and my customers; I only discovered the issue after upgrading several systems from ubuntu hardy to intrepid and then jaunty all in one sesson. Please see my comments on launchpad: https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/293207/comments/42 [Conversion of existing mailboxes fails silently] Note: I did try the "delete the .summary and .ev-summary files" method - found that I needed to delete all but the mailbox files to even read my existing msgs & then open mail via the calendar applet & let the index get regenerated. That worked & I could then open via the email applet. I only tested on one system as I do not wish to risk losing msgs on the production systems.
setting the right component.
The 2.26 version of Evolution in Fedora 11 no longer crashes on the conversion, however it takes it about 14 hours to complete the conversion. The problem with virtual folders is still there, it keeps the folders highlighted even though you've read the mail, it only clears the highlights when you empty the trash (which doesn't always work) or when you restart Evolution. Also sometimes mail doesn't appear in the unread mail virtual folder even though it does appear in a physical folder. It's also still crashing in F10 after it's been open for a while. The Evolution developers need to focus on fixing the bugs, it's getting close to unusable. The last good version of Evolution was in Fedora 8 (I'm guessing that was 2.20), it's been getting progressively more unstable with every subsequent release. Evolution was always a solid application prior to the the version in Fedora 9, what's happened to it?
Hmm. I rsynced my mailbox from F-9 to F-11. (rsync ~/.evolution to remote server) I did not rsync anything else. Then, on remote server started Evolution. It asked me to configure my accounts. I did. ...and I think it worked. I have ~ 9 GB e-mail in 3 accounts. All I needed is to remap my search folders. Weird :)
(In reply to comment #62) > The Evolution developers need to focus on fixing the bugs, it's getting close > to unusable. The last good version of Evolution was in Fedora 8 (I'm guessing > that was 2.20), it's been getting progressively more unstable with every > subsequent release. Evolution was always a solid application prior to the the > version in Fedora 9, what's happened to it? +1 on the whole.
Update: I updated my Fedora-9 laptop to Fedora-11. After upgrading, evo decided to migrate folders to sqlite (ok...), but it crashed at some point (I even could not connect to process via strace or such) . I killed evo with --force-shutdown, and started it. It worked for a few hours. However, after a reboot, it now: * does not update my search folders * does not update unread message count in search folders. Congrats on evo developers for breaking a software which used to work in 2.22. I still cannot understand why all people are forced to use sqlite. I was happy with on-disk-summary format. Please, please add an option to get rid of sqlite. I believe it cannot handle my 10 GB email. D.
*** Bug 589114 has been marked as a duplicate of this bug. ***
Let me just express my utter frustration with how spectacularly Evo devs are failing to handle such a simple, and at the same time such a critical use case. From what I gather (and I may be wrong) the *summary files in the ~/.evolution folder are attempted to be converted to the "new" format, and because of some archaic code that had rendered it so difficult, the current code fails at its task. Now, given the knowledge that simply removing the *summary files from one's .evolution folder helps (in my case Evo started immediately after having deleted these files), why on Earth this shouldn't not be the default action when old user files need to be upgraded to the new format? I just don't understand why we're having this issue at all? Let's just delete these files when upgrading, and be done. So much time lost for nothing.
I think it's also worth adding that this will become a HUGE issue when businesses and many individuals move from 8.04 to 10.04 next spring. Those are the people who will depend on Evolution working right in a large-scale environment. If this isn't fixed before 10.04 then a lot of people in large organizations are going to be very angry. Ubuntu's recently made the jump from Rhythmbox to Banshee, and if this gets a lot of people angry they might just consider moving from Evolution to Thunderbird+Lightning. I know I am - like Devrim, search is still broken for me as well. The old summary system worked great.
*** Bug 564938 has been marked as a duplicate of this bug. ***
Ok, this is really becoming embarrassing. The last comment from an Evo developer is at 2009-04-15. <capital letters>Can you hear us?</capital letters>
Knock knock! I just saw "-4 unread e-mails" in one of my search folders. It is getting worse. <capital letters> please revert sqlite commit </capital letters>
I just changed the name of one of my folders and ALL the emails inside it were lost. I've tried unchecking "hide deleted messages", "show hidden messages", the works. They're simply gone. That was my working folder - the data in there was invaluable. I'm DONE with Evolution. This insane move to sqlite will cost me MANY hours of work and the respect of people who now won't receive replies to their messages, and action on the work assigned me. Thunderbird, here I come. It can't possibly be less reliable than this.
When I click evo icon, I leave my chair and get a beer. It tries to read some data from sqlite, and it takes about 5 minutes -- no need to say that iostat shows 100% disk activity. PLEASE REVERT SQLITE SHIT. We tried to be polite until now -- but Evo developers don't care us, so why should we? I will return using pine, or say, alpine.
EVO DEVELOPERS? CAN YOU HEAR US?
This is going fscking insane. Evo developers tend to behave us rudely.
*** Bug 592249 has been marked as a duplicate of this bug. ***
(In reply to comment #73) > When I click evo icon, I leave my chair and get a beer. It tries to read some > data from sqlite, and it takes about 5 minutes -- no need to say that iostat > shows 100% disk activity. > > PLEASE REVERT SQLITE SHIT. We tried to be polite until now -- but Evo > developers don't care us, so why should we? > > I will return using pine, or say, alpine. Devrim, Just to set a few records clear, The sqlite work was done by Srinivasaragavan and Me. Both of us do not represent Novell for Evolution at the moment. Had I known that my manager is planning to move me out of the team, I wouldn't have started this work at all. As I know it is tough for someone to fix a regression/bug created by someone else's code (and also I knew Matthew will be busy with kill-bonobo work which is important for GNOME reasons) Moving to sqlite is important because databases are vital if mails have to be more queriable for project like Tracker/Beagle. Also, moving to sqlite has resulted in memory savings. The old summary format was not transferrable across 32 and 64 bit machines as well. All these are addressed with the sqlite summary - http://tinyurl.com/yzt2pb2 http://tinyurl.com/yjdq8xl The initial version of migration code that we wrote was not performing well and was without even a progress dialog. We modified it to show progress (even though in an ugly manner) and made it work a little faster. Search-folders and some aspects of migration were not fully complete by the time, I was moved out of the team. And, for personal reasons, I was not in a mood to contribute to Evo. after I got moved out of the team. Reverting back to pre-sqlite summaries will not solve any problem. IIUC, There could be some issues in: 1) Migration 2) Search folders 3) Performance (on some operations) so, the right way to go ahead will be to identify the culprit code-paths in the above issues and fix them. may be profile the operations for performance bottlenecks and fix the search-folder/migration bugs. Complaining about all the issues in a single bug will not be helpful. Evolution bugzilla gets a lot of mails and the officially paid engineers to work on Evolution is not as big as it used to be. So your bugzilla complaints could've been overlooked by mistake. If you are really interested in solving the issue, Attend a evolution team meeting at wednesday 10:00 utc and try to get your issues to the priority list of the current evolution developers. The fact that this migration issue is not on their current list could mean that not many people have reported it (or they de-prioritized it because there are other more critical things to do).
*** Bug 603602 has been marked as a duplicate of this bug. ***
*** Bug 612549 has been marked as a duplicate of this bug. ***
Created attachment 156645 [details] First gdb trace
Created attachment 156646 [details] Second gdb trace
Created attachment 156647 [details] Third gdb trace
Created attachment 156648 [details] Fourth gdb trace
Created attachment 156649 [details] Fifth gdb trace
Created attachment 156650 [details] Sixth gdb trace
Created attachment 156651 [details] Seventh gdb trace
Created attachment 156652 [details] Migrating dialog screenshot
I am having the same issue migrating from 32 bit Ubuntu 8.04.4 LTS Evolution package 2.22.2-0ubuntu1.2 to 64 bit Ubuntu 9.10 Evolution package 2.28.1-0ubuntu1. This was a fresh install of Ubuntu 9.10. I made the backup using the menu option in Evolution 2.22.2. I transferred the file to the new machine and selected it on the first run of Evolution 2.28.1 as the file to restore. In my case the "Migrating..." dialog (attachment 156652 [details]) pops up and silently dies. I have deleted some mail but it always appears to die. Subsequent restarts of Evolution begin the migration process and the silent failure happens again. I have tried several workarounds. First I completely purged the package and then deleted the .evolution and .gconf/apps/evolution folders and their contents. I then moved the backup tarball to my home directory and reinstalled evolution. Starting up evolution got the same results as before. After playing around with gdb a little bit. I noticed some warning messages in the traces about summary files. I guessed that the content of these files may be the problem so I went into Evolution 2.22.2 and deleted the mail in those folders. I then closed Evolution 2.22.2 and reopened it (I have purge deleted files set every time). Then I made a new backup with Evolution 2.22.2 and transferred it to the new machine. What I noticed is that the warning messages about the summary files go away. You can see the progression from the attached files in trace1.txt (attachment 156645 [details]), trace2.txt (attachment 156646 [details]), and trace3.txt (attachment 156647 [details]). I'm not sure if this is significant, I'm adding it here in an attempt to be complete. So the sequence went like this: 1. Uninstall Evolution 2.28.1 apt-get --purge remove evolution 2. Remove directories rm -rf ~/.evolution ~/.gconf/apps/evolution ~/.camel_certs 3. Install Evolution 2.28.1 sudo apt-get install evolution-dbg evolution-plugins evolution-indicator evolution-exchange evolution-couchdb 4. Start gdb and produce trace1.txt (attachment 156645 [details]) 5. Delete emails in folder that has a summary file warning message. Stop and start Evolution 2.22.2 and then create a new backup file using the menu option. 6. Transfer the backup file to the new machine. 7. On the new machine, repeat steps 1-3. 8. Start gdb and produce trace2.txt (attachment 156646 [details]) 9. Repeat steps 5-7. 10. Start gdb and produce trace3.txt (attachment 156647 [details]) trace3.txt seems to have no warnings but the program still crashes. trace4.txt (attachment 156648 [details]), trace5.txt (attachment 156649 [details]), trace6.txt (attachment 156650 [details]), and trace7.txt (attachment 156651 [details]) show subsequent runs with backtraces. I did not create a new backup file after step 10 above. Instead I simply stopped gdb and started it again using the currently installed evolution. The sequence is like this for each of the trace files 4-7: 1. Start gdb and run evolution. 2. It tries to migrate and crashes. 3. Produce trace file and quit. In case it matters, the backup file: jesse@jthink:~$ ll -h evolution-backup-2010-03-20.tar.gz -rw-r--r-- 1 jesse jesse 220M 2010-03-20 17:20 evolution-backup-2010-03-20.tar.gz I also have the files that produced the warnings in traces 1-3. I have diffed them but I don't know anything about how evolution works so I can't comment on it intelligently. If you want them let me know. I don't want to attach them because they contain email addresses. If there is anything else I can do to help solve the problem please let me know. Jesse
*** Bug 578535 has been marked as a duplicate of this bug. ***
WONTFIX as per "its not fixable through code".
*** Bug 631001 has been marked as a duplicate of this bug. ***
Pardon me if I'm a bit confused, but what is the official work around for this bug if it will not be fixed? I've hit the predicted Ubuntu LTS migration problem foretold by ctenorman and need to understand how to resolve this issue to continue supporting Evolution.
(In reply to comment #92) > what is the official work around for this bug if it will not be fixed? There is none.
(In reply to comment #92) > Pardon me if I'm a bit confused, but what is the official work around for this > bug if it will not be fixed? I've hit the predicted Ubuntu LTS migration > problem foretold by ctenorman and need to understand how to resolve this issue > to continue supporting Evolution. I was able to work around this with some data loss. Unfortunately, I did not come back here and immediately document it. When evolution crashed, I just assumed nothing was being loaded. That is not true. All my mail folders and mail were brought back. Something in the start up kept trying to do a restore. So this kept me from getting evolution to start. I can't remember for sure but I believe I deleted the files that indicated I was doing a backup. All my mail and inbox folders were fine. I lost ALL my address book and calendar. That turned out to not be a problem because I synced them regularly with my palm pilot. So, to recap: 1. Do the restore, let it crash. 2. Investigate the mail folders and delete whatever it is (sorry) that indicates a backup is being done. 3. Get evolution to start and see what made it and what didn't. 4. Use synch to get back what you're missing (hopefully).
Sad. After long years as an evolution advocate it has come to this..... I've wasted way too much time trying to get the evolution in the new Ubuntu LTS distro to just import my existing mailbox files. Since I can't stay on the previous LTS and since evolution has let me down, I'm cutting my losses and saying goodbye. --- Sweet, so would I, yet I should kill thee with much cherishing. Good night, good night! Parting is such sweet sorrow, that I shall say good night till it be morrow. -- Shakespeare For others that find there way here looking for enlightenment there is a way out of this mess.... Install thunderbird. You can copy your evolution mailbox files from ${HOME}/.evolution/mail/local to ${HOME}/.thunderbird/<some string>.default/Mail/Local Folders. Don't copy the extraneous evolution junk .cmeta, .summary, etc. Just copy the mailbox files (the big files with no suffix) and it takes the subdirectories if you had them. Crank up thunderbird and it will index the files just fine and with no fuss. There is a utility named vcfconvert that you can use to import your contact list into thunderbird. To use it, export your contacts from evolution into a vcf file then use the utility to convert it to ldif format. You can import the ldif into thunderbird. Thunderbird has a plugin named lightning that does a simple calendar function. I haven't looked for a way to transport calendar info across, since I don't use it at home. I may have to work on that for the office machine at some point but its easy enough to start over from scratch. If Thunderbird doesn't float your boat, there are many other mailers that are probably just as easy to convert to. Good luck.
*** Bug 639693 has been marked as a duplicate of this bug. ***
*** Bug 648874 has been marked as a duplicate of this bug. ***