GNOME Bugzilla – Bug 714100
Database backup and recovery
Last modified: 2020-09-04 03:02:30 UTC
---- Reported by jim@yorba.org 2012-11-15 08:55:00 -0800 ---- Original Redmine bug id: 6074 Original URL: http://redmine.yorba.org/issues/6074 Searchable id: yorba-bug-6074 Original author: Jim Nelson Original description: This morning we had a power outage while I was using Geary. This resulted in a corrupted database. Geary did not notify the user of the problem, and so when I ran it I was presented with an empty Inbox (although, interestingly, I could see a message in the Drafts folder; I'll examine later to see if other folders are accessible too). Like Shotwell, Geary should backup the database and then recover from the last known good one when one fails to load. Note that with the current implementation, Geary's database will grow in size forever. With Shotwell's database backup strategy, Geary's disk space usage will approximately double (not counting attachments, which we won't backup). **Note:** See #6189 for examples of databases that opened without error but then failed to read at runtime. The test if a database is valid should perform a simple SELECT operation. Related issues: related to geary - 7250: Local drafts folder (Open) related to geary - 7197: Turn on SQLite synchronous mode (Open) related to geary - 7345: Warn user when database is corrupted (Fixed) duplicated by geary - 6189: Geary won't sync inbox (Duplicate) duplicated by geary - 7195: Inbox show nothing (Duplicate) ---- Additional Comments From geary-maint@gnome.bugs 2013-08-14 15:14:00 -0700 ---- ### History #### #1 Updated by Jim Nelson 10 months ago * **Description** updated (diff) #### #2 Updated by Jim Nelson 10 months ago * **Tracker** changed from _Bug_ to _Feature_ #### #3 Updated by Jim Nelson 9 months ago * **Target version** changed from _0.3.0_ to _0.4.0_ #### #4 Updated by Jim Nelson 8 months ago This was reported on Launchpad as well: https://bugs.launchpad.net/geary/+bug/1163440 #### #5 Updated by Bruno Girin 7 months ago I just had the same issue. As a workaround, deleting the geary.db file and attachments directory in ~/.local/share/geary/user@domain.com/ forces Geary to re-create them and display mail properly. #### #6 Updated by Jim Nelson 7 months ago The basic strategy we want to use can be found in Shotwell today: Backing up database: http://redmine.yorba.org/projects/shotwell/repository/revisions/master/entry/s rc/main.vala#L430 Restoring database: http://redmine.yorba.org/projects/shotwell/repository/revisions/master/entry/s rc/db/DatabaseTable.vala#L55 Shotwell and Geary's database layers are different, but this code gives a basic pattern we can use. Because Geary has its own Db unit that does a lot of the abstraction, that's the place I vote to start looking to implement this code. One thing: in the case of restoring a backup, the restorer should go through the attachments table and delete any attachment files no longer represented there. In other words, database coherency should also be contingent on coherency with external files; today there are only attachments to worry about. #### #7 Updated by Andrea Corbellini 4 months ago Is it really useful to backup the database? I mean: currently it's only used as a cache. Wouldn't be better to just destroy the database and start over if it is found corrupt? #### #8 Updated by Eric Gregory 4 months ago Two reasons that come to mind: * Time: If you have Geary set to download your entire account, it can take a lot of time to rebuild the database. (On the flip side, this will mean a large backup.) * Outbox: Geary saves outbound messages to the database and only removes them after they've been successfully sent. We'd also like to do the same for drafts, see #7250. This is the one case where Geary's database isn't strictly a cache. #### #9 Updated by Jim Nelson 4 months ago There's a third reason to avoid blowing away the database if at all possible: most service providers and servers have a download threshold setting. When you reach certain limits, your bandwidth may be throttled or your account made unavailable. For most providers it's a high limit, but if you like having your entire mail store downloaded and your database is corrupted twice in one month (not inconceivable), then even Gmail will throttle you severely. (I experienced that when developing Geary's account prefetching code.) Even if Geary doesn't perform a backup-and-restore, it should at least detect when the database is corrupted and delete it. However, for the above reasons we'd like to have a fallback plan. #### #10 Updated by Jim Nelson 3 months ago * **Target version** changed from _0.4.0_ to _0.5.0_ --- Bug imported by chaz@yorba.org 2013-11-21 20:24 UTC --- This bug was previously known as _bug_ 6074 at http://redmine.yorba.org/show_bug.cgi?id=6074 Unknown milestone "unknown in product geary. Setting to default milestone for this product, "---". Setting qa contact to the default for this product. This bug either had no qa contact or an invalid one. Resolution set on an open status. Dropping resolution
Maybe a compromise solution would be to split the database into two - metadata and body parts. The former should usually be a lot smaller than the latter. E.g. One of my accounts has ~58k messages. Its geary.db (before & after a vacuum) is 3.5G. After dropping the body column from both MessageTable and MessageSearchTable and vacuuming, it is down to 763M. After dropping MessageSearchTable altogether, it's only 178M. This would obviously significantly complicate the DB code, but perhaps it's worth it.
It's unlikely that anything here is going to happen any time soon unless someone contributes a patch, and in the end backups are the responsibility of person whose machine it is, not Geary. Closing as wontfix.