After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 714100 - Database backup and recovery
Database backup and recovery
Status: RESOLVED WONTFIX
Product: geary
Classification: Other
Component: engine
master
Other All
: High normal
: ---
Assigned To: Geary Maintainers
Geary Maintainers
Depends on:
Blocks:
 
 
Reported: 2012-11-15 04:55 UTC by Jim Nelson
Modified: 2020-09-04 03:02 UTC
See Also:
GNOME target: ---
GNOME version: ---



Description Charles Lindsay 2013-11-21 20:24:38 UTC


---- Reported by jim@yorba.org 2012-11-15 08:55:00 -0800 ----

Original Redmine bug id: 6074
Original URL: http://redmine.yorba.org/issues/6074
Searchable id: yorba-bug-6074
Original author: Jim Nelson
Original description:

This morning we had a power outage while I was using Geary. This resulted in a
corrupted database. Geary did not notify the user of the problem, and so when
I ran it I was presented with an empty Inbox (although, interestingly, I could
see a message in the Drafts folder; I'll examine later to see if other folders
are accessible too).

Like Shotwell, Geary should backup the database and then recover from the last
known good one when one fails to load.

Note that with the current implementation, Geary's database will grow in size
forever. With Shotwell's database backup strategy, Geary's disk space usage
will approximately double (not counting attachments, which we won't backup).

**Note:** See #6189 for examples of databases that opened without error but then failed to read at runtime. The test if a database is valid should perform a simple SELECT operation.

Related issues:
related to geary - 7250: Local drafts folder (Open)
related to geary - 7197: Turn on SQLite synchronous mode (Open)
related to geary - 7345: Warn user when database is corrupted (Fixed)
duplicated by geary - 6189: Geary won't sync inbox (Duplicate)
duplicated by geary - 7195: Inbox show nothing (Duplicate)



---- Additional Comments From geary-maint@gnome.bugs 2013-08-14 15:14:00 -0700 ----

### History

####

#1

Updated by Jim Nelson 10 months ago

  * **Description** updated (diff)

####

#2

Updated by Jim Nelson 10 months ago

  * **Tracker** changed from _Bug_ to _Feature_

####

#3

Updated by Jim Nelson 9 months ago

  * **Target version** changed from _0.3.0_ to _0.4.0_

####

#4

Updated by Jim Nelson 8 months ago

This was reported on Launchpad as well:
https://bugs.launchpad.net/geary/+bug/1163440

####

#5

Updated by Bruno Girin 7 months ago

I just had the same issue. As a workaround, deleting the geary.db file and
attachments directory in ~/.local/share/geary/user@domain.com/ forces Geary to
re-create them and display mail properly.

####

#6

Updated by Jim Nelson 7 months ago

The basic strategy we want to use can be found in Shotwell today:

Backing up database:

http://redmine.yorba.org/projects/shotwell/repository/revisions/master/entry/s
rc/main.vala#L430

Restoring database:

http://redmine.yorba.org/projects/shotwell/repository/revisions/master/entry/s
rc/db/DatabaseTable.vala#L55

Shotwell and Geary's database layers are different, but this code gives a
basic pattern we can use. Because Geary has its own Db unit that does a lot of
the abstraction, that's the place I vote to start looking to implement this
code.

One thing: in the case of restoring a backup, the restorer should go through
the attachments table and delete any attachment files no longer represented
there. In other words, database coherency should also be contingent on
coherency with external files; today there are only attachments to worry
about.

####

#7

Updated by Andrea Corbellini 4 months ago

Is it really useful to backup the database? I mean: currently it's only used
as a cache. Wouldn't be better to just destroy the database and start over if
it is found corrupt?

####

#8

Updated by Eric Gregory 4 months ago

Two reasons that come to mind:

  * Time: If you have Geary set to download your entire account, it can take a lot of time to rebuild the database. (On the flip side, this will mean a large backup.)
  * Outbox: Geary saves outbound messages to the database and only removes them after they've been successfully sent. We'd also like to do the same for drafts, see #7250. This is the one case where Geary's database isn't strictly a cache.

####

#9

Updated by Jim Nelson 4 months ago

There's a third reason to avoid blowing away the database if at all possible:
most service providers and servers have a download threshold setting. When you
reach certain limits, your bandwidth may be throttled or your account made
unavailable. For most providers it's a high limit, but if you like having your
entire mail store downloaded and your database is corrupted twice in one month
(not inconceivable), then even Gmail will throttle you severely. (I
experienced that when developing Geary's account prefetching code.)

Even if Geary doesn't perform a backup-and-restore, it should at least detect
when the database is corrupted and delete it. However, for the above reasons
we'd like to have a fallback plan.

####

#10

Updated by Jim Nelson 3 months ago

  * **Target version** changed from _0.4.0_ to _0.5.0_



--- Bug imported by chaz@yorba.org 2013-11-21 20:24 UTC  ---

This bug was previously known as _bug_ 6074 at http://redmine.yorba.org/show_bug.cgi?id=6074

Unknown milestone "unknown in product geary. 
   Setting to default milestone for this product, "---".
Setting qa contact to the default for this product.
   This bug either had no qa contact or an invalid one.
Resolution set on an open status.
   Dropping resolution 

Comment 1 Michael Gratton 2016-06-14 13:40:59 UTC
Maybe a compromise solution would be to split the database into two - metadata and body parts. The former should usually be a lot smaller than the latter.

E.g. One of my accounts has ~58k messages. Its geary.db (before & after a vacuum) is 3.5G. After dropping the body column from both MessageTable and MessageSearchTable and vacuuming, it is down to 763M. After dropping MessageSearchTable altogether, it's only 178M.

This would obviously significantly complicate the DB code, but perhaps it's worth it.
Comment 2 Michael Gratton 2020-09-04 03:02:30 UTC
It's unlikely that anything here is going to happen any time soon unless someone contributes a patch, and in the end backups are the responsibility of person whose machine it is, not Geary.

Closing as wontfix.