GNOME Bugzilla – Bug 778276
EmailFlagWatcher using CPU usage every ~3 min
Last modified: 2018-02-07 22:12:13 UTC
Might be related to Bug 766547 (or not) and/or Bug 713134, but I still have CPU bursts issue. Here is what I gathered: - Geary hits from 5% to 60% CPU usage on a bursts basis: approx 2min idle (0%), 4min busy like it: 10:42:42 <-- BUSY --> 10:47:06 <-- IDLE --> 10:49:11 <-- BUSY --> 10:53:44 <-- IDLE --> 10:55:35 <-- BUSY --> 10:58:49 <-- IDLE --> 11:00:43 <-- BUSY --> 11:04:12 <-- IDLE --> - This happens if I have only a gmail account, only my personnal account, or both accounts at the same time. However, in the case of gmail only account, bursts are only in 5% to 15% CPU range! - Both my accounts are perfectly synced (all emails grabbed) so Geary is not syncing anything - I don't see anything in standard logs (eg if I enable only --debug) - I disabled IPV6 so I am not having the issue from Bug 776042 anyway - I am running geary with --hidden option, so no UI is present/doing anything Any idea what should I do? I tried using --log-X options, but for a single burst period it leads to 2M lines of output! It stranges that I am the only one to see that, though.
I finally found the culprit with is EmailFlagWatcher which forces full synchronization of currently open folders every 3 minutes (https://github.com/GNOME/geary/blob/master/src/engine/imap-engine/imap-engine-email-flag-watcher.vala#L18). But since my inbox contains ~10K, it takes several minutes to fetch them. Maybe we should reconsider the refresh frequency (30m? 24h?). I am not sure it really makes sense to synchronize every 3 minutes, because users won't use 2 different mails clients simultaneously most of the time - and even if so, does it matter if geary keeps some emails unsynced for, say, 24h?
The correct fix would be to implement CONDSTORE/QRESYNC (Bug 713117). In the mean time (and for servers that don't support CONDSTORE), I'm thinking that maybe it would be good to do a two-tier sync: - Full sync less often (once every hour or two?) - Partial sync of n most recent messages (n = 100? n = "count(internal_date >= (now - 7d))"?) People use multiple clients maybe a bit more often then you might expect, it's not uncommon for me to switching between my mail on my phone and desktop several times an hour for example. As an aside, I wonder why this only runs on open folders and not others?
Looking into this some more, it only runs on open folders since it hooks into folders opening and closing. This is weird since open folders always have a mailbox selected, and hence any changes made to messages in that folder should get reported by an via IDLE or an unsolicited FETCH. Geary is currently ignoring those however, and hence instead needs the EmailFlagWatcher. So it seems like the right fix for this is to get Geary to pay attention to these unsolicited FETCH responses, then we can either remove EmailFlagWatcher altogether, or just run it once when the folder is opened of the background sync isn't updating flags. I've been investigating this a bit, will report back when I make some useful progress.
WIP branch for this in git as: wip/778276-better-flag-updates Currently it's a large-ish rework of GenericAccount that adds yet another operations queue but for account-wide operations, and has converted local folder load, remote folder refresh and folder unseen updates to operations in this queue. It also ensures that when Geary changes a folder locally we signal it has happened, so background sync can fire up only when needed, and not periodically. This also lets increase the remote folder refresh period right up from every 3 minutes to every 10 minutes since we don't need it to find changes to folders that Geary has caused any more. Finally, it's now taking notice of unsolicited FETCH responses, so the flag watcher doesn't need to run when a folder is open. Still to do: Convert background sync into an account operation, maybe integrate the flag watcher into that, and maybe do something about Bug 713117.
One last note, we still need to check for flags having changed when first opening a folder, so need to look at the best solution for that, hence integrating flag watcher into the background sync, and/or Bug 713117.
Fix pushed for this as commit 6c5a7d5.
*** Bug 714693 has been marked as a duplicate of this bug. ***