GNOME Bugzilla – Bug 667891
Rapid message list regen when downloading a folder
Last modified: 2013-09-13 01:05:05 UTC
After upgrading to 3.3, I see the following - open a big folder, so that the "Fetching new messages for folder X" operation will take a while - messages are apparently loaded in chunks to populate the message list - in between chunks, for a fraction of a second, a "Generating message list..." message will appear in the statusbar, making it bounce with the "Fetching new messages for folder X" message in there Is it really necessary to show that message there? If it really could take so long that a notification to the user is due, I think a better approach would be - show a spinner icon directly in the view, maybe on the first column header? - only show the message after we've been generating the message list for a while (say half a second), to avoid flashing UIs or a combination of the two.
The status bar is a symptom of a deeper problem. The deeper problem is we're regenerating the entire message list with each incremental update from the IMAPX backend, which hammers the CPU unnecessarily and shortens battery life. (With the old IMAP backend, we wouldn't generate the message list at all until the entire folder summary was downloaded, which could take several minutes for massive folders. So we've gone from one extreme to the other.) My thought was to put a small timeout between receiving a "folder-changed" signal and regenerating the message list, and if another "folder-changed" signal is received while the timeout is in effect, restart the timeout. The exact timeout interval may need to be tuned, but I think that should throttle down the rapid message list regens.
(In reply to comment #1) > My thought was to put a small timeout between receiving a "folder-changed" > signal and regenerating the message list, and if another "folder-changed" > signal is received while the timeout is in effect, restart the timeout. The > exact timeout interval may need to be tuned, but I think that should throttle > down the rapid message list regens. [ Note that I don't know what's happening behind the scenes so forgive me if this is bogus ] Couldn't this approach lead to a situation where the folder is never updated if all the folder-changed signals are received before the timeout expires? I think a better solution would be compressing the folder-changed signals received within the timeout to a single refresh that will happen at the end of the timeout instead. Anyway, my question is more: since that statusbar usually has a lot of noise already, does it make sense to display at all the "Generating message list" message there?
I'm only confirming the issue, same here, IMAPX went crazy. About the "Generating message list" in the status bar, you can (theoretically) cancel the message list generation while it is in there. Maybe that's the only reason to have it there.
Created attachment 207189 [details] [review] proposed evo patch for evolution; I would like to ask Matthew to review this. My ideas about this change: - regen list immediately, if folder changed for the first time - or the time between two last changed calls is long enough - or there are too many pending changes The thing is that this change will not penalize "usual" folders. I chose timeout 1.5 second and for "too many" I chose 1000 pending message updates. It's slower in updating, but with value like 100 I have regen called about once a second, which is still too much flashing.
I'll give this a try, but I think the real root of the problem lies in IMAPX bombarding Evolution with CamelFolder::changed signals. I think it's coming from imapx_command_step_fetch_done(). The rate limiting really needs to happen there instead of in the message list.
I thought of it a bit too, but then I decided to change it in message-list. It makes sense, as it's the main consumer here. On the other hand, virtual folders are also listening to these "changed" signals :-/
Created attachment 212220 [details] [review] eds patch for evolution-data-server; Aah, this is sad, it's not that much imapx issue, even the fix comes there. The camel_folder_changed had been called by CamelFolderSummary code, when changing flags in CamelMessageInfo. I added there these automatic notifications when playing with it during 3.3.x. The fix here makes sure that the message info is not added to the summary before the flags are updated (it's a new message anyway), thus the factory will not do its own notifications and only imapx will notify about changes, in 500 messages chunks.
Created commit 42e5b06 in eds master (3.5.1+) Created commit 1e4c04a in eds gnome-3-4 (3.4.2+)