GNOME Bugzilla – Bug 787514
[IMAPx] Overaggressive folder change saving after filtering
Last modified: 2017-09-20 09:41:08 UTC
evolution-3.25.92.2-1.fc27.x86_64 evolution-data-server-3.25.92-1.fc27.x86_64 1. Select some files in my inbox 2. Press Ctrl+Y and wait for those mails to be filtered into my NAS 3. "Filtering new messages 100%" shows up in the status bar, and stays there while evolution eats close to 100% CPU And stays there.
Thanks for a bug report. It's hard to tell why it does so without any further information. Could you catch a backtrace of the running evolution in this state, to see what it does, please? Please install debuginfo packages for evolution-data-server and evolution, to have the backtrace usable. You can get the backtrace with command like this: $ gdb --batch --ex "t a a bt" -pid=`pidof evolution` &>bt.txt Please check the bt.txt for any private information, like passwords, email address, server addresses,... I usually search for "pass" at least (quotes for clarity only).
This seems to be the only thread not stuck in poll() or g_cond_wait() in most of my backtraces.
+ Trace 237963
Thread 21 (Thread 0x7f0f990b3700 (LWP 22011))
sqlite-3.20.1-1.fc27.x86_64
sqlite is saving changes from memory to the disk there. There used to be an issue with fsync() calls in the past, on certain partition types, where the fsync() flushed not only that file, but everything from any application. I do not recall many details, thus it can be partly distorted. The On This Computer account is storing its data into ~/.local/share/evolution/mail/local/folders.db The other (remote) accounts usually at ~/.cache/evolution/mail/<account-UID>/folders.db I suppose either of them is large (several GBs? mine largest has only 182MB). $ ls -hl ~/.cache/evolution/mail/*/folders.db > 2. Press Ctrl+Y and wait for those mails to be filtered into my NAS I'm sorry, but what do you call "NAS" here? I suppose all the folders.db files are on your local computer, or not?
(In reply to Milan Crha from comment #3) > sqlite is saving changes from memory to the disk there. There used to be an > issue with fsync() calls in the past, on certain partition types, where the > fsync() flushed not only that file, but everything from any application. I > do not recall many details, thus it can be partly distorted. I'm using ext4. > The On This Computer account is storing its data into > > ~/.local/share/evolution/mail/local/folders.db I don't use it in this case. > The other (remote) accounts usually at > > ~/.cache/evolution/mail/<account-UID>/folders.db > > I suppose either of them is large (several GBs? mine largest has only 182MB). > > $ ls -hl ~/.cache/evolution/mail/*/folders.db No, the largest is 450 megs. > > 2. Press Ctrl+Y and wait for those mails to be filtered into my NAS > > I'm sorry, but what do you call "NAS" here? I suppose all the folders.db > files are on your local computer, or not? a Networked Attached Storage. I have a remote IMAP server hosted by my provider, and a local one on the network with folders and such. evolution has been running at 100% CPU for more than an hour now, how long is it supposed to be stuck at "Filtering Selected Messages 100%" and eating 100% CPU? The full stack trace:
+ Trace 237982
Thread 26 (Thread 0x7fb0b57fa700 (LWP 3297))
Thread 25 (Thread 0x7fb09f7fe700 (LWP 1768))
Thread 24 (Thread 0x7fb13bfff700 (LWP 386))
Thread 23 (Thread 0x7fb0b5ffb700 (LWP 32400))
Thread 22 (Thread 0x7fb158cfe700 (LWP 31657))
Thread 21 (Thread 0x7fb0b7fff700 (LWP 31597))
Thread 20 (Thread 0x7fb0b77fe700 (LWP 31098))
Thread 19 (Thread 0x7fb0b6ffd700 (LWP 30676))
Thread 18 (Thread 0x7fb1594ff700 (LWP 23248))
Thread 17 (Thread 0x7fb0d0371700 (LWP 22157))
This time with: evolution-3.26.0-1.fc27.x86_64 evolution-data-server-3.26.0-1.fc27.x86_64
(In reply to Bastien Nocera from comment #4) > No, the largest is 450 megs. Okay, not so small, but as you have your ~/.cache/evolution ont he local machine, it should not be a problem. > evolution has been running at 100% CPU for more than an hour now, how long > is it supposed to be stuck at "Filtering Selected Messages 100%" and eating > 100% CPU? Definitely not that long. The backtrace shows one thread in IMAP IDLE call, which is fine. Then several (6) threads with just-finished filtering, waiting in camel_folder_synchronize_sync() for the other call to finish. There is another similar thread (seventh, Thread 17), which does store local changes and refreshes with remote changes. You still didn't install debuginfo for evolution-data-server and evolution (see comment #1), thus I cannot tell where exactly this is the problem. I know roughly where it is, but there are two places and different circumstances which can eventually reproduce this. It can even be that all your messages in certain folder changed, maybe in the Inbox, because it's where the filters had been probably running, while the server changed UID validity, which means that the whole folder content should be fetched again.
This is with debuginfo installed:
+ Trace 237984
Thread 24 (Thread 0x7fb13bfff700 (LWP 5153))
And with the right version of debuginfo installed:
+ Trace 237985
Thread 24 (Thread 0x7fb09ffff700 (LWP 5732))
This might be related to changes from bug #782844. I tried to address it with the below change. The change allows cancelling of the filtering at the stage where you've got the backtraces (at the end of the filtering, when the changes had been asked to be propagated to the server and the folder refreshed), thus it should do something now when you request the filtering to cancel. I also made a test package for you at: https://koji.fedoraproject.org/koji/taskinfo?taskID=21972040 Created commit dbd70b9f0 in eds master (3.27.1+) Created commit 083ca6b08 in eds gnome-3-26 (3.26.1+)
That seems to work correctly. After filtering, the summary of the remote folder is fetched in a timely manner, and operations stop as expected. Thanks!
Great. Thanks for a quick testing.