GNOME Bugzilla – Bug 570438
Infinite wait on "Filtering new nessages"
Last modified: 2013-09-26 11:54:20 UTC
Please describe the problem: Sometimes I get the "Filtering new messages(s) (0% complete)" for very long periods (an hour or more before killing Evo). Since the message doesn't mention the server, it's hard to know why, but I have 1 POP and 2 IMAP accounts. The POP account is very low traffic. One IMAP account is with Gmail, but is configured not to run filters. The other IMAP account appears to work normally i.e. I can visit folders, read and send messages and even receive new mail etc., just that the new mail is not being filtered, including Junk. This happens sporadically with no obvious pattern. Steps to reproduce: 1. Visit IMAP account 2. See "Filtering new messages" status message 3. Wait forever Actual results: Eternal wait Expected results: Quick filtering of new messages and disappearance of staus message Does this happen every time? No Other information: Running 2.24.4 from Fedora Koji (pre updates-testing). Specific packages: evolution-debuginfo-2.24.4-1.fc10.x86_64 evolution-webcal-2.23.91-1.fc10.x86_64 evolution-conduits-2.24.4-1.fc10.x86_64 evolution-data-server-debuginfo-2.24.4.1-1.fc10.x86_64 evolution-data-server-2.24.4.1-1.fc10.x86_64 evolution-spamassassin-2.24.4-1.fc10.x86_64 evolution-data-server-doc-2.24.4.1-1.fc10.x86_64 evolution-help-2.24.4-1.fc10.x86_64 evolution-bogofilter-2.24.4-1.fc10.x86_64 evolution-2.24.4-1.fc10.x86_64
Further info: on quitting Evo, the messages filtering suddenly wakes up (you can see it faintly in the staus bar). After restarting, all the messages have been appropriately filtered. So it looks like a locking problem of some sort.
It would be great if you can provide gdb traces of evolution when you see filtering in status bar which will let us know what is going on, thanks. (see http://live.gnome.org/GettingTraces/Details#gdb-not-yet-running for details about how to do this)
Patrick, I guess your problem could be that the db might be too bad. Just vaccum it once. You can use this script: http://www.gnome.org/~sragavan/evolution-rebuild-summarydb I'm planning to do it programatic.
Thanks Srini. I ran the script and we'll see what happens. In fact it was only after running it that I saw several new messages, including the posts from you and Akhil! If I don't complain again, assume it worked :-)
It didn't work, so I'm complaining again :-( In fact it has happened three times in a couple of hours, which seems to be more often than before. Usually I could just quit and restart, and it would recover, but the last time it simply hung indefinitely while quitting and I had to use --force-shutdown. I'm willing to run with logging and/or gdb if you tell me what you need.
Sorry to be insistent, but this formerly sporadic problem is now happening to me *all the time*, i.e. my impression is that filters are being run on new messages when starting up Evo, but the indefinite wait prevents them from being triggered afterward (so I miss some new messages until I notice). I've rebuilt the SQL db several times using Srini's script but it doesn't seem to help this particular problem.
Patrick run on gdb and when u see the hang break into gdb and do 't a a bt' and collect the traces. 2 - 3 samples would help me. Thanks
Created attachment 128032 [details] gdb thread trace taken while Evo in "Filtering new message(s) (0%)" state Here's a trace. I'll send a couple more as they become available.
Created attachment 128033 [details] gdb thread trace taken while Evo in "Filtering new message(s) (0%)" state Second trace
Created attachment 128048 [details] gdb thread trace taken while Evo in "Filtering new message(s) (0%)" state Third and last trace.
I turned off automatic filtering (both normal and Junk) and now apply filters manually as required. This has worked every time so far with no hangs.
Patrick, Awesome.
+ Trace 212340
Thread 2 (Thread 0x7fae75931950 (LWP 23324))
So, in Mail Preferences, under Junk you would have a option, "Do not mark messages as junk if sender is in address book" Disable it, it would solve. Your junk check on addressbook is taking time. Do you have a remote addressbook or ldap addressbook? That should be the culprit. Try using the option 'lookup in local addressbook only'
Thanks Srini, that's well-spotted. I do indeed check against a remote LDAP instance. I'll disable that and see how it goes. Nonetheless, I don't quite see why this affects the automatic filtering but not the manual kind. Surely they both use the same rules? I'll try it anyway.
Actually you have 'check for junk'. It uses the same thing. But when you manually do filtering, you don't do junk-test.
I may not have been clear. I was also applying Junk filtering manually.
After a week or so of happy mailing with no problems, this issue (or a very similar one) has reared its head again. Twice today Evo has shown "Filtering new messages(s) (50% complete)" (note that it's now 50%. not 0% as before). On quitting Evo, it just hangs and does nothing. The first time, I ran gdb on the process, did a trace and quit gdb, whereupon Evo woke up and finished. The second tjime, I did the same thing but this time it didn't wake up so I had to use --force-shutdown. I'm attaching the trace of this second instance. I should emphasize that I haven't altered my config in any way. Particularly, the "check non-local address books" setting for Junk is still off.
Created attachment 128925 [details] gdb trace of Evo hanging during Quit
+ Trace 212652
The trace says that junk test is stil looking in addressbook. But its for a local book. Either EDS is hung or died.
Luckily I still had the terminal session where I did this, so I could scroll back to where I listed the processes: % pgrep -fl evol 9047 /usr/libexec/evolution-data-server-2.24 --oaf-activate-iid=OAFIID:GNOME_Evolution_DataServer_CalFactory:1.2 --oaf-ior-fd=20 9692 /usr/bin/evolution 9728 /usr/libexec/evolution/2.24/evolution-alarm-notify --oaf-activate-iid=OAFIID:GNOME_Evolution_Calendar_AlarmNotify_Factory:2.24 --oaf-ior-fd=18 17855 /usr/libexec/evolution-data-server-2.24 --oaf-activate-iid=OAFIID:GNOME_Evolution_DataServer_InterfaceCheck --oaf-ior-fd=22 As you can see, there are two e-d-s procs running. Presumably that's wrong. Of course at this point I don't know if either or both of them was hung. If it happens again I'll check this.
Could you please confirm if this bug is still happening at your end ? Please try in 2.28.x or 2.30.0 and report back, thanks.
Hasn't happened in several months, possibly since upgrading to 2.28.
And we're back. Now on 3.2.2 (Fedora 16). The problem has been happening very frequently -- more than once a day -- and Evo can only be killed by zapping the process, i.e. it ignores the usual Quit. However the hang is not complete because some thread does seem to update the folder view periodically. During the hang, there are two messages in the notification area: "Formatting Message" and "Filtering New Mail". I turned off automatic checking of subscribed folders and the frequency has dropped considerably (several days without problems). However it hasn't gone away, so I've captured a trace and will upload it. A change from my original reports is that I now have 1 POP account (very low traffic), 1 public Gmail account, 1 Google Apps for Education account and 1 Google Apps for Business account. All the Google/Gmail accounts are IMAP. The hang does not happen with the POP account, only with the others.
Re-submitting with correct Evo version, sorry.
Created attachment 204220 [details] Traceback of hung session
(In reply to comment #22) > And we're back. Now on 3.2.2 (Fedora 16). The problem has been happening very > frequently -- more than once a day -- and Evo can only be killed by zapping the > process, i.e. it ignores the usual Quit. However the hang is not complete > because some thread does seem to update the folder view periodically. > > During the hang, there are two messages in the notification area: "Formatting > Message" and "Filtering New Mail". Both tasks (Thread 7 and Thread 9) are stuck trying to compare email addresses with contacts from your address books, but it looks like the address book D-Bus service is not responding. Perhaps it crashed or deadlocked? Would be helpful if you can attach gdb to your e-addressbook-factory process ("gdb --pid $PID") and, next time Evolution gets stuck, stop the address book process if necessary and get a backtrace.
OK, I'll do that. Thanks Matt.
What I did was turn off an LDAP contact server which is sometimes unreliable, and the problem seems to have gone away for the moment. I don't really need that server so I'll probably just leave it that way unless the problem reappears.
Thanks, that would explain it. The mailer/contact integration bits are still not completely asynchronous.
I'm afraid this is not about asynchronicity. The filter is run one after an other, not in parallel. And if one is stuck, then the rest is waiting. There was a similar issue with "Formatting message", the preview panel was locked due to similar issue. Is the option from comment #12 on or off? If it's off, then probably other option invokes this addressbook checking, or there's a bug somewhere.
Patrick, do you still see this, say with at least 3.6.4, please?
I haven't seen this in quite some time, but IIRC I stopped using the flaky LDAP server over a year ago, so that may or may not be useful information. I just updated to 3.6.4 yesterday.
Closing as per comment 31.