GNOME Bugzilla – Bug 547763
Logs from disk summary db monitoring
Last modified: 2009-01-29 16:25:02 UTC
Attaching from the first run: - startup in --offline mode - go online and fetch a few messages - read some mail in one folder - switch to another folder and read more mail - then I hit ctrl+a/ctrl+d in a folder with 62k messages in it and it just started hanging for a few minutes but recovered after that - another fetch mail operation was kicked off (fetching mail is way too slow compared with earlier) - this hung on message 20, and I cancelled it - another fetch mail operation was started and this one finished successfully - expunge in the folder with 62k messages after having marked them all 1) The expunge looks like it deletes messages one by one from the db which is crazy when all messages are marked for deletion. 2) Fetching mail is very slow compared to earlier. I had something like tens of messages downloaded per second earlier while it now uses seconds per message at the worst 3) The expunge operation above takes long enough to be victim of another "fetch mail" operation to kick in and fight for disk/IO which makes it all break down further
Created attachment 116574 [details] log from e-d-s/camel with debugging enabled Logs from evo with debugging from e-d-s trunk
Awesome data. I think I will give you some more test patches. Thanks a lot for your data. Tell me, do you have some filter or something that archives messages locally? Meaning copying message to a folder or something?
I've got about 40 of them :-)
Fixed majority of the issue. Feel free to add more, if there are anything. Thanks for your help.
So, I'm running 2.23.91 now and I still think fetching mail is very slow compared to before the sqlite migration. Anything I can do to provide more data? Just more of the same as before?
kmaraas, any logging or anything you run with? I know I have a few debug logs, which can slow things a bit. Also, Is filtering that is slow or fetching mails? Also, Now we don't store immediately. We do bulk commit to db. So should be mostly fine. Anyways.. if you have any console o/p that will be useful.
I think I know of one place... we do check the db for every mail that we get that if uid present in db already. I must move it to memory. How many mails you receive a day?
I get a good chunk of mail. 1000-1500 messages a day from various mailing-lists which all are filtered out in different folders. Attaching a new log from fetching around 300 messages.
Created attachment 118376 [details] new log file from fetching ~300 messages
I will keep this bug open to fix more perf issues.
Too many such bugs open. Closing it.