After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 547763 - Logs from disk summary db monitoring
Logs from disk summary db monitoring
Status: RESOLVED FIXED
Product: evolution-data-server
Classification: Platform
Component: Mailer
2.24.x (obsolete)
Other Linux
: Normal normal
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
evolution[disk-summary]
Depends on:
Blocks: 543389
 
 
Reported: 2008-08-14 13:03 UTC by Kjartan Maraas
Modified: 2009-01-29 16:25 UTC
See Also:
GNOME target: ---
GNOME version: 2.23/2.24


Attachments
log from e-d-s/camel with debugging enabled (759.90 KB, application/octet-stream)
2008-08-14 14:04 UTC, Kjartan Maraas
Details
new log file from fetching ~300 messages (96.94 KB, text/plain)
2008-09-09 18:58 UTC, Kjartan Maraas
Details

Description Kjartan Maraas 2008-08-14 13:03:21 UTC
Attaching from the first run:

- startup in --offline mode
- go online and fetch a few messages
- read some mail in one folder
- switch to another folder and read more mail
- then I hit ctrl+a/ctrl+d in a folder with 62k messages in it and it just started hanging for a few minutes but recovered after that
- another fetch mail operation was kicked off (fetching mail is way too slow compared with earlier)
- this hung on message 20, and I cancelled it
- another fetch mail operation was started and this one finished successfully
- expunge in the folder with 62k messages after having marked them all

1) The expunge looks like it deletes messages one by one from the db which is crazy when all messages are marked for deletion.

2) Fetching mail is very slow compared to earlier. I had something like tens of messages downloaded per second earlier while it now uses seconds per message at the worst

3) The expunge operation above takes long enough to be victim of another "fetch mail" operation to kick in and fight for disk/IO which makes it all break down further
Comment 1 Kjartan Maraas 2008-08-14 14:04:46 UTC
Created attachment 116574 [details]
log from e-d-s/camel with debugging enabled

Logs from evo with debugging from e-d-s trunk
Comment 2 Srinivasa Ragavan 2008-08-14 18:31:12 UTC
Awesome data. I think I will give you some more test patches. Thanks a lot for your data.

Tell me, do you have some filter or something that archives messages locally? Meaning copying message to a folder or something? 
Comment 3 Kjartan Maraas 2008-08-14 23:29:31 UTC
I've got about 40 of them :-)
Comment 4 Srinivasa Ragavan 2008-08-18 09:38:44 UTC
Fixed majority of the issue. Feel free to add more, if there are anything. Thanks for your help.
Comment 5 Kjartan Maraas 2008-09-08 12:37:27 UTC
So, I'm running 2.23.91 now and I still think fetching mail is very slow compared to before the sqlite migration. Anything I can do to provide more data? Just more of the same as before?
Comment 6 Srinivasa Ragavan 2008-09-09 03:20:04 UTC
kmaraas, any logging or anything you run with? I know I have a few debug logs, which can slow things a bit. 

Also, Is filtering that is slow or fetching mails? Also, Now we don't store immediately. We do bulk commit to db. So should be mostly fine. Anyways.. if you have any console o/p that will be useful.
Comment 7 Srinivasa Ragavan 2008-09-09 03:21:20 UTC
I think I know of one place... we do check the db for every mail that we get that if uid present in db already. I must move it to memory. How many mails you receive a day?
Comment 8 Kjartan Maraas 2008-09-09 18:56:59 UTC
I get a good chunk of mail. 1000-1500 messages a day from various mailing-lists which all are filtered out in different folders. Attaching a new log from fetching around 300 messages.
Comment 9 Kjartan Maraas 2008-09-09 18:58:14 UTC
Created attachment 118376 [details]
new log file from fetching ~300 messages
Comment 10 Srinivasa Ragavan 2008-09-11 06:40:00 UTC
I will keep this bug open to fix more perf issues.
Comment 11 Srinivasa Ragavan 2009-01-29 16:25:02 UTC
Too many such bugs open. Closing it.