After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 552705 - Memory use explosion generating mail summaries
Memory use explosion generating mail summaries
Status: RESOLVED FIXED
Product: evolution
Classification: Applications
Component: Mailer
2.24.x (obsolete)
Other Linux
: Normal major
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
evolution[disk-summary]
Depends on:
Blocks: 543389
 
 
Reported: 2008-09-17 23:38 UTC by Hans Petter Jansson
Modified: 2009-01-31 14:14 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
VFolder rule that causes memory exhaustion. (2.10 KB, application/xml)
2008-10-16 19:11 UTC, Hans Petter Jansson
Details

Description Hans Petter Jansson 2008-09-17 23:38:06 UTC
Evolution's memory use increases during startup, while showing "Storing folder <folder>..." and "Updating vFolders for <folder>...", until my computer runs out of core memory and swap.

I'm using evolution-2.23.92+r36353 built from the GNOME:Factory project in openSUSE's build service.

Typical terminal output:

Triggering summary_reloaded on Organizations/Novell/SuSE Research 0xc253ac8
Triggering summary_reloaded on Organizations/Novell/SuSE Research 0xc253ac8
Triggering summary_reloaded on Organizations/Novell/SuSE Research 0xc253ac8
Triggering summary_reloaded on Organizations/Novell/SuSE Research 0xc253ac8
Triggering summary_reloaded on Organizations/Novell/SuSE Research 0xc253ac8
Saving 13181/13181 dirty records of Organizations/Novell/SuSE Research
removing cache for  Organizations/Novell/Ximian Desktop 4147 0xc385418
done .. now 4147
removing cache for  Organizations/Novell/NLD General 9136 0x94a0d38
done .. now 9136
removing cache for  Organizations/Novell/Novell Bugzilla Team 24619 0x8ecd2d8
done .. now 24619
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Triggering summary_reloaded on Organizations/Novell/Ximian Services 0x120d0c00
Saving 209/209 dirty records of Organizations/Novell/Ximian Services
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
removing cache for  Organizations/Novell/Ximian Consultants 110 0xc7fea08
done .. now 110
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Triggering summary_reloaded on Organizations/Novell/Ximian Mexico 0x12223ea0
Saving 804/804 dirty records of Organizations/Novell/Ximian Mexico
removing cache for  Organizations/Novell/NLD Development 1907 0xc3d3788
done .. now 1907
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
removing cache for  Organizations/Novell/Ximian HP External 191 0x86e1830
done .. now 191
removing cache for  Organizations/Novell/Active Directory 304 0x9395520
done .. now 304
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
removing cache for  Organizations/Novell/Ximian General 1551 0x8fde720
done .. now 1551
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
removing cache for  Organizations/Novell/Ximian Hackers 1107 0xa10db58
done .. now 1107
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
removing cache for  Organizations/Novell/Ximian Words 832 0x8d88e70
done .. now 832
removing cache for  Organizations/Novell/Novell Bugzilla Personal 4931 0xa5a79c0
done .. now 4931
removing cache for  Organizations/Novell/People 5764 0xcbf5488
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
done .. now 5764
removing cache for  Organizations/Novell/Ximian HP Patches 370 0x909fc08
done .. now 370
removing cache for  Organizations/Novell/NLD Patches 377 0x9122860
done .. now 377
removing cache for  Organizations/Novell/Ximian Spidermonkey 2469 0x8bd0a28
done .. now 2469
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
removing cache for  Organizations/Novell/SuSE MBuild 461 0xbe29400
done .. now 461
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
Triggering summary_reloaded on Organizations/Novell/Ximian HP Internal 0x12463718
removing cache for  Organizations/Novell/Ximian Dung 1699 0xbef87b0
done .. now 1699
Saving 512/512 dirty records of Organizations/Novell/Ximian HP Internal
removing cache for  Organizations/Novell/SuSE Bugzilla 20 0xa749090
done .. now 20
removing cache for  Organizations/Novell/Ximian Desktop 4147 0xc385418
done .. now 4147
removing cache for  Organizations/Novell/NLD General 9136 0x94a0d38
done .. now 9136
removing cache for  Organizations/Novell/Novell Bugzilla Team 24619 0x8ecd2d8
done .. now 24619
removing cache for  Organizations/Novell/Ximian Consultants 110 0xc7fea08
done .. now 110
removing cache for  Organizations/Novell/NLD Development 1907 0xc3d3788
done .. now 1907
Comment 1 Srinivasa Ragavan 2008-09-22 03:35:14 UTC
HPJ, What is the approx folders count and per folder mail size? Lemme valgrind it more.
Comment 2 Hans Petter Jansson 2008-09-22 05:10:53 UTC
Total of 3.5GB of mail in ~160 folders.
Comment 3 Srinivasa Ragavan 2008-10-06 11:12:10 UTC
This is mostly to do with the number of folders. We had one sqlite handle per folder and hte cache ate this up. I have redesigned a bit and we now use two handles per store and should be lot fine now. on stable/trunk
Comment 4 Srinivasa Ragavan 2008-10-13 06:44:19 UTC
HPJ, did you try at this?
Comment 5 Hans Petter Jansson 2008-10-14 23:35:42 UTC
I just bought some more RAM so I can build Evolution in a separate VM and test there. Expect results later this week.
Comment 6 Hans Petter Jansson 2008-10-16 19:09:38 UTC
Just tested this again with trunk - one of my vFolders is causing the memory exhaustion. It has a very heavy rule, doing full-text searches on all my local folders, but it used to work (albeit slowly).
Comment 7 Hans Petter Jansson 2008-10-16 19:11:09 UTC
Created attachment 120740 [details]
VFolder rule that causes memory exhaustion.
Comment 8 Srinivasa Ragavan 2008-10-17 03:20:54 UTC
<part name="body">
<value name="body-type" type="option" value="contains"/>
<value name="word" type="string">
<string>Petter</string>
</value>
</part>

This (body-contains search) defaults to the old mode of searching than the sqlite disk search, by loading contents to memory and search. HPJ, even then, it just must be the first time. Since vfolder results are cached. Second restart should be much better afaics

In effect, first time, it would load the entire local store's summaries and build a vfolder. It will be expensive. 
Comment 9 Srinivasa Ragavan 2008-11-04 04:00:35 UTC
Is it better second time or its the samething everytime ?
Comment 10 Srinivasa Ragavan 2008-11-17 09:28:13 UTC
Any updates?
Comment 11 Hans Petter Jansson 2009-01-18 16:55:49 UTC
If I repeatedly stop and start Evolution until it's done building indexes or whatever it's doing, it will eventually start up without exhausting available memory.
Comment 12 Hans Petter Jansson 2009-01-18 18:55:52 UTC
However, see bug 568199. That one happens every time I shut down.
Comment 13 Hans Petter Jansson 2009-01-18 21:05:59 UTC
Part of the problem is that I don't have enough RAM to run Evolution anymore. With Evolution 2.22 and the same mail folders, the index is much smaller on disk and I can run it comfortably without running out of RAM.

So this is a regression.
Comment 14 Srinivasa Ragavan 2009-01-29 16:41:31 UTC
I hope 2.24.3 is better for you, keep this open for some time, till you think its all fine. Thanks HPJ
Comment 15 Hans Petter Jansson 2009-01-31 14:14:59 UTC
Version 2.24.3 seems to have fixed this issue for me. Very nice!