GNOME Bugzilla – Bug 738453
Inefficient sort by subject in message list
Last modified: 2014-10-15 17:46:54 UTC
Sorting a huge maildir by subject in 3.12.6 and 3.12.7 can result in gigabytes of memory usage in leaks. I can end up at 14GB before the computer dies a horribly swapping/OOM death on folders with 300k mails. Unfortunately https://git.gnome.org/browse/evolution/commit/?h=evolution-3-12&id=ffa74c5b96c03d63518d48dfaba09fbac30590a5 didn't fix it (at least to the point it wouldn't swapdie), as I'm on todays releases of eds, gtkhtml and evo now.
Thanks for a bug report. I'm not able to reproduce this, somehow. I use evolution*-3.12 branches, pretty much the same as today's 3.12.7 release and there is nothing really claimed as 'definitely' lost related to the message list sorting by valgrind when I run evolution under it and sort a small folder (167 messages) by subject there, threaded or not threaded view. The command I use (with debug info for at least evolution-data-server and evolution available/installed): $ G_SLICE=always-malloc valgrind --num-callers=20 --leak-check=full \ evolution &>leak.log Evolution will be significantly slow, thus count with it. Also, there is no need to try this on a huge folder, if the memory is really lost, then it'll be noticed by the valgrind in smaller folders too. Make sure the application will be closed properly, otherwise the numbers and the leak places might not be accurate. Then attach the resulting log here, please.
Created attachment 288556 [details] Memory leaks after subject sorting a 1532 mail folder
Thanks for the log. Even the debug information is missing for evolution the log doesn't show anything really leaking in the related part, from which I'd guess that some sort-value-caching grows too high, reaches the memory limit and the killer kills the application. That's what I would understand from the log.
oops, I had debugs off for an earlier webkit build; really should convert those to per-package. I'll grab the .7.1 eds and rebuild that stack with full debug and re-run soon.
I think I found the culprit, there were multiple of them: a) each row required an open & close of a GSettings with a read of the value, twice or more times - this caused a huge D-Bus buzz too b) the memory pool of the subjects was never freed, it was only growing, thus it could result in state of having all subjects from all folders in all accounts stored in memory and ready to use, regardless it not being needed at all c) finally, some repeating allocations and deallocations had been done unnecessarily All together the sort-by-subject for 100K+ messages could cause easily a high CPU usage (multiple cores due to a)) and a waste of memory in a time. I changed this with below commit, which fills the tree with 150K messages in about 3-5 seconds, with compare of like 20-30 seconds of the fill plus a fallout of the D-Bus buzz which I never let to settle, because it wasn't done even after a minute (time values are subjective). Created commit c0da2e6 in evo master (3.13.7+) [1] Created commit 44a9f95 in evo evolution-3-12 (3.12.8+) [1] https://git.gnome.org/browse/evolution/commit/?id=c0da2e6