After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 738453 - Inefficient sort by subject in message list
Inefficient sort by subject in message list
Status: RESOLVED FIXED
Product: evolution
Classification: Applications
Component: Mailer
3.12.x (obsolete)
Other Linux
: Normal major
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
Depends on:
Blocks:
 
 
Reported: 2014-10-13 12:26 UTC by Mart Raudsepp
Modified: 2014-10-15 17:46 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Memory leaks after subject sorting a 1532 mail folder (114.00 KB, application/octet-stream)
2014-10-14 23:27 UTC, Mart Raudsepp
Details

Description Mart Raudsepp 2014-10-13 12:26:52 UTC
Sorting a huge maildir by subject in 3.12.6 and 3.12.7 can result in gigabytes of memory usage in leaks. I can end up at 14GB before the computer dies a horribly swapping/OOM death on folders with 300k mails.
Unfortunately https://git.gnome.org/browse/evolution/commit/?h=evolution-3-12&id=ffa74c5b96c03d63518d48dfaba09fbac30590a5 didn't fix it (at least to the point it wouldn't swapdie), as I'm on todays releases of eds, gtkhtml and evo now.
Comment 1 Milan Crha 2014-10-13 15:25:11 UTC
Thanks for a bug report. I'm not able to reproduce this, somehow. I use evolution*-3.12 branches, pretty much the same as today's 3.12.7 release and there is nothing really claimed as 'definitely' lost related to the message list sorting by valgrind when I run evolution under it and sort a small folder (167 messages) by subject there, threaded or not threaded view. The command I use (with debug info for at least evolution-data-server and evolution available/installed):

   $ G_SLICE=always-malloc valgrind --num-callers=20 --leak-check=full \
        evolution &>leak.log

Evolution will be significantly slow, thus count with it. Also, there is no need to try this on a huge folder, if the memory is really lost, then it'll be noticed by the valgrind in smaller folders too. Make sure the application will be closed properly, otherwise the numbers and the leak places might not be accurate. Then attach the resulting log here, please.
Comment 2 Mart Raudsepp 2014-10-14 23:27:40 UTC
Created attachment 288556 [details]
Memory leaks after subject sorting a 1532 mail folder
Comment 3 Milan Crha 2014-10-15 08:25:05 UTC
Thanks for the log. Even the debug information is missing for evolution the log doesn't show anything really leaking in the related part, from which I'd guess that some sort-value-caching grows too high, reaches the memory limit and the killer kills the application. That's what I would understand from the log.
Comment 4 Mart Raudsepp 2014-10-15 10:18:11 UTC
oops, I had debugs off for an earlier webkit build; really should convert those to per-package. I'll grab the .7.1 eds and rebuild that stack with full debug and re-run soon.
Comment 5 Milan Crha 2014-10-15 17:46:54 UTC
I think I found the culprit, there were multiple of them:
a) each row required an open & close of a GSettings with a read of the value,
   twice or more times - this caused a huge D-Bus buzz too
b) the memory pool of the subjects was never freed, it was only growing, thus
   it could result in state of having all subjects from all folders in all
   accounts stored in memory and ready to use, regardless it not being needed
   at all
c) finally, some repeating allocations and deallocations had been done
   unnecessarily

All together the sort-by-subject for 100K+ messages could cause easily a high CPU usage (multiple cores due to a)) and a waste of memory in a time. I changed this with below commit, which fills the tree with 150K messages in about 3-5 seconds, with compare of like 20-30 seconds of the fill plus a fallout of the D-Bus buzz which I never let to settle, because it wasn't done even after a minute (time values are subjective).

Created commit c0da2e6 in evo master (3.13.7+) [1]
Created commit 44a9f95 in evo evolution-3-12 (3.12.8+)

[1] https://git.gnome.org/browse/evolution/commit/?id=c0da2e6