GNOME Bugzilla – Bug 625609
housekeeping: Needs to be a bit more clever
Last modified: 2013-09-30 15:39:51 UTC
The original bug report for this is at: https://bugs.launchpad.net/ubuntu/+source/gnome-settings-daemon/+bug/597512 The Ubuntu guys asked me to post it here too; I couldn't see any similar bugs in the current list. When I log into my laptop I regularly see huge CPU usage of which only a small part is doing anything useful and 75% is spent in I/O wait. The culprit is the gnome-settings-daemon, and the whole system is unusable until it stops what it's doing. What it appears to be doing according to strace is calling lstat() on thousands and thousands and thousands of stored thumbnail images, as fast as it can do so. This prevents anything else from accessing the disk. I presume you're trying to find the oldest files and delete them, which makes sense. But I believe there are a number of problems here: 1. Calling lstat() on thousands and thousands of files without a break is going to completely trash system performance unless you have an SSD, at least on ext3; newer filesystems may be more efficient at directory reads but I don't think you can rely on that. This is particularly problematic when it's run when the user logs in and is eager to use the computer to do something as they have to wait thirty seconds or more before it's usable. 2. I suspect this performance hit is made even worse when using ecryptfs, since that's adding an extra layer or three between lstat() and the actual hard disk. 3. I'm sure I read somewhere that Ubuntu set the thumbnail lifetime higher than other Linux distros, so it's going to have even more files to scan. 4. There's also the question of whether you really want to be putting 20,000+ thumbnail files into a single directory in the first place? Putting them into multiple directories and scanning individual directories on an intermittent basis would seem more sensible. This doesn't happen every time I log in, but when it does happen it's painful.
That's the code in the housekeeping plugin that purges old files, and trims the size of the thumbnail cache. See plugins/housekeeping/gsd-housekeeping-manager.c:purge_thumbnail_cache(). The code to cleanup on shutdown is also utterly stupid, as it will list, and sort the files, before removing every single one of them. This code could really do with moving to GIO... About 4), that still wouldn't fix the problem of listing and removing those files...
Downstreambug on launchpad: https://bugs.launchpad.net/gnome-settings-daemon/+bug/505085
At the very least, purge_thumbnail_cache() could check whether "max_age" and "max_size" are both < 0 BEFORE actually reading the thumnail dirs! If it did, you could at least disable thumbnail purging by setting both to -1 with gconf-editor. Presently, the only way to avoid a performance hit (e.g. with a large thumbnail cache under ecryptfs) seems to be disabling the whole housekeeping plugin.
Created attachment 191205 [details] iotop screenshot
How can I disable the plugin while this isn't solved? gnome-settings-daemon makes my system almost unresponsive for about 8 minutes on every startup… $ find .thumbnails | wc 30486 30486 1736812 $ du -sm .thumbnails 819 .thumbnails
workaround: start gconf-editor and untick /apps/gnome_settings_daemon/plugins/housekeeping/active Apparently this will also disable the low disk space check and warning (and maybe something else as well?), but I guess this isn't essential...
Thanks! For me it was in dconf, /org/gnome/settings-daemon/plugins/housekeeping/ I wasn't finding it in gconf, so I checked the files the package installed, found the org.gnome.settings-daemon.plugins.housekeeping.gschema.xml and inside there's the path="/org/gnome/settings-daemon/plugins/housekeeping/". :) I should have said I'm using the version 3.1.2-0ubuntu1 of gnome-settings-daemon, Ubuntu oneiric (development branch).
Noob suggestion: why not fix this bug just by runnig the house keeping thread with an appropriate ionice priority ?
Still problem in GNOME 3.2. Someone with sufficient privileges (like the reporter), please change Version to 3.2 or to master. Thanks.
(In reply to comment #9) > Still problem in GNOME 3.2. Someone with sufficient privileges (like the > reporter), please change Version to 3.2 or to master. Thanks. There's no need to do that. If the bug wasn't current anymore, it would be closed.
Created attachment 256088 [details] [review] patch to properly allow disabling of thumbnail cache cleaning Per comment #3, the attached patch checks if max_age and max_size are both -1, ie thumbnail cleaning is disabled, before scanning the thumbnail dirs. This enables the performance hit to be removed through a simple configuration option.
Review of attachment 256088 [details] [review]: Looks good otherwise for gnome-3-8 and master. ::: gnome-settings-daemon.orig/plugins/housekeeping/gsd-housekeeping-manager.c @@ +230,3 @@ + + /* if both are set to -1, we don't need to read anything */ + if ((purge_data.max_age < 0) && (purge_data.max_size < 0)) No need for the extra brackets.
Created attachment 256099 [details] [review] housekeeping: Optimise for "do nothing" when cleaning thumbnails If max_age and max_size are both -1, thumbnail cleaning is disabled, so there is no need to scan the cache directories. https://bugzilla.gnome.org/show_bug.cgi?id=625609 https://bugs.launchpad.net/ubuntu/+source/gnome-settings-daemon/+bug/505085
Review of attachment 256088 [details] [review]: > The referenced bugs concern a serious > performance issue at login, which is caused by the > thumbnail cache scan. You might be looking in the wrong place for this, the purge only runs 2 minutes after g-s-d's startup. I've removed this from the commit message. Furthermore, please attach patches in git format, not whatever format Ubuntu packages use, please.
Attachment 256099 [details] pushed as 12f2121 - housekeeping: Optimise for "do nothing" when cleaning thumbnails