GNOME Bugzilla – Bug 680172
Tracker can end up eating >3.8 GiB of disk for meta.db
Last modified: 2012-08-10 13:58:21 UTC
For some unknown reason, .config/user-dirs.dirs was pointing all dirs to be $HOME/. When org.freedesktop.Tracker.Miner.Files index-recursive-directories is ['&DESKTOP', '&DOCUMENTS', '&DOWNLOAD', '&MUSIC', '&PICTURES', '&VIDEOS'] And index-single-directories is ['$HOME'], this caused tracker to index massive amounts of data, including jhbuild checkouts and other projects. In turn, this leads to a situation where ~/.cache/tracker Exceeds 3.8 GiB of data. There should be some kind of safeguard against this situation as to not consume too much data.
It may make sense to simply skip the XDG directories that are set to $HOME.
Yes, I agree. I must admit, we do depend (currently) on the XDG locations being sane. I guess some distros don't have directories for each type of file.
Even more likely that this will bite old-time users that upgrade from a distribution that didn't have the files, or users who drank the Kool-Aid back in the day of using $HOME as ~/Desktop ( Spatial Nautilus, gnome 2.2x days ?)
Yea :) It's actually a hard problem to solve nicely. You see if we don't index $HOME recursively then we miss files (in your above situation). We normally don't index $HOME recursively but then we do for the other defined folders. Also, we should be filtering duplicates so if all those locations are $HOME we on't do it once anyway. So the solution might be to just make sure we DON'T recursively index any XDG locations set to $HOME.
Created attachment 219537 [details] [review] Proposed patch The attached patch should fix this bug. Testing and review appreciated.
Comment on attachment 219537 [details] [review] Proposed patch Looks good to me. The usual test for this is to save/load the tracker-preferences with different targets and make sure the the config file isn't messed up and the list of target locations is sane in the miner-fs debugging. But AFAICS, this patch looks quite good!
commit 06f7ac0928e75a1a63f115f9d8cfd7e3096d1c60 Author: Jürg Billeter <j@bitron.ch> Date: Tue Jul 24 00:20:37 2012 +0200 tracker-miner-fs: Ignore XDG directories set to $HOME This prevents accidental recursive indexing of $HOME.
Question, What happens if ~/Documents is a symlink to ~ When Documents is set to that?
The correct question is probably: how does g_get_user_special_dir () resolve symlinks. You might want to go a step further and make sure that you use g_file_read_link on the entry, rather than blindly comparing strings.
(In reply to comment #9) > The correct question is probably: how does g_get_user_special_dir () resolve > symlinks. It doesn't. The function returns a string based on a value in a file and that's it. It doesn't have to exist at all. > You might want to go a step further and make sure that you use g_file_read_link > on the entry, rather than blindly comparing strings. We already do this since 2009-12-11 12:44:05 (and before even) in commit 74286929: g_file_enumerate_children_async (ed->dir_file, attrs, G_FILE_QUERY_INFO_NOFOLLOW_SYMLINKS, G_PRIORITY_LOW, ed->cancellable, file_enumerate_children_cb, ed);