GNOME Bugzilla – Bug 676713
tracker-miner-fs using 100% of cpu time
Last modified: 2021-05-26 22:24:16 UTC
Created attachment 214841 [details] Some backtraces of the tracker-miner-fs process while using 100% of cpu time Shortly after tracker-miner-fs (0.14.1, running on Debian Wheezy AMD64) is started, it starts to use 100% of cpu time. strace is only showing this: brk(0xc2d7000) = 0xc2d7000 brk(0xc2d6000) = 0xc2d6000 brk(0xc2f7000) = 0xc2f7000 brk(0xc2f6000) = 0xc2f6000 After setting the log level to debug and restarting tracker, the last lines before the CPU hogging happens, are this: 24 May 2012, 09:38:06: Tracker: Added monitor for path:'file:///home/frederik/Werk/fcg-net/wordpress/wp-includes/js/tinymce/plugins/inlinepopups/skins/clearlooks2/img', total monitors:1837 24 May 2012, 09:38:06: Tracker: Added monitor for path:'file:///home/frederik/Werk/fcg-net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/o2k7/img', total monitors:1838 24 May 2012, 09:38:06: Tracker: Added monitor for path:'file:///home/frederik/Werk/fcg-net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/default/img', total monitors:1839 24 May 2012, 09:38:06: Tracker: Added monitor for path:'file:///home/frederik/Werk/fcg-net/wordpress/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/img', total monitors:1840 No more messages are added to the log once it starts using 100% of cpu time. $ tracker-control -FS Store: 24 May 2012, 09:44:43: ✓ Store - Idle Miners: (tracker-control:23135): Tracker-CRITICAL **: Could not get miner progress for 'org.freedesktop.Tracker1.Miner.Files': Timeout was reached Could not get status from miner: org.freedesktop.Tracker1.Miner.Files (tracker-control:23135): Tracker-CRITICAL **: Could not get miner progress for 'org.freedesktop.Tracker1.Miner.Applications': Timeout was reached Could not get status from miner: org.freedesktop.Tracker1.Miner.Applications24 May 2012, 09:45:33: ✗ E-mails - Not running or is a disabled plugin Press Ctrl+C to end follow of Tracker state I attach some gdb backtraces of the tracker-miner-fs process.
Created attachment 282965 [details] tracker taking 100% CPU time
Created attachment 282968 [details] tracker taking 2GoB of RAM
Hi, I know it's more than two years since the original message but i've the same problem than freggy1. I'm running on Ubuntu Gnome 14.04 with ppa gnome3-team [1] and gnome3-team staging [2] activated. This means that I've tracker version 1.0.2-1ubuntu3-trusty1 (This version is different than the version assign to this bug. If that could be a problem I could open another bug). The only solution is to kill tracker process each time I boot my computer. After some research made yesterday I've read that tracker will stop when he will index all my files. So I let him go for several hours (3 hours and half). After that it still taking 100 % of CPU and at least 2 GiB of RAM (out of 8 Gib). See screenshot attached [1] https://launchpad.net/~gnome3-team/+archive/ubuntu/gnome3 [2] https://launchpad.net/~gnome3-team/+archive/ubuntu/gnome3-staging
Hi, can you provide the output of "tracker-control --collect-debug-info" here so I can see what sort of set up you have? Thanks :)
Hi Martyn. Thank you for your help. Here is the output : sebastien@trusty:~$ tracker-control --collect-debug-info [Package Details] version: 1.0.2 [Disk Information] remaining space on db partition: 6,9 Go (44,16%) [Data Set] /home/sebastien/.cache/tracker/ontologies.gvdb 355,1 ko /home/sebastien/.cache/tracker/meta.db-shm 32,8 ko /home/sebastien/.cache/tracker/meta.db 875,9 Mo /home/sebastien/.cache/tracker/no-need-mtime-check.txt.TARIHX 0 octet /home/sebastien/.cache/tracker/meta.db-wal 10,2 Mo [Configuration] Store.verbosity: 'errors' Store.graphupdated-delay: 1000 Extract.max-media-art-width: 0 Extract.verbosity: 'errors' Extract.wait-for-miner-fs: false Extract.max-bytes: 1048576 Extract.sched-idle: 'first-index' Writeback.verbosity: 'errors' Files.initial-sleep: 15 Files.index-optical-discs: false Files.index-recursive-directories: ['&DESKTOP', '&DOCUMENTS', '&DOWNLOAD', '&MUSIC', '&PICTURES', '&VIDEOS', '/media/Documents/administratif', '/media/Documents/Musique'] Files.enable-monitors: false Files.index-on-battery: true Files.sched-idle: 'first-index' Files.ignored-directories: ['po', 'CVS', 'core-dumps', 'lost+found'] Files.crawling-interval: -2 Files.ignored-files: ['*~', '*.o', '*.la', '*.lo', '*.loT', '*.in', '*.csproj', '*.m4', '*.rej', '*.gmo', '*.orig', '*.pc', '*.omf', '*.aux', '*.tmp', '*.po', '*.vmdk', '*.vm*', '*.nvram', '*.part', '*.rcore', '*.lzo', 'autom4te', 'conftest', 'confstat', 'Makefile', 'SCCS', 'ltmain.sh', 'libtool', 'config.status', 'confdefs.h', 'configure'] Files.ignored-directories-with-content: ['backup.metadata'] Files.index-removable-devices: false Files.removable-days-threshold: 3 Files.index-single-directories: ['$HOME'] Files.low-disk-space-limit: -1 Files.throttle: 0 Files.verbosity: 'errors' Files.enable-writeback: true Files.index-on-battery-first-time: true [States] /home/sebastien/.cache/tracker/db-version.txt 24 /home/sebastien/.cache/tracker/first-index.txt 1.0.1 /home/sebastien/.cache/tracker/last-crawl.txt 1403713326 (65 jours 02 minutes 42 secondes) /home/sebastien/.cache/tracker/miner-applications-locale.txt fr_FR.UTF-8 /home/sebastien/.cache/tracker/db-locale.txt fr_FR.UTF-8 [Data Statistics] database is currently empty sebastien@trusty:~$
Hi, I've found a way to reproduce the bug. Yesterday I moved to Ubuntu Gnome 14.10 (beta 1) with a full reinstallation. Everything was fine (tracker included). But I've modified the favorites in nautilus (Pictures, Videos, Documents, etc...). Instead of pointing to the default directories (~/Pictures, ~/Videos, etc...), I changed it to point to my Documents partition (/media/Documents/Pictures, /media/Documents/Videos, etc...) Now each time I open a new session, tracker try to read all my documents. Step to reproduce the bug : 1- edit ~/.config/user-dirs.dirs in order to define new shortcuts for default directories of nautilus 2- log out, log in and tracker take 100% of your CPU. Remark : Revert the user-dirs.dirs to the default doesn't help. sebastien@utopic:~$ apt-cache policy tracker tracker: Installé : 1.0.2-1ubuntu4 Candidat : 1.0.2-1ubuntu5 Table de version : 1.0.2-1ubuntu5 0 500 http://fr.archive.ubuntu.com/ubuntu/ utopic/main i386 Packages *** 1.0.2-1ubuntu4 0 100 /var/lib/dpkg/status sebastien@utopic:~$ tracker-control --collect-debug-info [Package Details] version: 1.0.2 [Disk Information] remaining space on db partition: 13,9 Go (88,14%) [Data Set] /home/sebastien/.cache/tracker/meta.db-wal 62,1 Mo /home/sebastien/.cache/tracker/ontologies.gvdb 355,1 ko /home/sebastien/.cache/tracker/meta.db-shm 491,5 ko /home/sebastien/.cache/tracker/meta.db 232,4 Mo [Configuration] Store.verbosity: 'errors' Store.graphupdated-delay: 1000 Extract.max-media-art-width: 0 Extract.verbosity: 'errors' Extract.wait-for-miner-fs: false Extract.max-bytes: 1048576 Extract.sched-idle: 'first-index' Writeback.verbosity: 'errors' Files.initial-sleep: 15 Files.index-optical-discs: false Files.index-recursive-directories: ['&DESKTOP', '&DOCUMENTS', '&DOWNLOAD', '&MUSIC', '&PICTURES', '&VIDEOS'] Files.enable-monitors: true Files.index-on-battery: true Files.sched-idle: 'first-index' Files.ignored-directories: ['po', 'CVS', 'core-dumps', 'lost+found'] Files.crawling-interval: -1 Files.ignored-files: ['*~', '*.o', '*.la', '*.lo', '*.loT', '*.in', '*.csproj', '*.m4', '*.rej', '*.gmo', '*.orig', '*.pc', '*.omf', '*.aux', '*.tmp', '*.po', '*.vmdk', '*.vm*', '*.nvram', '*.part', '*.rcore', '*.lzo', 'autom4te', 'conftest', 'confstat', 'Makefile', 'SCCS', 'ltmain.sh', 'libtool', 'config.status', 'confdefs.h', 'configure'] Files.ignored-directories-with-content: ['backup.metadata'] Files.index-removable-devices: false Files.removable-days-threshold: 3 Files.index-single-directories: ['$HOME'] Files.low-disk-space-limit: -1 Files.throttle: 0 Files.verbosity: 'errors' Files.enable-writeback: true Files.index-on-battery-first-time: true [States] /home/sebastien/.cache/tracker/miner-applications-locale.txt fr_FR.UTF-8 /home/sebastien/.cache/tracker/first-index.txt 1.0.2 /home/sebastien/.cache/tracker/last-crawl.txt 1409433551 (1 jour 02 minutes 17 secondes) /home/sebastien/.cache/tracker/db-version.txt 24 /home/sebastien/.cache/tracker/db-locale.txt fr_FR.UTF-8 [Data Statistics] database is currently empty sebastien@utopic:~$
(In reply to comment #6) > Hi, > I've found a way to reproduce the bug. Yesterday I moved to Ubuntu Gnome 14.10 > (beta 1) with a full reinstallation. Everything was fine (tracker included). > But I've modified the favorites in nautilus (Pictures, Videos, Documents, > etc...). Instead of pointing to the default directories (~/Pictures, ~/Videos, > etc...), I changed it to point to my Documents partition > (/media/Documents/Pictures, /media/Documents/Videos, etc...) > Now each time I open a new session, tracker try to read all my documents. Hmm, what file system are you using for these locations? > > Step to reproduce the bug : > 1- edit ~/.config/user-dirs.dirs in order to define new shortcuts for default > directories of nautilus > 2- log out, log in and tracker take 100% of your CPU. Thanks for the steps. > Remark : Revert the user-dirs.dirs to the default doesn't help. I've never tested changing this, but I would imagine it can be quite interesting :) > [Disk Information] > remaining space on db partition: 13,9 Go (88,14%) > > > [Data Set] > /home/sebastien/.cache/tracker/meta.db-wal > 62,1 Mo > > /home/sebastien/.cache/tracker/ontologies.gvdb > 355,1 ko > > /home/sebastien/.cache/tracker/meta.db-shm > 491,5 ko > > /home/sebastien/.cache/tracker/meta.db > 232,4 Mo So the DB size isn't so bad and you have space - that's always a good sign. > [Configuration] > Store.verbosity: 'errors' > Store.graphupdated-delay: 1000 > Extract.max-media-art-width: 0 > Extract.verbosity: 'errors' > Extract.wait-for-miner-fs: false > Extract.max-bytes: 1048576 > Extract.sched-idle: 'first-index' > Writeback.verbosity: 'errors' > Files.initial-sleep: 15 > Files.index-optical-discs: false > Files.index-recursive-directories: ['&DESKTOP', '&DOCUMENTS', '&DOWNLOAD', > '&MUSIC', '&PICTURES', '&VIDEOS'] > Files.enable-monitors: true > Files.index-on-battery: true > Files.sched-idle: 'first-index' OK, this might be related... 'first-index' as a setting means - we use a lot of CPU on initial index. Sadly, it's not clear from the debug if an initial index has completed ever?
Hi Martyn ! Few things have changed since my last post. I've installed Ubuntu Gnome 14.10 last month (and activate gnome-staging ppa). So I'm running 1.0.4-0ubuntu2+utopic2. I've applied the step to reproduce the bug and I can confirm it is still applicable to my config :). However I've created another user profile and it helped to solve the problem. To answer your question : Hmm, what file system are you using for these locations? --> The file system is ext4. I only run linux on my computer. OK, this might be related... 'first-index' as a setting means - we use a lot of CPU on initial index. Sadly, it's not clear from the debug if an initial index has completed ever? --> I don't think so. It take too long (more than 3 hrs and a half). I still think that modifying default shortcut in nautilus create the bug. And when I run the debug info command you indicate I see that : Files.index-recursive-directories: ['&DESKTOP', '&DOCUMENTS', '&DOWNLOAD', '&MUSIC', '&PICTURES', '&VIDEOS'] Do you know if there is a way to ask tracker to remove the recursivity for this folder ? Thank for your help anyway !
(In reply to comment #8) > Hi Martyn ! > > Few things have changed since my last post. I've installed Ubuntu Gnome 14.10 > last month (and activate gnome-staging ppa). So I'm running > 1.0.4-0ubuntu2+utopic2. I've applied the step to reproduce the bug and I can > confirm it is still applicable to my config :). However I've created another > user profile and it helped to solve the problem. > > To answer your question : > > Hmm, what file system are you using for these locations? > --> The file system is ext4. I only run linux on my computer. Hmm, so ext4 should be fine. > OK, this might be related... 'first-index' as a setting means - we use a lot of > CPU on initial index. Sadly, it's not clear from the debug if an initial index > has completed ever? > --> I don't think so. It take too long (more than 3 hrs and a half). Hmm, I think this is the problem. It is on a setting which will really milk the system's resources until it's complete. I would go into tracker-preferences and change this to the top option "Only when computer is not being used", may be different if you're not using English - which I think is the case here. This is technically not true either, it just puts the kernel scheduler to the lowest possible setting for Tracker, so something is still going on. I've been thinking about adding a new setting that REALLY does do nothing unless (e.g. when you go to lunch). > I still think that modifying default shortcut in nautilus create the bug. And > when I run the debug info command you indicate I see that : > > Files.index-recursive-directories: ['&DESKTOP', '&DOCUMENTS', '&DOWNLOAD', > '&MUSIC', '&PICTURES', '&VIDEOS'] I think modifying these and restarting tracker is going to cause massive problems. If this was a one time thing, I would simply do: tracker-control -rs This will reindex all content using the new XDG locations. > Do you know if there is a way to ask tracker to remove the recursivity for this > folder ? Yes, there is a "Files.index-single-directories" for this. If you use tracker-preferences again, you can change this with a checkbutton quite easily using the buttons for the standard locations. I think it's on the second tab/page under "Locations". > Thank for your help anyway ! Happy to help!
Was going to file another 'tracker is always at 100% CPU' but then I found this. For the past few years tracker has always sat at 100% CPU usage, and the only thing I have figured to do is purge it from my system (and thus all the apps which rely on it). I'm using Debian Testing (tracker 1.6.1) with two disks and some of the folders in my home directory are symlinks to folders on other partitions. Not sure how to debug or provide more useful information otherwise.
(In reply to Hashem Nasarat from comment #10) > Was going to file another 'tracker is always at 100% CPU' but then I found > this. > For the past few years tracker has always sat at 100% CPU usage, and the > only thing I have figured to do is purge it from my system (and thus all the > apps which rely on it). > > I'm using Debian Testing (tracker 1.6.1) with two disks and some of the > folders in my home directory are symlinks to folders on other partitions. Tracker doesn't follow symlinks, unless it is told to look into them. E. g. it indexes/monitors ~/Music recursively by default, even if it's a symlink somewhere else, but it won't recurse into ~/Music/symlink-to-other-folder. > > Not sure how to debug or provide more useful information otherwise. There's no known reasons why tracker-miner-fs would sit at 100%, it is expected that it has some activity on startup (in order to check for changes, set up monitors...), but this activity is driven by the filesystem traversal in the indexed folders, which is guaranteed to be finite and loop free because we don't follow symlinks. This activity is obviously tied to the number of elements (folders mainly) to index, so it might be what happens for you? I'd suggest you check out the index-recursive-directories and index-single-directories settings in the /org/freedesktop/tracker/miner/files/ path with dconf-editor, reset these to defaults if those were modified, and re-run tracker-miner-fs in the terminal through: killall -15 tracker-miner-fs && /usr/lib/tracker/tracker-miner-fs -v 2 -v 2 will be telling the files being processed, etc, that might give you a clue about whether it's actually repeating itself over. You might also try selectively removing folders to be indexed from settings, in order to identify the folder it's stumbling upon. Also, in tracker 1.7/1.8 I made git repositories to be ignored by tracker by default (by adding '.git' in the ignored-directories-with-content setting in the path above), eg. checkouts of the linux kernel add some quarter million elements to be indexed. If this is not a case of infinite recursion, doing this could help for your case? I also suggest you check out the other miner settings and see how can they help you.
As I posted on https://bugzilla.gnome.org/show_bug.cgi?id=778166 this was contributing to my computer overheating. Tracker was running on boot and a snapshot showed miner-fs at 104%, store at 94%, and extract at 45%. I had to uninstall these.
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a new enhancement request ticket at https://gitlab.gnome.org/GNOME/tracker/-/issues/ Thank you for your understanding and your help.