GNOME Bugzilla – Bug 300131
beagled stuck in a directory
Last modified: 2005-09-08 04:36:45 UTC
Version details: 0.0.9 Distribution/Version: Ubuntu Hoary I am using Beagle 0.0.9 and Mono 1.1.4 on Ubuntu Hoary. 1. I started beagled and it began scanning directories, but after a while it just got stuck in /home/sam/. (beagle-status showed "File System Crawler" pointing to the same directory for more than half an hour.) 2. After killing beagled and restarting it again, it proceeded with some subdirectories first, but then got stuck in another seemingly random dir. 3. I restarted beagled again, but again it stuck in another directory. 4. Same procedure about 10 times. The command line output of "beagled --debug --fg ." when that happens: [...] DEBUG: *** Doing nothing to /home/sam/Whatever/video.mpg DEBUG: *** What should we do with /home/sam/Whatever? DEBUG: CTime check 10.04.2005 22:44:11 10.04.2005 21:44:12 '/home/sam/Whatever' DEBUG: *** Doing nothing to /home/sam/Whatever DEBUG: RemoteIndexer.Flush on 'FileSystemIndex' DEBUG: Requesting new proxy 'FileSystemIndex' DEBUG: Waiting for proxy 'FileSystemIndex' DEBUG: Waiting code=3 DEBUG: Close on FileSystemIndex DEBUG: Unsetting proxy 'FileSystemIndex' DEBUG: Finished task Crawling /home/sam/Whatever in ,65s DEBUG: Starting task File System Crawler DEBUG: Crawl Task Scheduling /home/sam/Whatever (state=PossiblyClean) DEBUG: Finished task File System Crawler in ,01s DEBUG: Starting task Crawling /home/sam/Whatever DEBUG: *** What should we do with /home/sam/Whatever/video.mpg? DEBUG: *** Doing nothing to /home/sam/Whatever/video.mpg DEBUG: *** What should we do with /home/sam/Whatever? DEBUG: CTime check 10.04.2005 22:44:12 10.04.2005 21:44:13 '/home/sam/Whatever' DEBUG: *** Doing nothing to /home/sam/Whatever DEBUG: RemoteIndexer.Flush on 'FileSystemIndex' DEBUG: Requesting new proxy 'FileSystemIndex' DEBUG: Waiting for proxy 'FileSystemIndex' DEBUG: Waiting code=3 DEBUG: Close on FileSystemIndex DEBUG: Unsetting proxy 'FileSystemIndex' DEBUG: Finished task Crawling /home/sam/Whatever in ,65s DEBUG: Starting task File System Crawler DEBUG: Crawl Task Scheduling /home/sam/Whatever (state=PossiblyClean) DEBUG: Finished task File System Crawler in ,01s DEBUG: Starting task Crawling /home/sam/Whatever DEBUG: *** What should we do with /home/sam/Whatever/video.mpg? DEBUG: *** Doing nothing to /home/sam/Whatever/video.mpg DEBUG: *** What should we do with /home/sam/Whatever? DEBUG: *** Doing nothing to /home/sam/Whatever DEBUG: RemoteIndexer.Flush on 'FileSystemIndex' DEBUG: Requesting new proxy 'FileSystemIndex' DEBUG: Waiting for proxy 'FileSystemIndex' DEBUG: Waiting code=3 DEBUG: Close on FileSystemIndex DEBUG: Unsetting proxy 'FileSystemIndex' DEBUG: Finished task Crawling /home/sam/Whatever in 1,21s DEBUG: Starting task File System Crawler DEBUG: Crawl Task Scheduling /home/sam/Whatever (state=PossiblyClean) DEBUG: Finished task File System Crawler in ,01s DEBUG: Starting task Crawling /home/sam/Whatever [...] (the /home/sam/Whatever directory is scanned endlessly.) (The directory is really named that way, the log is unedited.)
Additional information: inotify is not enabled in the kernel. The filesystem is mounted using the user_xattr option.
There have been known problems with .doc files. Can you please track it down to the specific file causing this? That would be very helpful.
Sorry, I can't track it down to one file because it happens in seemingly random directories. Definitely none of them holds any .doc files. I can even tell for sure that it /did/ one time hang in a directory that has /only/ subdirectories in it and no normal files at all.
Well is is nearly impossible for developers to track the error down if only few information is available. 1.) please upgrade to the latest version. 2.) googling I found this guide on tracking errors http://www.beaglewiki.org/index.php/Troubleshooting%20Beagle maybe this helps or you can narrow what causes the hang. in any case please report back.
DEBUG: CTime check 10.04.2005 22:44:11 10.04.2005 21:44:12 '/home/sam/Whatever' [...] DEBUG: CTime check 10.04.2005 22:44:12 10.04.2005 21:44:13 '/home/sam/Whatever' [...] The above two lines from your beagle Log says that the directory "/home/sam/Whatever" is frequently getting changed/modified, because of which (i think you also have inotify), beagle keeps circling around that directory. Can you just check whether any process is writing to that directory, by any chance?
#4: I really don't know how to narrow this down any further, as it is not reproduceable enough. It is obviously not related to one specific file type. #5: Yes, the loop is what happened over a period of more than half an hour, the process was stuck scanning that same directory again and again. The mentioned directory /home/sam/Whatever contains only the video, and while scanning I definitely was not watching a video or doing any other operation on the directory. So it is probably hard to imagine a process changing the ctime of that directory frequently. I will try to reproduce the problem, but I had the deamon running for several hours now and up to now it did not enter the endless loop anymore.
If was able to reproduce this again with another directory. Some facts: - While the crawler is stuck in the loop, the ctime of neither the directory nor any of it's files is changed. - lsof does not show any open files in that directory. - I left the crawler running and created a new directory, then moved some of the files from the currently-being-looped directory into the newly created one. Beagled detected that and processed the moved files. When those files were processed, it went back to crawling the NEW directory and got stuck there in the loop.
*** Bug 303298 has been marked as a duplicate of this bug. ***
Created attachment 47750 [details] Beagle logfile
Created attachment 47751 [details] IndexHelper logfile
I think this is related to bug 306688
I was able to reproduce the same scenario and its 100% reproducible for me. I dont think its due to the CTime check bug because 1) I have inotify disabled 2) Without inotify filesystem is periodically polled but I dont think the poll interval is that low - the looping happens continuously Needless to say, all these happened on a test directory and there was nothing changing in that directory. Furthermore, I happen to had a CVS checkout of 10-12 days back which doesnt show the problem (on the same filesystem root, if I may add). Both the good and buggy are consistent (at least during the 50 times I tried this evening :). If you think the logfiles/index-files/directory.tar.gz/cvs-checkout.tgz will be helpful in fixing the bug - pls post here. I have kept them separately.
Hello, I experience the same problem with beagle 0.0.12. The directory contains two AVI files, nothing writes or reads this directory, and beagle just keep crawling the directory, with "state=PossiblyClean" I have no inotify in kernel, it happens regardless of whether I am using the EXERCISE_THE_DOG option HTH
Some useful information for the current CVS code: - DirectoryModel.cs:CompareTo_Unlocked(...) compares last_activity_time, state, last_crawl_time, depth etc... to decide which of the two directories should be crawled first - After each directory is crawled, FileSystemQueryable.cs:DoneCrawlingOneDirectory(...) is called which is not setting any of the last_activity_time (which i dont think shud be set) or last_crawl_time (which I think should be set). In fact, nobody is setting the crawl_time. Though I am not sure what the fix should be - this doesnt seem to be the right behaviour. Obviously because of the behaviour, the last directory in the queue is always chosen as the directory to crawl and hence the looping in the same directory. I can provide additional info if needed.
Daniel checked in a fix in the CVS. The fix solves the problem for me (using CVS). From the nature of the fix, it doesnt seem that the fix is applicable to <= 0.0.12. So there might be additional bugs that needs to be fixed. If possible upgrade to cvs and report.
I'm closing this bug as FIXED, please re-open the bug if you see the orginal problem again.