After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 300131 - beagled stuck in a directory
beagled stuck in a directory
Status: RESOLVED FIXED
Product: beagle
Classification: Other
Component: General
0.0.x
Other Linux
: Normal critical
: ---
Assigned To: Jon Trowbridge
Jon Trowbridge
: 303298 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2005-04-10 21:03 UTC by Samuel Abels
Modified: 2005-09-08 04:36 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Beagle logfile (252.00 KB, text/plain)
2005-06-14 11:05 UTC, Nico Kaiser
Details
IndexHelper logfile (19.42 KB, text/plain)
2005-06-14 11:05 UTC, Nico Kaiser
Details

Description Samuel Abels 2005-04-10 21:03:14 UTC
Version details: 0.0.9
Distribution/Version: Ubuntu Hoary

I am using Beagle 0.0.9 and Mono 1.1.4 on Ubuntu Hoary.

1. I started beagled and it began scanning directories, but after a while it
just got stuck in /home/sam/. (beagle-status showed "File System Crawler"
pointing to the same directory for more than half an hour.)

2. After killing beagled and restarting it again, it proceeded with some
subdirectories first, but then got stuck in another seemingly random dir.

3. I restarted beagled again, but again it stuck in another directory.

4. Same procedure about 10 times.

The command line output of "beagled --debug --fg ." when that happens:

[...]
DEBUG: *** Doing nothing to /home/sam/Whatever/video.mpg
DEBUG: *** What should we do with /home/sam/Whatever?
DEBUG: CTime check 10.04.2005 22:44:11 10.04.2005 21:44:12 '/home/sam/Whatever'
DEBUG: *** Doing nothing to /home/sam/Whatever
DEBUG: RemoteIndexer.Flush on 'FileSystemIndex'
DEBUG: Requesting new proxy 'FileSystemIndex'
DEBUG: Waiting for proxy 'FileSystemIndex'
DEBUG: Waiting code=3
DEBUG: Close on FileSystemIndex
DEBUG: Unsetting proxy 'FileSystemIndex'
DEBUG: Finished task Crawling /home/sam/Whatever in ,65s
DEBUG: Starting task File System Crawler
DEBUG: Crawl Task Scheduling /home/sam/Whatever (state=PossiblyClean)
DEBUG: Finished task File System Crawler in ,01s
DEBUG: Starting task Crawling /home/sam/Whatever
DEBUG: *** What should we do with /home/sam/Whatever/video.mpg?
DEBUG: *** Doing nothing to /home/sam/Whatever/video.mpg
DEBUG: *** What should we do with /home/sam/Whatever?
DEBUG: CTime check 10.04.2005 22:44:12 10.04.2005 21:44:13 '/home/sam/Whatever'
DEBUG: *** Doing nothing to /home/sam/Whatever
DEBUG: RemoteIndexer.Flush on 'FileSystemIndex'
DEBUG: Requesting new proxy 'FileSystemIndex'
DEBUG: Waiting for proxy 'FileSystemIndex'
DEBUG: Waiting code=3
DEBUG: Close on FileSystemIndex
DEBUG: Unsetting proxy 'FileSystemIndex'
DEBUG: Finished task Crawling /home/sam/Whatever in ,65s
DEBUG: Starting task File System Crawler
DEBUG: Crawl Task Scheduling /home/sam/Whatever (state=PossiblyClean)
DEBUG: Finished task File System Crawler in ,01s
DEBUG: Starting task Crawling /home/sam/Whatever
DEBUG: *** What should we do with /home/sam/Whatever/video.mpg?
DEBUG: *** Doing nothing to /home/sam/Whatever/video.mpg
DEBUG: *** What should we do with /home/sam/Whatever?
DEBUG: *** Doing nothing to /home/sam/Whatever
DEBUG: RemoteIndexer.Flush on 'FileSystemIndex'
DEBUG: Requesting new proxy 'FileSystemIndex'
DEBUG: Waiting for proxy 'FileSystemIndex'
DEBUG: Waiting code=3
DEBUG: Close on FileSystemIndex
DEBUG: Unsetting proxy 'FileSystemIndex'
DEBUG: Finished task Crawling /home/sam/Whatever in 1,21s
DEBUG: Starting task File System Crawler
DEBUG: Crawl Task Scheduling /home/sam/Whatever (state=PossiblyClean)
DEBUG: Finished task File System Crawler in ,01s
DEBUG: Starting task Crawling /home/sam/Whatever
[...]

(the /home/sam/Whatever directory is scanned endlessly.) (The directory is
really named that way, the log is unedited.)
Comment 1 Samuel Abels 2005-04-10 21:07:58 UTC
Additional information:

inotify is not enabled in the kernel.
The filesystem is mounted using the user_xattr option.
Comment 2 Christian Kirbach 2005-04-11 07:03:44 UTC
There have been known problems with .doc files.

Can you please track it down to the specific file causing this?
That would be very helpful.
Comment 3 Samuel Abels 2005-04-11 12:48:02 UTC
Sorry, I can't track it down to one file because it happens in seemingly random
directories. Definitely none of them holds any .doc files.

I can even tell for sure that it /did/ one time hang in a directory that has
/only/ subdirectories in it and no normal files at all.
Comment 4 Christian Kirbach 2005-04-11 13:21:39 UTC
Well is is nearly impossible for developers to track the error down if only few 
information is available.

1.) please upgrade to the latest version.
2.) googling I found this guide on tracking errors
http://www.beaglewiki.org/index.php/Troubleshooting%20Beagle

maybe this helps or you can narrow what causes the hang.

in any case please report back.
Comment 5 Veerapuram Varadhan 2005-04-13 11:25:58 UTC
DEBUG: CTime check 10.04.2005 22:44:11 10.04.2005 21:44:12 '/home/sam/Whatever'
[...]
DEBUG: CTime check 10.04.2005 22:44:12 10.04.2005 21:44:13 '/home/sam/Whatever'
[...]

The above two lines from your beagle Log says that the directory
"/home/sam/Whatever" is frequently getting changed/modified, because of which (i
think you also have inotify), beagle keeps circling around that directory.

Can you just check whether any process is writing to that directory, by any chance?
Comment 6 Samuel Abels 2005-04-13 20:33:20 UTC
#4: I really don't know how to narrow this down any further, as it is not
reproduceable enough. It is obviously not related to one specific file type.

#5: Yes, the loop is what happened over a period of more than half an hour, the
process was stuck scanning that same directory again and again.

The mentioned directory /home/sam/Whatever contains only the video, and while
scanning I definitely was not watching a video or doing any other operation on
the directory. So it is probably hard to imagine a process changing the ctime of
that directory frequently.

I will try to reproduce the problem, but I had the deamon running for several
hours now and up to now it did not enter the endless loop anymore.
Comment 7 Samuel Abels 2005-04-13 22:20:06 UTC
If was able to reproduce this again with another directory. Some facts:

- While the crawler is stuck in the loop, the ctime of neither the directory nor
any of it's files is changed.

- lsof does not show any open files in that directory.

- I left the crawler running and created a new directory, then moved some of the
files from the currently-being-looped directory into the newly created one.
Beagled detected that and processed the moved files. When those files were
processed, it went back to crawling the NEW directory and got stuck there in the
loop.
Comment 8 Joe Shaw 2005-05-20 17:34:16 UTC
*** Bug 303298 has been marked as a duplicate of this bug. ***
Comment 9 Nico Kaiser 2005-06-14 11:05:21 UTC
Created attachment 47750 [details]
Beagle logfile
Comment 10 Nico Kaiser 2005-06-14 11:05:56 UTC
Created attachment 47751 [details]
IndexHelper logfile
Comment 11 Daniel Drake 2005-06-27 23:18:19 UTC
I think this is related to bug 306688
Comment 12 Debajyoti Bera 2005-08-14 02:21:27 UTC
I was able to reproduce the same scenario and its 100% reproducible for me.  
I dont think its due to the CTime check bug because  
1) I have inotify disabled 
2) Without inotify filesystem is periodically polled but I dont think the poll 
interval is that low - the looping happens continuously 
 
Needless to say, all these happened on a test directory and there was nothing 
changing in that directory. Furthermore, I happen to had a CVS checkout of 
10-12 days back which doesnt show the problem (on the same filesystem root, if 
I may add). Both the good and buggy are consistent (at least during the 50 
times I tried this evening :). 
 
If you think the logfiles/index-files/directory.tar.gz/cvs-checkout.tgz will 
be helpful in fixing the bug - pls post here. I have kept them separately. 
Comment 13 Clément Stenac 2005-08-15 13:20:25 UTC
Hello,

I experience the same problem with beagle 0.0.12.

The directory contains two AVI files, nothing writes or reads this directory,
and beagle just keep crawling the directory, with "state=PossiblyClean"

I have no inotify in kernel, it happens regardless of whether I am using the
EXERCISE_THE_DOG option

HTH
Comment 14 Debajyoti Bera 2005-08-18 23:10:32 UTC
Some useful information for the current CVS code: 
 
- DirectoryModel.cs:CompareTo_Unlocked(...) 
  compares last_activity_time, state, last_crawl_time, depth etc... to decide 
which of the two directories should be crawled first 
 
- After each directory is crawled, 
FileSystemQueryable.cs:DoneCrawlingOneDirectory(...) is called which is not 
setting any of the last_activity_time (which i dont think shud be set) or 
last_crawl_time (which I think should be set). In fact, nobody is setting the 
crawl_time. 
 
Though I am not sure what the fix should be - this doesnt seem to be the right 
behaviour. Obviously because of the behaviour, the last directory in the queue 
is always chosen as the directory to crawl and hence the looping in the same 
directory. 
 
I can provide additional info if needed. 
Comment 15 Debajyoti Bera 2005-08-19 00:59:07 UTC
Daniel checked in a fix in the CVS. The fix solves the problem for me (using CVS).

From the nature of the fix, it doesnt seem that the fix is applicable to <=
0.0.12. So there might be additional bugs that needs to be fixed. If possible
upgrade to cvs and report.
Comment 16 Jon Trowbridge 2005-09-08 04:36:45 UTC
I'm closing this bug as FIXED, please re-open the bug if you see the orginal
problem again.