GNOME Bugzilla – Bug 453543
Extremely Large Log Files
Last modified: 2008-03-25 13:39:44 UTC
Please describe the problem: For the first ever, i've decided to download a torrent file using deluge. Just overnight I ended up with three rather large log files created by beagle: 2007-07-03-12-12-39-IndexHelper (106.3 MB) 2007-07-02-22-35-13-Beagle (102.0 MB) 2007-07-02-21-23-43-IndexHelper (23.7 GB) <--- yes its that big Note that i'm using build for Ubuntu feisty gotten from the ubuntu repositories using apt-get. Steps to reproduce: 1. 2. 3. Actual results: Expected results: Does this happen every time? Other information: the torrent (~600 MB) is downloading into another partition (FAT32) that has just over 100 GB of space and that partition is being monitored by beagle.
Created attachment 91149 [details] Disk Usage Analyzer showing my home directory
I don't know if is useful but here is the output from beagle-index-info... Index information: Name: EvolutionMail Count: 14120 Crawling: False Name: EvolutionDataServer Count: 928 Crawling: False Name: Thunderbird Count: 0 Crawling: False Name: KMail Count: 0 Crawling: False Name: Files Count: 61734 Crawling: False Name: GaimLog Count: 4229 Crawling: False Name: IndexingService Count: 6964 Crawling: False Name: Tomboy Count: 0 Crawling: False Name: Labyrinth Count: 0 Crawling: False Name: Blam Count: 0 Crawling: False Name: Liferea Count: 0 Crawling: False Name: Akregator Count: 0 Crawling: False Name: KonquerorHistory Count: 0 Crawling: False Name: KonqBookmark Count: 0 Crawling: False Name: KNotes Count: 0 Crawling: False Name: KOrganizer Count: 0 Crawling: False Name: KAddressBook Count: 0 Crawling: False Name: Kopete Count: 0 Crawling: False Name: Konversation Count: 0 Crawling: False Name: applications Count: 416 Crawling: False Name: documentation Count: 25175 Crawling: False
Can you attach the first 1000 and last 1000 lines of the extremely large log file?
Also, what was the file deluge was downloading ? ISO ? If possible, can you give a link to the torrent ? It is theoritically possible that if beagle is trying to index a file constantly being written to, then the corresponding filter will fail each time. That will produce a few lines of error message. Depending upon the type of the file, there might be more verbose error message. Overnight this could really blow up the IndexHelper log file. The last 1000 lines would be helpful in diagnosing.
sorry for the delay in responding since my first post. i removed the largest file soon after making the post. i could have barely open the log file without my system resources being hammered to the point where i had to kill the text editor. i did however keep around the +100 MB files. is there a quick way to parse for the requested lines and dump them to a separate file through a script so i don't have to open them in a text editor because loading the whole files takes a very long time. yes, the download was an ISO. i don't remember the exact link but i'll check my torrent logs to see if its listed. from what i can remember, before i left the computer running for the night, download speeds were about 250 Kb/s and upload about 50 Kb/s so beagle may have been triggering to re-index the file heavily with the constant changes.
*** Bug 470252 has been marked as a duplicate of this bug. ***
Seems that the problem can also happen with actually large log files, as reported in Debian BTS #472569. May I ask some more info to submitter?
The last activity was over a year ago. So I am going to close this WONTFIX. The latest versions do not print the whole errors over and over again, they only print the whole line. But, that will only cause slightly smaller log files. You _have_ to keep out torrent download directories out of beagle. Add it as a directory to exclude in beagle-settings or beagle-config. There is no easy to cover all cases on how to deal with rapidly changing files without introducely complicated rules and configuration. It is much easier to ask the torrent apps to download in a non-indexed directory (like /tmp or ~/tmp).
(In reply to comment #7) > Seems that the problem can also happen with actually large log files, as > reported in Debian BTS #472569. May I ask some more info to submitter? > Same reply as in comment #8. Some people want to get results real time (imagine dashboard users, there are more use cases possible). It is advised that if users have large, rapidly changing files they just ask beagle to exclude the directory during indexing. It can be done real-time i.e. they do not need to restart beagled after that. Its all possible using beagle-settings (or beagle-config if the user does not use the GUI).