GNOME Bugzilla – Bug 328388
Some backends stops indexing without showing any errors
Last modified: 2006-02-02 07:40:29 UTC
Please describe the problem: Hello Everybody i am using SUSE 10, i recently installed beagle 0.20 from ULB-Gnome RPMs, after using Beagle for some time, backends such as IndexingService and KMail stops indexing, i found this earlier while monitoring beagle-status, but today did some diagnosis, Steps to reproduce: it seems reproducing the bug is bit hard, don't enable dubugging, for we must able to reproduce, then we can try debugging, For IndexingService 1. start beagle with Indexing Service and KMail backend enabled 2. browse internet after a bit gap then browse again, 3. try increasing the gap reasonablly until you feel that something not index, For KMail 1, receive emails after giving some long gap, Actual results: contents not indexed Expected results: indexers must index when i browse or receive mails after how much gap, Does this happen every time? FileSustem Indexer is working fine, Other information: this bug seems also occuring in Kopete backend, but for now we stick with above said backends, i am attaching the logs,
Created attachment 57991 [details] Logs before restarting beagle to to restart backends you can see in the end that i searched for many words but i didn't got any results,
Created attachment 57992 [details] Logs after restarting beagle you can see the words i searched that didn't match correct or returned less results in my previous logs are now match correctly and or returning more results, this is cycling problem, u must restaer beagle after some time to index the content that this restaer of beagle left out
I see a few inconsistent things in the log, remove ~/.beagle, clean build beagle and retry.
i recreated beagle folder, still same problem,
stop kmail (if running) to prevent arrival of any new mail remove ~/.beagle/Indexes/KMailIndex $ beagled --debug --allow-backend KMail wait for 2-3 minutes attach the Beagled _and_ IndexHelper log files for this session _only_. Also mention what happens - did it stop indexing after some time, did it return query results ? Also, add the output of beagle-info --index-info and beagle-info --status
Hello Everybody i done as last comment said, 1 > closed KMail, 2 > shutted down Beagled, 3 > started beagled --debug --allow-backend KMail 4 i waited 3 minutes and copied the Beagled _and_ IndexHelper log files to desktop, 5 > i saw 060127 1048108937 06626 IndexH DEBUG: The daemon appears to have gone away. 060127 1048109673 06626 IndexH DEBUG: Shutting down helper. 060127 1048110348 06626 IndexH DEBUG: (1) Waiting for 1 worker... 060127 1048110350 06626 IndexH DEBUG: waiting for server '/home/raghugs/.beagle/socket-helper' 060127 1048110646 06626 IndexH DEBUG: Server '/home/raghugs/.beagle/socket-helper' shut down 060127 1048112286 06626 IndexH DEBUG: Exiting while copying the log files, eventhough the deamon still runing, 6 > i waited for some time, 7 > went to may yahoo email account from Firefox, 8 > i forworded Anti-Virus Product Comparison Guide download offer mail as inline (it contains just the offer, not actual guide) from Ziff Davis to my Gmail Account, 9 > Started KMail and click Send/Receive then got the mail, 10 > fired up Holmes Beagle frontend and searched for Davis and got nothing, then searched for TiPS , Anti-Virus , Product , and still got nothing, i was searching untill i finish typing this comment, but still got nothing, 11 > i recopied Beagled _and_ IndexHelper log Files to show what happends before and after i receive the mail,
Created attachment 58194 [details] before receiving the email taken after 3 minutes of starting beagle with debug and Kmail only
Created attachment 58195 [details] before receiving the email taken after three minutes of starting beagle in debug and selecting Kmail as only backend
Created attachment 58196 [details] taken after receiving the mails
Created attachment 58197 [details] taken after receiving the mails
Created attachment 58198 [details] log taken from a terminal emulator log taken from terminal emulator, then coppied to text editor then added comment to show the difference between,
From the log: 1) When you sent the forwarded the email, beagle noticed it and index it. The attachment was a zip file and beagle doesnt know (yet) how to handle that, but if the body of that email had the words that you queried, then there is some problem. Can you open the file in some text editor and verify that this is indeed the email that you forwarded: "/home/raghugs/.kde/share/apps/kmail/mail/sent-mail/cur/1138281683.8523.4ZXPj:2,S" 2) Looks like new mail in your gmail account was not picked up. Are you using disconnected imap for your gmail-account. If that is the case, is it located at the standard location ? /home/raghugs/.kde/share/apps/kmail/dimap (attach the output of ls -lR /home/raghugs/.kde/share/apps/kmail/dimap) 3) Is this a new problem you are seeing in 0.2.0 ?
Unfortunatly all your prediction are wrong, 1 > i forwareded the mail as "Inline" and it has not had an attachment at all, yahoo even converted that html mail to plain text, i told you na about that in my 8th point in 6th comment, 2 > the file you told me to open was not Anti-Virus Product Comparison Guide download offer mail, that mail is received when i was runing beagle with many backends enabled, and it seems now only it indexed that, for that reason, don't assume that some backend conflicted with KMail backend, and now it received that other mail because those other backends diaabled, the mail received just because i restared beagle , when restarting beagle not only KMail backend but most backends except File backend search and index not index content in previous session, after restarting beagle Ziff Davis mail indexed without a hitch, and showed up in Holmes as one of result, 2 > no, i am using good old POP3 protocol to access my GMail account, 3 > it seems it is happening since 0.20 and in 0.20cvs releases. please check my 5th point in 6th comment, i think this is the reason that not only KMail backend but many backend to stop working after sometime,
The only possibility that is left and I can think of is some problem with inotify. You have inotify enabled - right ? Can you do $ export BEAGLE_INOTIFY_VERBOSE=1 $ beagled --fg --debug --allow-backend KMail wait for 3-4 minutes to let kmail start and then send/receive some mail. Then attach the beagle and indexhelper log files.
Hello Bera i still not sure which one is culprit, and i don't want to send attachment of attachment, i am discribing just what happend, stopped runing beagle, started again with your above said command, i went to hotmail.com and wrote mail to my gmail id, received that mail via KMail suddenly i saw those lines in VTE too, *** inotify: Modify 2 /home/raghugs/.kde/share/apps/kmail/mail .inbox.index (file) *** inotify: Modify 2 /home/raghugs/.kde/share/apps/kmail/mail .inbox.index.ids (file) *** inotify: Modify 2 /home/raghugs/.kde/share/apps/kmail/mail .inbox.index.ids (file) *** inotify: Modify 2 /home/raghugs/.kde/share/apps/kmail/mail .inbox.index.sorted (file) i assume mail won't be there in those files, i searched for many words in holmes and got no results, i opened current-beagle and current-IndexHelper, in those files there is no notification of anything happend, it just showing its indexing server memory usage, this is what happend, but i am sure this time you would be able to guess my problem, expecting your reply
Looks like some kind of inotify problem. Maybe thats why even IndexingService is not working. Is the Files backend working on newly dropped files or deletion of files when beagled is running ? Can you also paste the output of $ ls -lRa /home/raghugs/.kde/share/apps/kmail/mail
If it's an inotify issue, you could try running the Inotify.exe test program that gets created in the Util directory. It's not shipped as part of a package, so you'd have to build from source to use it.
Created attachment 58507 [details] out of ls -lRa in mail folder of my kmail folder
Hello i tested Inotify manually, Inotify always catching file operations without any problems, although it didn't detect file operations occured in /progs directory, i am attaching the output of that command,
Nops. Nothing abnormal to KMail/KMail backend. Shifting gears to IndexingService, its easier to debug with that one. IndexingService is also not working - right ? What do you mean "it didn't detect file operations occured in /progs directory" ? Is inotify failing sometimes ?
Sorry Bera i am bit tired of sending you details you asked, i understand your problem too, you can't reproduce it, i am bit unfortunate, i think i must live with it, thanks for tring to help me, i am closing this bug