GNOME Bugzilla – Bug 556068
beagle's thunderbird backend leaves masses of stray files in /tmp
Last modified: 2009-01-04 20:38:11 UTC
Please describe the problem: When using beagle's thunderbird extension, many files of the form /tmp/tmp*.tmp are left lying around. When I first noticed the problem, there were over 40000 of them. Aside from the file leakage, the indexing seems to actually work. Steps to reproduce: 1. Enable thunderbird's beagle extension 2. Fetch mail from imap server 3. Actual results: Mail is indexed, but many stray files are left in /tmp. Expected results: Indexed mail, with no persistent tmp files. Does this happen every time? Yes. Other information: I'm using Fedora rawhide packages: beagle-gnome-0.3.8-7.fc10.x86_64 beagle-0.3.8-7.fc10.x86_64 beagle-firefox-0.3.8-7.fc10.x86_64 beagle-thunderbird-0.3.8-7.fc10.x86_64 thunderbird-2.0.0.17-1.fc10.x86_64 My mail is stored in a Cyrus imap server.
Also filed as https://bugzilla.redhat.com/show_bug.cgi?id=465358
This is strange. Files do created in /tmp but they should be deleted once indexing is over. Is your /tmp on some non-standard kind of file system which is preventing them from getting deleted ? Can you check if the tmp files are attachments only, email body only or both.
/tmp is a standard ext3 filesystem. I have selinux enabled, but there are no selinux notifications. The files are a mixture of complete mails and just headers. The header-only files all seem to just have Date:, From:, Subject:, To:, and Message-Id: headers, rather than the complete mail headers; I'm guessing they're the only header elements that beagle bothers to index?
If you are still running with the thunderbird indexing enabled, can you send SIGUSR1 to beagled, let it run for 30 mins or so (basically wait for it to index a few thunderbird emails) and attach the beagle log files ~/.beagle/Log/current-*. If you not comfortable attaching the log files, can you look for Error and Exception in those files and report. One other thing, could you pick up any of the emails found in /tmp and then verify that the email is actually an email in thunderbird. I thought thunderbird emails go to a different location other than /tmp.
It's an imap mailbox, so the mail isn't really stored on /tmp. From the log entries, it looks like its putting the mail into tmp files for processing: 20081125 09:24:58.8518 18254 IndexH DEBUG: +file:///home/jeremy/.thunderbird/8wc krzqc.default/ImapMail/mail.goop.org/INBOX/?id=157835 (file:///tmp/tmp3b95fc76.t mp) 20081125 09:24:58.8609 18254 IndexH DEBUG: +file:///home/jeremy/.thunderbird/8wc krzqc.default/ImapMail/mail.goop.org/INBOX/?id=157836 (file:///tmp/tmp74b950d6.t mp) After running for a couple of minutes, it has accumulated 240 /tmp/tmp*.tmp files. There are no obvious errors or exceptions in the beagle log. Do you want me to attach it; it doesn't seem to have anything sensitive in it.
> After running for a couple of minutes, it has accumulated 240 /tmp/tmp*.tmp > files. There are no obvious errors or exceptions in the beagle log. Do you > want me to attach it; it doesn't seem to have anything sensitive in it. Hmm... lets see the log file anyway. Maybe I will spot something interesting. Please attach both the current-Beagle and current-IndexHelper. One more thing, do you know if this problem was present in earlier versions too ?
I wasn't using beagle before, so I don't know; the version I mentioned in the original report was the first I tried. But every subsequent version has the same problem.
Created attachment 123385 [details] IndexHelper log
Created attachment 123386 [details] IndexHelperExceptions log
I didn't include current-Beagle because it has no interesting messages timestamped for today. All it has is: 20081125 11:32:55.0842 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:05.2610 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:15.3521 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:15.3692 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:25.4192 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:25.4364 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:35.4825 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:35.5001 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:45.5482 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:33:55.6212 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex 20081125 11:34:00.7234 03200 Beagle DEBUG: New Thunderbird indexable generator launched for /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ToIndex
Sorry - I got busy in certain things, so I couldnt concentrate on this bug :-(. Can I request some more information ? Stop beagled, delete /home/jeremy/.beagle/Indexes/ThunderbirdIndex/ and start beagled (if necessary, start with "--backend Thunderbird" to only start the TB indexing). Now ... probably within a few minutes there will be stray tmp files, so stop as soon as a few tmp files get created. Can you check if _all_ the files in "+file:///home/jeremy/.thunderbird/8wckrzqc.default/ImapMail/mail.goop.org/INBOX/?id=157835 (file:///tmp/tmp3b95fc76.tmp)" are there ? The code looks perfectly ok; so I am trying to figure out if the tmp files for some specific kind of mails are not removed or it is the case with all emails. Also, if you have the lsof program, can you check if beagled or any other process has those tmp files opened even after beagled has apparently indexed them ?
Another user reported the same problem and with his help, I think I figured out the reason. This problem will show up with a recent mono (mono-2.something). The problem lies in how .net deletes a file. I checked in a workaround in svn r4897. That should fix this problem.
*** Bug 566538 has been marked as a duplicate of this bug. ***