GNOME Bugzilla – Bug 555276
corrupted vfolders; SIGSEGV in camel_folder_get_message()
Last modified: 2009-01-15 13:02:39 UTC
this report has been opened here: https://bugs.edge.launchpad.net/ubuntu/+source/evolution/+bug/279070 "Evolution has been corrupting vfolders and crashing lately; I have lots of duplicate or irretrievable messages that show up in vfolders. Deleting the contents of .evolution/vfolders/ does not appear to help." ".
+ Trace 207836
Thread 3 (process 14541)
Thread 1 (process 24297)
could be related to bug 554990
try deleting ~/.evolution/mail/vfolder/folders.db It is the vfolder cache. I would like to know if this happens just after migration or very repetitive
It is very repetitive. I have deleted vfolder/folders.db several times and the problem comes back. I see a lot of duplicated messages, and a number of "bogus" ones without subjects or text. Clicking on one of those almost invariably causes Evolution to hang for a long period of time and then eventually segfault.
(deleting folders.db *does* help temporarily)
Did this a few more times. Deleting folders.db doesn't always help. Sometimes it seems to actually make things worse (with the number of duplicate messages doubling).
I rely on search folders fairly heavily; this has made Evolution almost unusable for me.
I'm working on this bug next. Lemme try to fix it before 2.24.1
Can you tell me what sort of vfolder it corrupts? Hmm, can you gimme the vfolders.xml file? I can try to read the vfolder cfg. Also, export CAMEL_DEBUG=vfolder and use evolution for some time. It puts some logs on the console and preserve and forward that to me. Just for once, clear vfolder/folders.db and do this test. That way, I can try to understand,how this happens, and then I will fix it.
Created attachment 120604 [details] a log with CAMEL_DEBUG=vfolder I've attached a debug log with the output during a session of Evolution, up until the time it crashed. I had deleted vfolders/folder.db right before starting evolution, but I started seeing duplicate (and then garbage) vfolder entries very quickly, and it didn't take very long after that for evolution to crash.
Incidentally, all of the vfolders are search folders, created on recipient or mailing list addresses.
Tim, Im analysing the trace. Just yet another report. Looks like some mem corruption. 'valgrind --tool=valgrind evolution' This will be a lot slower. Just capture the logs and attach here. Again, clear vfolder/folders.db here also. Thanks a lot.
Created attachment 120649 [details] output from valgrind Here's valgrind output for a session which crashed.
Thanks alot Tim, Its a lot useful. Just a question, do you have vfolders for which sources are some vfolders ? I think, it might be due to that. Lemme do a testing anyways of such a case. Otherwise, I have fixed a code, where such issue would be gone for trash/junk vfolders. Can you give me your vfolders.xml file ? if you build from source, I have a patch, which fixes the same issue for trash/junk. I would like you to run with it. Thanks for your support.
I think I can reproduce the issue, when I have vfolders as source of some vfolders. Sigh. I will try to work on it now and fix before 2.24.1
Created attachment 120672 [details] vfolders.xml Here's my vfolders.xml.
Where is the patch available from?
Tim, forget that patch, its only for junk/trash vfolders. It doesn't fix it. Its due to vfolder of vfolders. I need to find a way to work this out.
Tim, if Im right, you must face this issue/crash in your 'Bug mail' vfolder only.
Nope, I have the problem with all of them.
(The one I use the most, and where I typically have problems, is "Personal Mail".)
Tim, I don't think, I can fix it before 2.24.1 and it might be huge changes with the new design. This vfolder of vfolders, stops other regular vfolders to be working correct. I'm tending towards not-loading such a sub-vfolder for a vfolder for now, so that every thing else works well. But you can reframe your query for such a vfolder, and it might just work well after that. Tim, your points would be valuable here. Thanks.
Tim, remove that vfolder 'Bug mail' vfolder for now, and remove vfolder/folders.db and restart, everything hsould be normal and fine.
Created attachment 120715 [details] [review] Test patch Tim, try this patch for Evo. It just avoids loading a vfolders a subfolder for a vfolder. With this, my vfolders are perfect. I just need to reset the query so that I cover what I want.
I have pushed this to 2.24.1. It must fix vfolder issues. Such vfolders need to be redone. I might still try to do this vfolder of vfolders, a bit later. I would rename this bug post 2.24.1, once you confirm, it works for you. thanks.
The corruption would be fixed in 2.24.1. Please confirm. Thanks
*** Bug 557422 has been marked as a duplicate of this bug. ***
(In reply to comment #25) > The corruption would be fixed in 2.24.1. Please confirm. Thanks I'm afraid not. (In reply to comment #26) > *** Bug 557422 has been marked as a duplicate of this bug. *** This bug was reported on 2.24.1 on Ubuntu's Intrepid. Here's my package list: $ dpkg -l \*evolution\* | cat Desired=Unknown/Install/Remove/Purge/Hold | Status=Not/Inst/Cfg-files/Unpacked/Failed-cfg/Half-inst/trig-aWait/Trig-pend |/ Err?=(none)/Hold/Reinst-required/X=both-problems (Status,Err: uppercase=bad) ||/ Name Version Description +++-==============================================-=====================================-============================================ ii evolution 2.24.1-0ubuntu1 groupware suite with mail client and organiz ii evolution-common 2.24.1-0ubuntu1 architecture independent files for Evolution ii evolution-data-server 2.24.1-0ubuntu1 evolution database backend server ii evolution-data-server-common 2.24.1-0ubuntu1 architecture independent files for Evolution ii evolution-data-server-dbg 2.24.1-0ubuntu1 evolution database backend server with debug ii evolution-dbg 2.24.1-0ubuntu1 debugging symbols for Evolution ii evolution-dev 2.24.1-0ubuntu1 development library files for Evolution rc evolution-exchange 2.11.1-0ubuntu1 Exchange plugin for the Evolution groupware ii evolution-jescs 2.24.0-0ubuntu1 Evolution Connector for Sun Java Enterprise ii evolution-plugins 2.24.0-0ubuntu3 standard plugins for Evolution ii evolution-plugins-experimental 2.24.1-0ubuntu1 experimental plugins for Evolution ii evolution-rss 0.1.0-1ubuntu2 Evolution RSS Reader Plugin ii evolution-webcal 2.24.0-0ubuntu1 webcal: URL handler for GNOME and Evolution Unless you think one of those few ancillary packages which didn't get upgraded (for whatever unknown reason) is the cause. Seems doubtful though.
Yet another on 2.24.1: Program received signal SIGSEGV, Segmentation fault.
+ Trace 208601
Thread 2902457232 (LWP 29481)
Thread 138 (Thread 0xacfffb90 (LWP 29481))
FWIW, here's another. I've included the end of some of evolutions messages: bbdb: Buddy list has changed since last sync. ** (evolution:30670): WARNING **: bbdb: failed to get addressbook: e_book_new: no factories available for URI `' (evolution:30670): camel-CRITICAL **: camel_object_is: assertion `o != NULL' failed (evolution:30670): camel-CRITICAL **: camel_object_ref: assertion `CAMEL_IS_OBJECT(o)' failed [New Thread 0xb0521b90 (LWP 8552)] Program received signal SIGSEGV, Segmentation fault.
+ Trace 208646
Thread 2958171024 (LWP 8552)
Thread 5991 (Thread 0xb0521b90 (LWP 8552))
Thread 1 (Thread 0xb647e940 (LWP 30670))
This is getting annoying. Let me know if you don't need any more stack traces. bbdb: Buddy list has changed since last sync. ** (evolution:8869): WARNING **: bbdb: failed to get addressbook: e_book_new: no factories available for URI `' removing cache for Other Users/jennifer/vmINBOX 1 0xad77cbb8 done .. now 0 removing cache for sent 6 0xb3dda5b8 done .. now 6 (evolution:8869): camel-CRITICAL **: camel_object_is: assertion `o != NULL' failed (evolution:8869): camel-CRITICAL **: camel_object_ref: assertion `CAMEL_IS_OBJECT(o)' failed Program received signal SIGSEGV, Segmentation fault.
+ Trace 208656
Thread 2995776400 (LWP 21555)
Thread 983 (Thread 0xb28feb90 (LWP 21555))
Thread 1 (Thread 0xb62ba940 (LWP 8869))
Did you try removing vfolder/folders.db post 2.24.1 ? Its possible the old vfolder cache is being used. But as I tested it was due to the vfolder being a subfolder of a vfolder. I had made a patch not to load a vfolder as a subfolder, in which case you might have to rework the query better.
(In reply to comment #30) Can you give me your vfolder.xml file?
(In reply to comment #31) > Did you try removing vfolder/folders.db post 2.24.1 ? Not until now, no, but have just done it.
Created attachment 122102 [details] vfolders.xml per comment #30 Per request in comment #30, here is my vfolders.xml.
This also continues to happen to me with 2.24.1. I deleted my vfolder/folder.db also, it works for a while, but then breaks down. It seems like the counting or something like that has a problem. I've gotten in cases where Evo is reporting 4 million unread messages in a folder. I've attached my vfolders.xml also.
Created attachment 122151 [details] vfolders.xml
Ted, which vfolder do you see that corruption of message list, or this crash?
(In reply to comment #36) > Created an attachment (id=122151) [edit] > vfolders.xml > Inkscape, GNOME, To Me, GNOME/Screensaver , Ubuntu/Wiki, Ubuntu/Devel Discuss, Inkscape/GSoC Mentor, GNOME/Pulse Audio, , NTLUG/Discuss, Wikipedia Daily, are the vfolders, which has a vfolder as a subfolder. This doesn't work now. Do you see the crash in this folder? Or any other folder?
So I went through all my instances of "vfolder:" in my vfolders.xml file and I removed them. Which is a little odd, because only one of them was actually active while the others weren't using the specific folders entries. But, they're all gone. And Evolution has only hung once on me today (huge improvement) and I've been using vFolders almost all day. It still doesn't get the folder count right (attached a PNG) though. So, I guess the patch is to catch the instances of "vfolder:" in the XML earlier so that the rest of the code doesn't get confused. Either that or a migration script for users.
Created attachment 122209 [details] Preemptive reading of mail. I'm just that good.
I think it should also be noted that when creating a new saved search when you have a vfolder selected auto-populates the list of folders for that saved search folder with the vfolders. It seems that becasue of that, there are likely to be a bunch of users that have folders configured that way.
Ted, is your issue is with counts, then it is a different bug. Im loosing track on this bug, which is on corruption of vfolders and the crasher. gnome-bugs@interlinx.bc.ca: You happen to see this again?
This is still alive and well, on 2.24.1 as of just 30 minutes ago: Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xa60b3b90 (LWP 26776)] 0xb7d9acf9 in vee_get_message (folder=0xa764dc0, uid=0xd873b20 "����DLQC50483", ex=0xbb484d4) at camel-vee-folder.c:636 636 camel-vee-folder.c: No such file or directory. in camel-vee-folder.c (gdb) apply thread all bt full Undefined command: "apply". Try "help". (gdb) thread apply all bt full
+ Trace 209602
Thread 19076 (Thread 0xa60b3b90 (LWP 26776))
This goes along with not always seeing all new/unread messages in my vfolder. Frequently I have to go back to the real folders to see new/unread messages that the vfolder did not show.
(In reply to comment #43) > This is still alive and well, on 2.24.1 as of just 30 minutes ago: > > > This goes along with not always seeing all new/unread messages in my vfolder. > Frequently I have to go back to the real folders to see new/unread messages > that the vfolder did not show. > These are the two bugs, that I'm just working on to figure out why it happens, and works for me and a few others only. Anyways... I hope I must fix it before 2.24.2
Do you see a warning like 'VFolder of VFolders not supporting. Ignoring loading this vfolder as a subfolder' on the console? Im just trying to make sure that Im on the right direction.
(In reply to comment #45) > Do you see a warning like 'VFolder of VFolders not supporting. Ignoring loading > this vfolder as a subfolder' on the console? Im just trying to make sure that > Im on the right direction. I have seen that warning in the past, yes. Im not sure if I've seen it with any particular crash though.
I think Im good. I see some cases where vfolders are still loaded as subfolders. There is more than one path, which I must seal. Thanks for the help. lemme do some more, before asking for more. Thanks for all your support.
Created attachment 122395 [details] [review] Test patch Can you try with this patch?
Patch installed. I notice that I am now seeing mails in the vfolder that I was not seeing before and had to go back to the real folder to see. Also, I notice that the vfolders of vfolders that I do have are all showing as empty when you click on them despite them showing as having unread mail in them. I suspect this is expected. This limitation of vfolders of vfolders is really quite annoying though. I would have opted to hold back the sqlite paradigm rollout and keep what we had until the sqlite paradigm was at least as functional as the status quo. :-(
> This limitation of vfolders of vfolders is really quite annoying though. I > would have opted to hold back the sqlite paradigm rollout and keep what we had > until the sqlite paradigm was at least as functional as the status quo. :-( > I understand. But you know, in earlier code, everything was in memory and searched and kept. Now with this, a vfolder is just a sql query on db. And querying a vfolder for another vfolder is just impossible. I can hack it and still make it, but it won't be efficient. Lemme know how this goes. I hope it shouldn't crash anymore. I will wait for some time before closing this bug. Thanks a lot for your patience and support. Im off to unread vfolder fully now.
(In reply to comment #50) > > I understand. But you know, in earlier code, everything was in memory and > searched and kept. Now with this, a vfolder is just a sql query on db. But with a couple of pretty visible and serious regressions. vfolders in vfolders is one and read (or more unexpectedly, replied-to) messages all of a sudden disappearing from a vfolder is another. For some people, those are very visible and painful regressions. Having to go back to the real folder to delete a message that was in the vfolder that you just replied to is a pain too. > And > querying a vfolder for another vfolder is just impossible. Which is why I would have advocated holding back the sqlite feature until it was feature compatible. > Lemme know how this goes. I hope it shouldn't crash anymore. Good. We will see I guess. > I will wait for > some time before closing this bug. OK. > Thanks a lot for your patience and support. No, thank you for your effort. I have just been a pain in the ass for you. :-)
Committed to stable/trunk.
As I’m also affected by the regression of search folders: would it not be possible to implement vfolders of vfolders by combining the search criteria with and? I don’t know how the implementation exactly works but I would imagine it something like this: Search folder A: SELECT … FROM … WHERE argumentsA Search folder B: SELECT … FROM … WHERE argumentsB Then if folder B queries folder A the query would be: SELECT … FROM … WHERE argumentsA AND argumentsB
If it is not possible to do the user interface should disallow selecting a search folder as a search folder source and when upgrading from earlier versions update the existing search folders so that they don’t use other search folders as source. The situation now is just bad. How will a random user know of this limitation?
This is still possible to be done, not a limitation. But surely its not a combination of such SQL queries. But this needs a bit of redesign to the core piece, where as Im still fixing bugs around. This, Unmatched vfolders are the two limitations that Im aware of.
*** Bug 567754 has been marked as a duplicate of this bug. ***