GNOME Bugzilla – Bug 202917
IMAP provider: when doing a LIST, use "%" as the wildcard rather than "*" so it's not recursive
Last modified: 2005-11-15 02:22:21 UTC
When trying to retrieve an IMAP folder that has *cough* 7000 messages, nothing happens until eventually the machine locks up. (The progress window pops up, but I don't think it's designed to show anything other than 0% or 100%?) I tried running a "cat /proc/meminfo" and saw free memory steadily decline by at least 100MB while waiting for the imap folder to be retrieved. The machine locked hard shortly afterwards. (Actually I have done 5 hard resets today, all locks while Evolution was running). I have a smaller IMAP folder (usually only 10-20 messages) that seems to work ok. If you have any other suggestions for debugging this I'll be happy to help.
What exactly were you doing when it locked up? Had it connected to the IMAP server, had you clicked on a folder yet, etc. You should be able to get IMAP debugging info by running evolution as "evolution --debug filename" (and it will output debug stuff, including an IMAP trace, to that file).
(Sorry if this shows up 2x, I think Galeon didn't submit correctly...?) Thanks, I tried the --debug option as "evolution --debug=ev-debug" and I didn't get much out of it - on startup I got <snip> Folder registered successfully -- /local.cshrc local.cshrc Folder registered successfully -- /.profile .profile Folder registered successfully -- /Trash Trash and then starting send/receive yielded no further messages. FWIW, I'm running from the Ximian RPM - does it have debugging enabled? The system hang was traced to bad memory, but it still seems to be evolution chewing through memory that got me to the bad portion of my ram. :) In terms of when this happened, I fire up evolution, click the OK button after the hellfire-and-damnation warning dialog, and then click "send/receive" The status dialog comes up, showing my small imap mailbox quickly completed, but shows no progress on my large mailbox. Again, the lockup is likely bad memory, but I think the reason I hit that bad memory was that evolution was chewing up memory like mad. And the end result is the same, I'm unable to load this large imap mailbox. Thanks!
Dan was mistaken, you'll have to run evolution-mail in its own xterm. export CAMEL_VERBOSE_DEBUG=1 evolution-mail wait like 10 seconds and then start evolution in another xterm Dan will probably want most of the debug output that the evolution-mail terminal prints out (I forget if it goes to stderr or stdout) Also, 7,000 messages isn't all that much. I have a good 70,000 and it works fine for me. Is this the first time you've connected to this IMAP server? or the first time you've ever opened this IMAP folder in Evolution? The first time you open a folder on an IMAP server takes a bit longer because it has to generate a summary of the folder based on retrieving the message headers from all the messages in the folder first. This could be the problem?
I did try clearing out the whole evolution dir tree in my home directory, so yes, this is effectively the first time. How long should 7000 messages take, though? This is on the order of at least 10 minutes. The only other interesting thing is that I'm doing this through an ssh tunnel, so my "imap server" is localhost:9143, which gets forwarded. I'll try the debugging output and see what I get. Thanks, -Eric
Ahah, this debugging info can be useful. :) Looks like it's going through every file in my home directory on the server... thousands of lines like this: received: * LIST (\NoInferiors \UnMarked) "/" Desktop/sandeen/xfs/linux/workarea/linux/drivers/media/video/tvmixer.c and so on, through a couple Linux and Irix kernel source trees. :) Is this a misconfiguration on my end, or on the server end? I didn't have this trouble with previous evolution versions, AFAIK... but I may have just put more stuff in my home dir so now it takes longer.
Ok, taking the kernel trees out of my home dir solved the problem, and things are working fairly well at this point. Thanks for the pointer to the debugging info, that cleared it up, still wondering where the misconfiguration is, though. Thanks!
Okay, what you might want to do is in the account configuration dialog, set it to "Only view subscribed folders" under the "Receiving Options" tab. In the future, we plan on making it not LIST recursively, so it's kinda of a bug on our part as well.
when doing a LIST, use "%" as the wildcard rather than "*" so it's not recursive. this means we'll need to allow for incremental folder listing
fejj or danw: is this future? can we reduce the priority from Blocker?
We need to do *something* about this issue for 1.0
*** bug 204957 has been marked as a duplicate of this bug. ***
The subscribe dialog implements this correctly now. The main folder tree doesn't, because it has no way of knowing when a node has been expanded. (If this is implemented, the code to do this in subscribe-dialog.c will drop in to the shell stuff pretty well, I think.)
Since this has been done correctly in the subscribe dialogue, can we drop this from Blocker?
Um.... hm. As far as I know this is still a problem with the folder tree, where it's going to be a much bigger problem. Also, the solution for the subscribe dialog is not really portable to the folder tree because we need to traverse the CORBA bridge. So I guess this is still a Blocker, but I have no clue what we're going to do about it.
Hm. We should reorganize how it works so that the mailer code makes a series of not-recursive calls like it does for the subscribe dialog, and adds each level to the folder tree as it gets it, and wrap the whole thing inside a single cancellable operation. So then, if the user configures it to suck in his entire homedir, it will be obvious that it's doing it, and he'll be able to stop it.
In my reading this is no longer blocker, but I'm a little iffy. Either way, it does sound like something we should at least think about for immediately post-1.0 so putting it on that radar. If anyone disagrees please feel free to comment/mod/whatever.
i'm not sure a user will be able to stop it anyway (you should see the delays yout get when you try and add folders as they're discovered, the shell overhead is enormous). What about if we just abort if it goes over a certain total size (and make an env variable for those inevitable people that have a million folders?). The size wouldn't even have to be that big by default, like 1000 or so would probably do. And it should be pretty easy to write ... (although i think the only way to cancel the list is to drop the connection?). I think we need something that'll at least not use up all their memory!
Because of the decision to remap 1.1->1.2 and 1.2->1.4, I'm going to be moving a large number of bugs around in the bugzilla. You can just search on 'body contains' 'Because of the decision to remap' and mark all as read. Please direct all questions about this change to evolution@ximian.com, not the bug. Luis
Danw, did your "Connecting..." folder tree fix affect this?
There's no reason to suggest it would have.
i dont think this should be for 1.2, as we're just going to throw the code away later anyway. but i'll look into it.
ok, futuring, unless there is an easy way to fix this.
im going to m,ark his fixed. it no longer uses * for list - but i think the original intention was to have the code only list as the user selects the folder. this doesnt play well with vfolders, and infact most uses of vfolders would mean it opens all folders anyway, so i dont think the effort is worth it.