GNOME Bugzilla – Bug 311656
Slow pop3 - due to spamassassin
Last modified: 2009-11-20 05:28:20 UTC
Please describe the problem: When getting my e-mail from usa.net (pop.netaddress.com), Evolution always download all email headers, which is very very slow. I have "Leave messages on server" enabled. Mozilla Thunderbird, Outlook and Outlook Express do not have this problem with the same server and the same settings. I have seen this reported several times: http://bugzilla.gnome.org/show_bug.cgi?id=234322 Marked as FIXED, but it clearly isn't. http://bugzilla.gnome.org/show_bug.cgi?id=212064 Marked as NOTABUG. http://bugzilla.gnome.org/show_bug.cgi?id=233750 Marked as NOTABUG. The explanation of why it isn't a bug isn't at all convicing. The fact is, other e-mail clients obviously don't have this problem, so it clearly is a bug. I recently switched from Thunderbird myself and while everything else in Evolution rocks, the slow POP3 access is almost intolerable. Steps to reproduce: Evolution has a problem with pop.netaddress.com. Unfortunately accounts are not free there, so reproducing it freely might be a problem. Judging from other bug reports Yahoo pop access also has the problem, but again pop access there is not free either. Actual results: I wait ... Expected results: I expect the mail check to be almost instant. Does this happen every time? Yes. Other information: Please, fix this ! It is a real shame that OutlookExpress behaves better than Evolution ;-)
> http://bugzilla.gnome.org/show_bug.cgi?id=234322 > Marked as FIXED, but it clearly isn't. actually, it *is* fixed. and is also not completely related to your problem... the problem in that report is that some POP servers do not support the CAPA command and so Evolution had to be fixed to test if UIDL worked even if CAPA failed. It does this now. YOUR problem is that the server does not support UIDL at all and so in order to come up with unique identifiers for each message (to ensure we don't keep downloading the same messages over and over again and also to assure that we don't mistakenly think we've already downloaded a message that we haven't), we get the md5sums of the headers to use as our identifiers. Now, if you can come up with a better way, please share. otherwise this is NOTABUG since we are doing what we must in order to solve the problem of redownloading messages and/or not downloading messages that we should. This problem takes a back seat to either one of those.
I am not a POP3 expert, so I don't know how else to explain this. Here is the evidence: All other e-mail clients that I have used: Outlook, Outlook Exress, Mozilla Thunderbird, _do not have this problem_. Thunderbird and Outlook Express take 1 second to check my e-mail on the same server. Evolution takes 20 seconds or more. So, obviously there is a better way of doing it. I am not a POP3 expert, so I have no idea how they do it, but if neccessary I can attach logs from Outlook Express, which can be examined. Thunderbird is open source and also can be examined. Perhaps they are using a method that is not 100% reliable ? Even so, on the rare occasions when e-mail messages are mistakenly downloaded twice, they can be discarded at that time based on the md5 sum. In the face of such evidence, why is this considered not worth fixing ? I can understand if it is not first priority, but this is a problem that we, the users, do have, it is a big pain and for some probably a sufficient reason not to use Evolution (for me the benefits of Evolution outweigh this problem, but I am also hoping that it will eventually be fixed)
I have received confirmation from USA.NET that their POP3 server does indeed support UIDL. Apparently many servers do not correctly report that they support UIDL, but they support it nevertheless. KMail allows the user to override this setting and when I test with KMail it works fast with all my pop3 accounts. Currently Evolution is unbearably slow with two out of three accounts. Since changing e-mail servers is much harder than changing e-mail programs, I implore you to reconsider fixing this problem. Otherwise I am forced to use KMail from within Gnome.
well, if you have some time, rewrite evolution's code, i'd say. :-/
I have to say that your problem don't ocour on my pc. I have 2 pop3 account. One is free (freemail.web.de) and the other is my provider. In my optinon evolution is faster than MS Outlook and other stuff. You could try to check the "Disable pop3 extentions" (or enable them?)
I've the same problem, I posted a bit explanation in: http://bugzilla.gnome.org/show_bug.cgi?id=432470#c2
I attach a file with info provided by wireshark
Created attachment 94862 [details] wireshark output
pacho plz repeat sniffing with wireshark and then press the Right mouse offer a line and press "Follow tcp stream" (the line should be a line where protocol is pop) then please poste the resulting text ( but erease your password &username BEFOR posting !!) then please install wireshark on windows and do the same thing with an other email client (eg. outlook ) or try with sunderbird if it is faster yours
(In reply to comment #9) > pacho plz repeat sniffing with wireshark and then press the Right mouse offer a > line and press "Follow tcp stream" (the line should be a line where protocol is > pop) > The problem is that I get a text with: ....!....j.K..& ...:.. ..Y...-.. ...........u.c.......N.....~....S.......,:J..y....S.<..........,..d..k...N.po_..+zV6v.....C..}-h?u?.~`.a....>.NI.t....A.1t..Lxx..{!*.....C...h.K.\..w..x...R..D..jpr.....@c47.x...d....3#.h..W..&...8..{.....~w.......H...c.[2{o..}.c...}.tR.r<U...9...nh.RU.G.U."Q..Fx...2.....(..}...dF.k.M/(>...l .U.B.]...5..v..c.....6.....r.Z..+....Y%...#.....a.-...lk..2....$r...~.%..k.../...q{.!.` .. .... ... ... Is this normal? > then please poste the resulting text ( but erease your password &username BEFOR > posting !!) > I don't find my password or username in the text :-/ Thanks alot
ok i'll try a step by step: 1. start wireshark 2. First button in Toolbar ("List the avalible capture interfaces") 3. press options at the correct interface (the internet interface) 4. "Capture Filter "(the button) 5. Filtername : xxx Filter string : "tcp port 110" 6. ok > start 7. send/recieve your mails 8. stop capturing 9. search for lines where "protocol" is pop 10. right click on that line and choose "Follow TCP Stream" 11. window with text pops up 12. try choosing ascii oder ebcdic if there is no readable text on my system this looks like this: > http://pastebin.ubuntu-nl.org/36327/ if you have problems plz check if you are using encrypted pop. You can check that at the preferences at your mailprogramm. If you don't caputure anything do the following things: check if you use secure pop (> other port number) check if you used the correct interface yours
OK, thanks, I will attach the files, I needed to disable TLS for getting readable files :-)
Created attachment 94951 [details] wireshark evolution Evolution test
Created attachment 94952 [details] thunderbird wireshark With thunderbird
As you can see, during the test, thunderbird downloaded a lot of mail when evolution only downloaded a few mails ... :-/
seems to be very strange: the pop interaction between evolution and the pop server seems to be quite the same that thunderbird. cause i'm not an developer on evolution i cant investigate that further. But i hope that the evolution team will be able to track that bug down with the information you provided. the only thing i see is that evolution does not send an AUTH command. yours
What about this anoying bug? Will never be fixed?
Evolution is using the exact same commands as thunderbird (your server advertises that it supports UIDL and both evo and thunderbird use that) Perhaps your slowness is filtering or fsync()
Sorry buy, How could I check this fsync or filtering problem? Thanks a lot :-)
After each message that Evolution downloads and saves to disk, it calls fsync() to make sure all the buffers for that file are flushed to disk so that it is sure that the message has successfully been written to disk. So... if you've got a lot of disk activity, that might suggest that fsync() is slowing pop fetching down (which it most likely is, just depends on how noticeable the slowdown is). Probably most noticeable w/ slow disks (like 5400 RPM laptop drives). It could be filters if you have a lot of them and/or if you have filters that search the message body.
I have 8 filters now (for mailling lists and summaries), I have just removed one rule that searched message body. I am trying now Also, about fsyc, it is always slow, also when disk activity is low :-/ Thanks a lot :-)
Without the rule that searched on body it's still slow :-(
http://www.gnome.org/~federico/news-2007-10.html#30 that blog entry suggests that spam filtering is pretty slow... perhaps your experiences are related to spam filtering as well?
Yes, you are true :-O Now it's faster but, Is this a spamassassin bug? A evolution bug with its spamassassin plugin? Can I use other spam filter with evolution? Thanks a lot :-D
> Is this a spamassassin bug? not sure... > A evolution bug with its spamassassin plugin? possibly, not really sure > Can I use other spam filter with evolution? Yes :) There's a bogofilter plugin somewhere. Sadly I'm not sure where to find it. Google will probably know... > Thanks a lot :-D No prob, glad to be of help... At least the slowness has been narrowed down a bit now ;)
Maybe if evolution only checks mail after downloading ALL of them (like thunderbird seems to do) instead of checking each message while it's getting them could help a bit :-/
Bogofilter is much faster than spamassassin, I am now using it :-) on evolution-2.10.3 (evolution-2.12 includes it as "official")
Can anybody modify the summary for showing that this is caused by spamassasin? Thanks
I am the original submitter of this bug. Since I submitted it 2 years ago, I unfortunately had to abandon Evolution because it is unusable for most of my e-mail accounts. I am replying to the suggestion to change he summary: What I experienced and submitted was definitely not caused by spamassassin because I didn't have it installed anywhere. IIRC, there was some misunderstanding between Evolution and the POP3 server about supporting the UIDL command.
To whoever changed the bug summary. This bug is NOT caused by spamassasin. Perhaps there is problem caused by spamassasin - in that case a new bug should be opened ! Changing the summary so that it describes a different problem is certainly a very novel way of fixing bigs. Good work !!
Camel disk summary has been merged in 2.23.x, it should definitely speed up the process. There are improvements in the way evolution handles spam/junk in 2.22.
This version is no longer maintained, which means that it will not receive any further security or bug fix updates. The current stable GNOME and Evolution version is 2.26. Can you please check again whether this issue still happens in Evolution 2.24 or 2.26 and update this report by adding a comment and changing the "Version" field? Thanks a lot. Again thank you for reporting this bug and we are sorry it could not be fixed for the version you originally used here. Without feedback this report will be closed as INCOMPLETE in 6 weeks.
Seems to be faster for me with 2.24 (bogofilter is still faster, but speed with spamassassin seems reasonable to me)
Please feel free to reopen the bug if the problem still occurs with a newer version of GNOME 2.28.1 or later, thanks.