After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 311656 - Slow pop3 - due to spamassassin
Slow pop3 - due to spamassassin
Status: RESOLVED OBSOLETE
Product: evolution
Classification: Applications
Component: Mailer
2.2.x (obsolete)
Other All
: Normal normal
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
evolution[pop]
Depends on:
Blocks:
 
 
Reported: 2005-07-26 20:38 UTC by Tzvetan Mikov
Modified: 2009-11-20 05:28 UTC
See Also:
GNOME target: ---
GNOME version: 2.7/2.8


Attachments
wireshark output (111.51 KB, text/plain)
2007-09-03 12:56 UTC, Pacho Ramos
Details
wireshark evolution (61.04 KB, text/plain)
2007-09-04 18:56 UTC, Pacho Ramos
Details
thunderbird wireshark (958.11 KB, text/plain)
2007-09-04 19:07 UTC, Pacho Ramos
Details

Description Tzvetan Mikov 2005-07-26 20:38:28 UTC
Please describe the problem:
When getting my e-mail from usa.net (pop.netaddress.com), Evolution always
download all email headers, which is very very slow. I have "Leave messages on
server" enabled.

Mozilla Thunderbird, Outlook and Outlook Express do not have this problem with
the same server and the same settings.

I have seen this reported several times:
http://bugzilla.gnome.org/show_bug.cgi?id=234322
  Marked as FIXED, but it clearly isn't.
http://bugzilla.gnome.org/show_bug.cgi?id=212064
  Marked as NOTABUG. 
http://bugzilla.gnome.org/show_bug.cgi?id=233750
  Marked as NOTABUG.

The explanation of why it isn't a bug isn't at all convicing. The fact is, other
e-mail clients obviously don't have this problem, so it clearly is a bug. I
recently switched from Thunderbird myself and while everything else in Evolution
rocks, the slow POP3 access is almost intolerable.


Steps to reproduce:
Evolution has a problem with pop.netaddress.com. Unfortunately accounts are not
free there, so reproducing it freely might be a problem.
Judging from other bug reports Yahoo pop access also has the problem, but again
pop access there is not free either.

Actual results:
I wait ...

Expected results:
I expect the mail check to be almost instant.

Does this happen every time?
Yes.

Other information:
Please, fix this ! It is a real shame that OutlookExpress behaves better than
Evolution ;-)
Comment 1 Jeffrey Stedfast 2005-07-27 15:19:54 UTC
> http://bugzilla.gnome.org/show_bug.cgi?id=234322
>  Marked as FIXED, but it clearly isn't.

actually, it *is* fixed. and is also not completely related to your problem...
the problem in that report is that some POP servers do not support the CAPA
command and so Evolution had to be fixed to test if UIDL worked even if CAPA
failed. It does this now.

YOUR problem is that the server does not support UIDL at all and so in order to
come up with unique identifiers for each message (to ensure we don't keep
downloading the same messages over and over again and also to assure that we
don't mistakenly think we've already downloaded a message that we haven't), we
get the md5sums of the headers to use as our identifiers.

Now, if you can come up with a better way, please share. otherwise this is
NOTABUG since we are doing what we must in order to solve the problem of
redownloading messages and/or not downloading messages that we should.

This problem takes a back seat to either one of those.
Comment 2 Tzvetan Mikov 2005-07-27 16:55:15 UTC
I am not a POP3 expert, so I don't know how else to explain this. Here is the
evidence:

All other e-mail clients that I have used: Outlook, Outlook Exress, Mozilla
Thunderbird, _do not have this problem_. Thunderbird and Outlook Express take 1
second to check my e-mail on the same server. Evolution takes 20 seconds or more. 

So, obviously there is a better way of doing it. I am not a POP3 expert, so I
have no idea how they do it, but if neccessary I can attach logs from Outlook
Express, which can be examined. Thunderbird is open source and also can be examined.

Perhaps they are using a method that is not 100% reliable ? Even so, on the rare
occasions when e-mail messages are mistakenly downloaded twice, they can be
discarded at that time based on the md5 sum. 

In the face of such evidence, why is this considered not worth fixing ? I can
understand if it is not first priority, but this is a problem that we, the
users, do have, it is a big pain and for some probably a sufficient reason not
to use Evolution (for me the benefits of Evolution outweigh this problem, but I
am also hoping that it will eventually be fixed)

Comment 3 Tzvetan Mikov 2005-11-17 23:23:04 UTC
I have received confirmation from USA.NET that their POP3 server does indeed
support UIDL. Apparently many servers do not correctly report that they support
UIDL, but they support it nevertheless.

KMail allows the user to override this setting and when I test with KMail it
works fast with all my pop3 accounts. Currently Evolution is unbearably slow
with two out of three accounts.

Since changing e-mail servers is much harder than changing e-mail programs, I
implore you to reconsider fixing this problem. Otherwise I am forced to use
KMail from within Gnome.
Comment 4 André Klapper 2006-07-05 19:29:10 UTC
well, if you have some time, rewrite evolution's code, i'd say. :-/
Comment 5 bernhardredl 2006-11-13 22:04:07 UTC
I have to say that your problem don't ocour on my pc. I have 2 pop3 account. One is free (freemail.web.de) and the other is my provider. 

In my optinon evolution is faster than MS Outlook and other stuff.

You could try to check the "Disable pop3 extentions" (or enable them?)
Comment 6 Pacho Ramos 2007-09-03 11:37:34 UTC
I've the same problem, I posted a bit explanation in:
http://bugzilla.gnome.org/show_bug.cgi?id=432470#c2
Comment 7 Pacho Ramos 2007-09-03 12:55:59 UTC
I attach a file with info provided by wireshark
Comment 8 Pacho Ramos 2007-09-03 12:56:31 UTC
Created attachment 94862 [details]
wireshark output
Comment 9 bernhardredl 2007-09-03 18:28:07 UTC
pacho plz repeat sniffing with wireshark and then press the Right mouse offer a line and press "Follow tcp stream" (the line should be a line where protocol is pop)

then please poste the resulting text ( but erease your password &username BEFOR 
posting !!)

then please install wireshark on windows and do the same thing with an other email client (eg. outlook ) or try with sunderbird if it is faster

yours
Comment 10 Pacho Ramos 2007-09-03 19:17:09 UTC
(In reply to comment #9)
> pacho plz repeat sniffing with wireshark and then press the Right mouse offer a
> line and press "Follow tcp stream" (the line should be a line where protocol is
> pop)
> 

The problem is that I get a text with:
....!....j.K..&
...:..
..Y...-..
...........u.c.......N.....~....S.......,:J..y....S.<..........,..d..k...N.po_..+zV6v.....C..}-h?u?.~`.a....>.NI.t....A.1t..Lxx..{!*.....C...h.K.\..w..x...R..D..jpr.....@c47.x...d....3#.h..W..&...8..{.....~w.......H...c.[2{o..}.c...}.tR.r<U...9...nh.RU.G.U."Q..Fx...2.....(..}...dF.k.M/(>...l
.U.B.]...5..v..c.....6.....r.Z..+....Y%...#.....a.-...lk..2....$r...~.%..k.../...q{.!.`
..
....

...
...


Is this normal?


> then please poste the resulting text ( but erease your password &username BEFOR 
> posting !!)
>

I don't find my password or username in the text :-/

Thanks alot 
Comment 11 bernhardredl 2007-09-04 14:05:23 UTC
ok i'll try a step by step:

1. start wireshark
2. First button in Toolbar ("List the avalible capture interfaces")
3. press options at the correct interface (the internet interface)
4. "Capture Filter "(the button)
5. Filtername : xxx Filter string : "tcp port 110"
6. ok > start
7. send/recieve your mails
8. stop capturing
9. search for lines where "protocol" is pop
10. right click on that line and choose "Follow TCP Stream"
11. window with text pops up
12. try choosing ascii oder ebcdic if there is no readable text

on my system this looks like this:
> http://pastebin.ubuntu-nl.org/36327/

if you have problems plz check if you are using encrypted pop. You can check that at the preferences at your mailprogramm. 

If you don't caputure anything do the following things:
 check if you use secure pop (> other port number)
 check if you used the correct interface

yours
Comment 12 Pacho Ramos 2007-09-04 18:53:55 UTC
OK, thanks, I will attach the files, I needed to disable TLS for getting readable files :-)
Comment 13 Pacho Ramos 2007-09-04 18:56:27 UTC
Created attachment 94951 [details]
wireshark evolution

Evolution test
Comment 14 Pacho Ramos 2007-09-04 19:07:43 UTC
Created attachment 94952 [details]
thunderbird wireshark

With thunderbird
Comment 15 Pacho Ramos 2007-09-04 19:08:57 UTC
As you can see, during the test, thunderbird downloaded a lot of mail when evolution only downloaded a few mails ... :-/
Comment 16 bernhardredl 2007-09-04 20:29:43 UTC
seems to be very strange:
the pop interaction between evolution and the pop server seems to be quite the same that thunderbird.

cause i'm not an developer on evolution i cant investigate that further. But i hope that the evolution team will be able to track that bug down with the information you provided.

the only thing i see is that evolution does not send an AUTH command.

yours
Comment 17 Pacho Ramos 2007-10-26 17:23:01 UTC
What about this anoying bug?

Will never be fixed?
Comment 18 Jeffrey Stedfast 2007-10-26 17:33:42 UTC
Evolution is using the exact same commands as thunderbird (your server advertises that it supports UIDL and both evo and thunderbird use that)

Perhaps your slowness is filtering or fsync()
Comment 19 Pacho Ramos 2007-10-27 10:35:22 UTC
Sorry buy, How could I check this fsync or filtering problem? 

Thanks a lot :-)
Comment 20 Jeffrey Stedfast 2007-10-27 14:08:20 UTC
After each message that Evolution downloads and saves to disk, it calls fsync() to make sure all the buffers for that file are flushed to disk so that it is sure that the message has successfully been written to disk.

So... if you've got a lot of disk activity, that might suggest that fsync() is slowing pop fetching down (which it most likely is, just depends on how noticeable the slowdown is). Probably most noticeable w/ slow disks (like 5400 RPM laptop drives).


It could be filters if you have a lot of them and/or if you have filters that search the message body.
Comment 21 Pacho Ramos 2007-10-27 15:30:17 UTC
I have 8 filters now (for mailling lists and summaries), I have just removed one rule that searched message body. I am trying now

Also, about fsyc, it is always slow, also when disk activity is low :-/

Thanks a lot :-)
Comment 22 Pacho Ramos 2007-10-27 16:00:33 UTC
Without the rule that searched on body it's still slow :-(
Comment 23 Jeffrey Stedfast 2007-10-30 19:01:15 UTC
http://www.gnome.org/~federico/news-2007-10.html#30

that blog entry suggests that spam filtering is pretty slow... perhaps your experiences are related to spam filtering as well?
Comment 24 Pacho Ramos 2007-10-30 19:31:54 UTC
Yes, you are true :-O

Now it's faster but, Is this a spamassassin bug? A evolution bug with its spamassassin plugin? Can I use other spam filter with evolution?

Thanks a lot :-D
Comment 25 Jeffrey Stedfast 2007-10-30 20:49:53 UTC
> Is this a spamassassin bug?

not sure...

> A evolution bug with its spamassassin plugin?

possibly, not really sure

> Can I use other spam filter with evolution?

Yes :)

There's a bogofilter plugin somewhere. Sadly I'm not sure where to find it. Google will probably know...

> Thanks a lot :-D

No prob, glad to be of help...

At least the slowness has been narrowed down a bit now ;)
Comment 26 Pacho Ramos 2007-10-31 18:48:50 UTC
Maybe if evolution only checks mail after downloading ALL of them (like thunderbird seems to do) instead of checking each message while it's getting them could help a bit :-/
Comment 27 Pacho Ramos 2007-11-01 11:24:04 UTC
Bogofilter is much faster than spamassassin, I am now using it :-) on evolution-2.10.3 (evolution-2.12 includes it as "official")
Comment 28 Pacho Ramos 2007-12-22 08:59:14 UTC
Can anybody modify the summary for showing that this is caused by spamassasin?

Thanks
Comment 29 Tzvetan Mikov 2007-12-22 19:23:59 UTC
I am the original submitter of this bug. Since I submitted it 2 years ago, I unfortunately  had to abandon Evolution because it is unusable for most of my e-mail accounts. 

I am replying to the suggestion to change he summary: What I experienced and submitted was definitely not caused by spamassassin because I didn't have it installed anywhere. IIRC, there was some misunderstanding between Evolution and the POP3 server about supporting the UIDL command.

Comment 30 Tzvetan Mikov 2007-12-23 01:16:14 UTC
To whoever changed the bug summary. This bug is NOT caused by spamassasin. Perhaps there is problem caused by spamassasin - in that case a new bug should be opened !

Changing the summary so that it describes a different problem is certainly a very novel way of fixing bigs. Good work !!


Comment 31 Akhil Laddha 2008-08-08 10:09:35 UTC
Camel disk summary has been merged in 2.23.x, it should definitely speed up the process. There are improvements in the way evolution handles spam/junk in 2.22.  
Comment 32 Akhil Laddha 2009-09-10 13:15:07 UTC
This version is no longer maintained, which means that it will not receive any
further security or bug fix updates.
The current stable GNOME and Evolution version is 2.26.

Can you please check again whether this issue still happens in Evolution 2.24
or 2.26 and update this report by adding a comment and changing the "Version"
field? Thanks a lot.

Again thank you for reporting this bug and we are sorry it could not be fixed
for the version you originally used here.

Without feedback this report will be closed as INCOMPLETE in 6 weeks.
Comment 33 Pacho Ramos 2009-09-10 15:54:06 UTC
Seems to be faster for me with 2.24 (bogofilter is still faster, but speed with spamassassin seems reasonable to me)
Comment 34 Akhil Laddha 2009-11-20 05:28:20 UTC
Please feel free to reopen the bug if the problem still occurs with a newer
version of GNOME 2.28.1 or later, thanks.