After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 245683 - default evo configuration results in unneeded >From munging
default evo configuration results in unneeded >From munging
Status: RESOLVED FIXED
Product: evolution
Classification: Applications
Component: Mailer
pre-1.5 (obsolete)
Other All
: Normal enhancement
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
: 401665 428728 440634 441865 475359 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2003-07-01 14:58 UTC by Dan Winship
Modified: 2009-10-15 18:46 UTC
See Also:
GNOME target: ---
GNOME version: Unversioned Enhancement


Attachments
proposed patch 1/2 (1.37 KB, patch)
2007-05-23 07:10 UTC, Milan Crha
rejected Details | Review
proposed patch 2/2 (7.29 KB, patch)
2007-05-23 07:13 UTC, Milan Crha
rejected Details | Review
proposed evo patch ][ (5.24 KB, patch)
2009-09-08 15:56 UTC, Milan Crha
committed Details | Review

Description Dan Winship 2003-07-01 14:58:59 UTC
In the default configuration, all outgoing messages get passed through
CamelMimeFilterFrom, because Outbox is an mbox folder. This isn't very
noticeable to other evo users since the display code hacks around it,
but it looks bad in other mailers, or when you reply to the message.

An easy fix for this would be to make Outbox be Maildir by default.
Comment 1 Jeffrey Stedfast 2003-10-28 18:36:33 UTC
can't do this anymore for 2.0 so closing
Comment 2 Matthew Barnes 2007-05-21 16:49:14 UTC
Reopening.  Milan Crha is able to reproduce this bug in Evolution 2.10.
Comment 3 Milan Crha 2007-05-21 17:04:00 UTC
I could reproduce this by simply typing new mail with text:"
From line1
From line2
>From line3 already marked
From line4
". And when I save this mail as draft on local folder, then I could see after opening this message this text:"
>From line1
From line2
>From line3 already marked
>From line4
", also in message source in text/plain part is this instead:"
>From line1
>From line2
>From line3 already marked
>From line4
".
Comment 4 Milan Crha 2007-05-23 07:10:12 UTC
Created attachment 88647 [details] [review]
proposed patch 1/2

for evolution;
Comment 5 Milan Crha 2007-05-23 07:13:19 UTC
Created attachment 88649 [details] [review]
proposed patch 2/2

for evolution-data-server;

I added support of encode/decode direction for CamelMimeFilterFrom and improved mbox support, so it works with "From " in text like described in http://qmail.org/qmail-manual-html/man5/mbox.html
Comment 6 Jeffrey Stedfast 2007-05-23 14:29:37 UTC
The qmail man page is wrong.


To quote JWZ:


      > You should assume that "From " lines are quoted, not mangled.
      > On systems that mangle, mail gets corrupted no matter what you do.
      > If you assume quoting is used, you might save mail sometimes.

I completely disagree.

I was describing what the historical truth of this file format is: the facts about what is out there in the world today. If you do quoting, not mangling, you are following different rules than have been followed for decades.

If, when I send a message, I type ``>From '' at the beginning of a line, and you treat it as quoting instead of mangling, you will alter the message. And you will alter it in a different way than all the other software out there which is following the ancient de-facto standard.

Whether you treat it as quoting or mangling, either method will cause certain messages to be changed: neither is foolproof, as both make the assumption that all software is following the same rules, whichever they may be.

I say, since neither method is foolproof, use the older and more prevalent rules, since that's what will get you maximal interoperability.

You may think that turning ``>From '' into ``From '' is somehow more harmless than turning ``From '' into ``>From '', but I don't see why.

For example, consider the case of a message with a clear-signed cryptographic signature (multipart/signed, say.) Any change to the body will cause the signature verification to fail. Now, suppose that the software doing the signing, being aware of the prevalence of From-mangling, were to do that mangling before signing. This would interoperate perfectly with software that did mangling in the traditional way (like, say... sendmail, or /bin/Mail).

But software that tried to ``improve'' matters by changing the rules -- by mangling ``>From '' to ``>>From '', or by mangling ``>From '', to ``From '' -- would cause the signature to no longer match.

It is in nobody's best interest for random pieces of software to try and ``improve'' on the BSD mbox format. It's a crummy format, but it is what it is. If you don't want to use it, don't -- use something else. But please, use something else that can be self-identified, that is programatically distinguishable from the historical format, so that software can know what it's parsing.

Don't make us need to guess even more than we do now. 
Comment 7 Jeffrey Stedfast 2007-05-23 14:31:33 UTC
fwiw, the logic in your patch is also broken.
Comment 8 Jeffrey Stedfast 2007-05-23 14:42:45 UTC
there are a couple possible ways to solve this problem:


1. make evolution use maildir instead of mbox (see the original suggestion, it was suggested because trying to make Evolution use From-quoting instead of the widely used and accepted From-munging is a broken idea)

2. in the composer (or mailer after the composer is done with it), scan for lines starting with "From " - if any exist, use Quoted-Printable encoding and transform "From " into "=46rom " (I believe the CamelMimeFilterCanon filter does this) before it gets appended to an mbox (Outbox or Drafts)
Comment 9 Jeffrey Stedfast 2007-05-23 14:51:57 UTC
*** Bug 440634 has been marked as a duplicate of this bug. ***
Comment 10 Matthew Barnes 2007-05-23 14:58:40 UTC
(In reply to comment #8)
Thanks for the detailed explanation, Jeff.  It sounds like your second suggestion about using CamelMimeFilterCanon would be a good temporary solution to this, while your first suggestion about using maildir instead of mbox is really the long-term goal to shoot for.
Comment 11 Jeffrey Stedfast 2007-05-23 15:11:02 UTC
yea, switching to maildir would be a bit of a pain at this point :(

I kinda regret pushing for mbox back when we made the decision, but oh well.

Comment 12 Milan Crha 2007-05-23 15:16:55 UTC
(In reply to comment #7)
> fwiw, the logic in your patch is also broken.
> 

Why do you think that?

My logic is very simple:
- make changes only when accessing data from/to mbox (when reading/writing on disk) and only after found proper message beginning
- when writing, add quote
- when reading, remove quote

This is broken only for already stored messages, and only there, where user already typed his own (one or more) quote(s) before From, in this particular case it could break a signature, but nowhere else.
Comment 13 Dan Winship 2007-05-28 13:19:13 UTC
another possibility is to have the mbox folder code record the locations of
the mangling ">"s in the summary, so that when it fetches the message out of
the mbox later, it knows which ">"s it should remove to recreate the original
message text
Comment 14 Jeffrey Stedfast 2007-05-28 13:23:03 UTC
yea, but that would be overly complicated and would make the summary less regeneratable (ok, that's not a word...)
Comment 15 Pascal Terjan 2007-05-29 20:42:42 UTC
*** Bug 441865 has been marked as a duplicate of this bug. ***
Comment 16 Milan Crha 2007-07-23 11:13:48 UTC
Jeff, what about add new option to mbox file/dir, called "qmail compliant store" (or any better text, which I know exists), which will turn on those replacements.
I know it's nothing extra, but it may be better than nothing. User will have an option to get rid of these unnecessary ">From "s in the text, when he/she will know it's safe. This option could be turned on on Evolution's Draft,Sent,Inbox,... local folders as default.

What do you think, could this work?
Comment 17 Jeffrey Stedfast 2007-07-23 16:13:48 UTC
not if the mbox already has mail in it, no.
Comment 18 Milan Crha 2009-06-17 17:05:06 UTC
All this seems to be a WontFix from the discussion above. There is a bug for a mialdir, I'm pretty sure, thus I tend to close this. Any objections? (Except of functionality, but that one cannot change with a broken design/rfc anyway.)
Comment 19 Milan Crha 2009-09-07 10:24:59 UTC
*** Bug 475359 has been marked as a duplicate of this bug. ***
Comment 20 Milan Crha 2009-09-07 10:25:41 UTC
*** Bug 401665 has been marked as a duplicate of this bug. ***
Comment 21 Milan Crha 2009-09-07 10:26:05 UTC
*** Bug 428728 has been marked as a duplicate of this bug. ***
Comment 22 Milan Crha 2009-09-07 10:26:27 UTC
*** Bug 586412 has been marked as a duplicate of this bug. ***
Comment 23 Eddy Petrişor 2009-09-07 23:32:11 UTC
Please, please, please start using maildir as a storage format for Evolution, instead of the broken mbox.
Comment 24 Milan Crha 2009-09-08 15:56:34 UTC
Created attachment 142705 [details] [review]
proposed evo patch ][

for evolution;

This is a change on a composer side, using quoted-printable whenever possible and escaping "\nFrom " properly, thus the message is not broke by the underlying store. Of course, moving to maildir is a long term fix, with other couple benefits, but that will take much longer (and other bug is for that too).
Comment 25 Dan Winship 2009-09-08 16:58:29 UTC
disclaimer: i don't use evo any more, but since I originally filed this bug, I'm going to comment anyway. :)

it would be extra-awesome if you could make the Outbox->CamelTransport codepath undo the qp encoding. IIRC you should be able to just always undo it, because the transport code will re-encode it if needed (eg, if it's non-ASCII and the SMTP server doesn't support 8BITMIME).

Also:

+text_requeires_quoted_printable (const gchar *text, gsize len)

typo. ("requires".) Also, I think there definitely used to be a method that did that somewhere in Camel, though maybe it's not there any more, or maybe it's not available in a form that useful from the composer.
Comment 26 Milan Crha 2009-10-15 18:46:26 UTC
Created commit 3766e19 in evo master (2.29.1+)

Thanks for your comments. The typo is fixed (that was easy) :) Otherwise I kept the patch as it is, because I'm not sure with that transport part, and the function from camel, I saw there couple filters using something similar, but didn't find any particular function.