GNOME Bugzilla – Bug 650645
Replies to messages sent from various mail agents miss line-breaks - must manually change format to "Normal" from "Preformatted", resulting in one big paragraph
Last modified: 2021-05-19 12:28:37 UTC
Created attachment 188207 [details] Bad message "Sent from my iPhone"(R) When I reply to some friends that have an iPhone, the quoted text has no line-breaks. Thus, the text goes beyond the right side of the window, and it's painful to answer. As workaround is simply to change the paragraph style from Preformatted to Normal, and line breaks appear. The way Evolution deals with those messages is weird: in the source of the message (see attachment), lines are correctly broken. But when viewing it, Evo seems to reconstitute complete paragraphs from those broken lines, which is good since it allows for a better use of screen space. But when replying, instead of using the original source with line breaks, Evo uses the version with no lines breaks, and decides that its form has to be respected, thus choosing the Preformatted paragraph style. In the source of the message, lines end with = [U000D], while in the messages Evo sends it's only [U000D]. I guess you know what it means. Have a look at the attached source. (This is related to bug 647947.)
Saving the message to an mbox seems to have changed it to a single full line, just like it appears when you reply to it. So that's an interesting information. The last line appears in the source view from Evo as: Ska=C3=AF jeux dizbfizkf zizi a dj=C3=A9bels n'ai j'ai je. Islckeb roc b=C3=A2=[U000D] clent cl le. Isl bel kel osi ne. Islxp kab. Ososb f jenxk!!!! Wiskxb=20[U000D]
Most iPhone messages I receive use quoted-printable encoding and have soft line breaks such that paragraphs are decoded as a single line of text [1]. Evolution is simply quoting the original decoded text verbatim. Unfortunately, automatically line wrapping the original text isn't always the right thing to do since the original text may be structured and purposefully have long lines, such as source code. [1] RFC 2045, Section 6.7: (Soft Line Breaks) The Quoted-Printable encoding REQUIRES that encoded lines be no more than 76 characters long. If longer lines are to be encoded with the Quoted-Printable encoding, "soft" line breaks must be used. An equal sign as the last character on a encoded line indicates such a non-significant ("soft") line break in the encoded text. ... This provides a mechanism with which long lines are encoded in such a way as to be restored by the user agent.
(In reply to comment #2) > Most iPhone messages I receive use quoted-printable encoding and have soft line > breaks such that paragraphs are decoded as a single line of text [1]. Is that the case with the message I pasted/attached? IIUC, the RFC says the line must end with an equal sign, and here there's none. So does that mean the iPhone is doing something wrong, i.e. it formats its messages in such a way that Evo doesn't consider it can be wrapped as needed? And that it cannot distinguish between lines that should be wrapped, and lines that shouldn't? If that's the case, that's really stupid - if the iPhone doesn't respect RFCs, what can we do? :-(
The attachment in comment #0 appears to be a decoded mbox file that you saved to disk, so I can't tell what encoding was used during transport. If you do View -> Message Source on one of your iPhone messages there should be a header named Content-Transfer-Encoding which will either be "7bit", "quoted-printable" or "base64". I did some Googling on the issue to see if other iPhone message recipients were having the same problem, and from the threads I saw I gathered that Apple probably started encoding messages that way to satisfy Exchange users who were complaining about seeing arbitrary line breaks, probably because Exchange or Outlook doesn't decode the message properly and of course Exchange users all use HTML mail. Sending paragraphs of text as a single line would be fine if the Content-Type were text/html. This is just my guess at what's going on. I suppose as a dirty hack we could try to detect iPhone messages by looking for a tell-tale header, and then apply line wrapping when quoting them in a reply. All of my iPhone messages have "Mime-Version: 1.0 (iPhone Mail 8H7)". But I hate these kinds of hacks. It's really Apple's fault.
Indeed: > Content-Transfer-Encoding: quoted-printable Of course, special-casing the iPhone would be kind of ugly, just because they don't respect RFCs... What's really weird is that messages sent from Apple Mail work OK, so it seems Apple didn't think it was worth applying the hack there. And yet, these messages are very similar, except Content-Transfer-Encoding: > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes > Content-Transfer-Encoding: 7bit > Mime-Version: 1.0 (Apple Message framework v936) > Subject: XXXXXXXXXX > Date: Sat, 14 May 2011 18:44:58 +0200 > X-Mailer: Apple Mail (2.936) > > > ccccccccskjfaefjlajljl fejlfekjgeafkuhdf e gdrgeje nnmkhjkdzd [U000D] > zzzzzzzzzzzzzzz fffffff fefelfkeztzorksf zdzaekfjkzfg ezfkrjgkr grkjgzm [U000D]
So what do you think? Detect and hack around iPhone messages or close this as NOTABUG and blame Apple?
Me? I'm not sure I'm the best person to decide, honestly. ;-) Personally, I can live with this small bug, but the question is, does that benefit to Evo enough to be worth a hack. I'll check what Apple Mail does with a friend, and it would be good to see what Thunderbird does too. But I guess it also depends on how easy/clean a hack would be: if it fits nicely in the code, that's less of an issue to add a hack. Or maybe Apple has something like an open bug reporting platform to complain? :-)
Based on the pasted Content-Type, Apple is following RFC 3676: http://tools.ietf.org/html/rfc3676 It would be good to have an exact message as an attachment for testing, not saved by evolution.
In all iPhone messages I've ever received, the Content-Type is text/plain with a "Charset" parameter. No "Format" or "DelSp" parameters, unless Camel is stipping them out.
(In reply to comment #8) > It would be good to have an exact message as an attachment for testing, not > saved by evolution. How can I do that? The examples above were copied/pasted from Evo's Ctrl+U dialog. I can ask my friend to send you an iPhone message, if that helps...
(In reply to comment #10) > How can I do that? File > Save as mbox (in 2.32 at least, don't have 3.0 on this machine).
(In reply to comment #11) > File > Save as mbox (in 2.32 at least, don't have 3.0 on this machine). Yeah, but you might have noticed the condition is "not saved by evolution."
(In reply to comment #9) > In all iPhone messages I've ever received, the Content-Type is text/plain with > a "Charset" parameter. No "Format" or "DelSp" parameters, unless Camel is > stipping them out. I noticed it is stripping them out, thus my requirement for saving from elsewhere, not from evolution. I would use another mail client, which shows message correctly, and hope that it'll not break the message and keep it in the original content without many inner modifications (it can add its own headers, for example, but may not strip parameters as evolution does).
The same message in Thunderbird looks like this: > Content-Type: text/plain; > charset=utf-8 > X-Mailer: iPhone Mail (8F190) > Content-Transfer-Encoding: quoted-printable > Mime-Version: 1.0 (iPhone Mail 8F190) > > Ska=C3=AF jeux dizbfizkf zizi a dj=C3=A9bels n'ai j'ai je. Islckeb roc b=C3=A2= > clent cl le. Isl bel kel osi ne. Islxp kab. Ososb f jenxk!!!! Wiskxb=20 Is the equal sign on the line the sign that it's marked as soft line break? In that case, TB would either have a different message source, or detects iPhone and fixes line breaks. At any rate, if I reduce the window's width, lines are broken as needed.
I'm also having problems with mails sent using the Zimbra 5.0 webmail client. Lines also end with =[U000D], and the header contains: > MIME-Version: 1.0 > Content-Type: text/plain; charset=utf-8 > Content-Transfer-Encoding: quoted-printable Plus, it happens relatively often with e-mails from random people on mailing lists. So I think it's worth a change in Evo - it's not only Apple's madness.
I also found out that messages sent from the Nabble mailing list "archive" suffer from his problem. There, lines end with U+000D, and the header contains: > MIME-Version: 1.0 > Content-Type: text/plain; charset="us-ascii" > Content-Transfer-Encoding: 7bit (so that's apparently a different situation)
(In reply to comment #16) > I also found out that messages sent from the Nabble mailing list "archive" > suffer from his problem. There, lines end with U+000D, and the header contains: > > MIME-Version: 1.0 > > Content-Type: text/plain; charset="us-ascii" > > Content-Transfer-Encoding: 7bit If the headers are from evolution, then it makes sense, the message parser removed the necessary information about encoding, that > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
(In reply to comment #17) > If the headers are from evolution, then it makes sense, the message parser > removed the necessary information about encoding, that > > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Yeah, sorry, it still comes from Evo. Do you need headers from Thunderbird?
(In reply to comment #18) > > > Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes > Yeah, sorry, it still comes from Evo. Do you need headers from Thunderbird? It may either prove that the original message has those extra parameters or TB converts the message "properly" before storing it locally (which it probably doesn't, because it would break digital signatures on the message text). If it's easy for you to get it, then yes, paste it here from TB, please.
(In reply to comment #19) > If it's easy for you to get it, then yes, paste it here from TB, please. Silly, I can't find that message again, and other messages I have sent from Nabble don't have "format=flowed; delsp=yes" in their header. Though, I've another case: a message (I think sent from MS Outlook) in base64 encoding: > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: base64 Lines are like this: > TWVyY2kgTWlsYW4sIG91aSBiaWVuIHPDu3IgaWwgZXN0IHV0aWxlIGRlIHJlbm9tbWVyIGxlcyB0 Evo's Ctrl+U dialog shows a U+000D character at the end of the line that Thunderbird doesn't show. Not sure who is right.
As for messages sent from Apple Mail, they have the same headers in Evo and Thunderbird. > Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes > Content-Transfer-Encoding: quoted-printable The only difference is that in Evo they have U+000D at the end of lines, and with Thunderbird this character doesn't appear and isn't saved to .eml file.
(In reply to comment #20) > Though, I've another case: a message (I think sent from MS Outlook) in base64 > encoding: > > Content-Type: text/plain; charset="utf-8" > > Content-Transfer-Encoding: base64 > > Lines are like this: > > TWVyY2kgTWlsYW4sIG91aSBiaWVuIHPDu3IgaWwgZXN0IHV0aWxlIGRlIHJlbm9tbWVyIGxlcyB0 > > Evo's Ctrl+U dialog shows a U+000D character at the end of the line that > Thunderbird doesn't show. Not sure who is right. Outlook may code line-breaks as \r\n, and the 0x0d is for that \r. You can see the same in the source view of a GPG signed message. As long as it's shown only in the source view of the message it is no big deal. (In reply to comment #21) > As for messages sent from Apple Mail, they have the same headers in Evo and > Thunderbird. > > Content-Type: text/plain; charset=ISO-8859-1; format=flowed; delsp=yes > > Content-Transfer-Encoding: quoted-printable > > The only difference is that in Evo they have U+000D at the end of lines, and > with Thunderbird this character doesn't appear and isn't saved to .eml file. Interesting, those extra parameters got dropped in my evo. I might do something differently. After all, it just means that evolution-data-server (the Camel part) doesn't understand the "format=flowed; delsp=yes" Content-Type header arguments. Could you attached such raw test message, please? (Per comment #8)
Created attachment 206411 [details] iPhone message saved from Thunderbird Here's one example. I removed the body of the message, but I took care to keep all control characters and headers. There was nothing at the end of a line other than "=".
Hmm, this one doesn't have the Content-Type with those extra parameters. With them, I suppose, the '=' at the end of the line works as a soft line break.
Created attachment 206418 [details] Apple Mail message saved from Thunderbird Oh, sorry, I got completely confused with all this mess. ;-) The previous attachment was an iPhone message (as stated in the header). This one is really a message sent from Apple Mail. The bug seems to be a little more complex than I thought: - the iPhone message doesn't appear with line breaks, and exhibits all the symptoms described here (i.e. very long lines when replying, Preformed style) - the Apple Mail message *does appear* with line breaks in Evo, but I think these line breaks correspond to what the source message says. So even if it's shown as Preformed when you reply to it, the length of the line is correct. I guess this means the bug is here, but by chance it doesn't hurt too much.
Created attachment 206419 [details] iPhone message saved from Thunderbird Replacing the first attachment to fix the description field, since we cannot fix it in the comment.
*** Bug 595339 has been marked as a duplicate of this bug. ***
*** Bug 692743 has been marked as a duplicate of this bug. ***
Aaaah, so that's what's been going on! Quite annoying indeed. For what it's worth, this bug is still happening with the new webkit-based composer in 3.16.5
What's supremely annoying about this is that when you receive a mail sent by Apple Mail, all the lines are in HTML but with the "preformatted text" style, so they don't wrap... but then if you try to change the style from "preformatted" to "normal" in the quoted text, it just removes all the line breaks and turns it into one single paragraph, like so: http://jeff.ecchi.ca/public/evolution-650645-apple-mail.webm
Nekohayo: Which exact Evolution version is that about?
(In reply to André Klapper from comment #31) > Which exact Evolution version is that about? This is Evolution 3.18.5.1 (hence why I bumped up the version field).
The message from comment #26 is a plain text message which has a very long line defined. It's slightly different from the video from the comment #30, which looks like there were several line breaks in the source message, but once the Preformat->Normal paragraph style had been chosen in the quoted part all the paragraphs were merged into one long. I only guess this from the video. Actual message would help, but if it's similar to the comment #26, then I'd tell it's a correct behaviour, due to missing new line characters in the source message. You can change the default of the quoted part paragraph format in Edit->Preferences->Composer Preferences->Replies and Forwards section->[x] Wrap quoted text in replies, available in 3.20.0. Otherwise, it's hidden in gsettings (bug #756883), with: $ gsettings get org.gnome.evolution.mail composer-wrap-quoted-text-in-replies You can also force wrapping manually by Format->Wrap Lines (Ctrl+K) in the composer, instead of changing paragraph style.
Created attachment 322666 [details] Example message with incorrect line breaks on reply I'm seeing this problem with all messages in a recent thread on desktop-devel-list. Replies which were already quoted in the original message are cut, i.e. the few last words of each quote lines are rejected to the next one, which is awful. See the attached message for example. Bug 762549 (which I recently filed) might be related.
I understood this bug not being about links, but about whole paragraphs under citations. Your example message break the URL in the citation, but changing the format of that broken line to Preformatted makes it one line, just as expected. I think the email belongs to the new bug you cited, rather than here.
Hm, yes, sorry, I got confused. Maybe I file too many bugs. :-)
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version, then please follow https://wiki.gnome.org/Community/GettingInTouch/BugReportingGuidelines and create a new bug report ticket at https://gitlab.gnome.org/GNOME/evolution/-/issues/ Thank you for your understanding and your help.