After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 518312 - Attachments’ Content-Type header have filename last.
Attachments’ Content-Type header have filename last.
Status: VERIFIED FIXED
Product: evolution-data-server
Classification: Platform
Component: Mailer
2.22.x (obsolete)
Other Linux
: Normal normal
: ---
Assigned To: evolution-mail-maintainers
Evolution QA team
composer
Depends on:
Blocks:
 
 
Reported: 2008-02-23 21:46 UTC by Michael Trausch
Modified: 2008-06-03 03:42 UTC
See Also:
GNOME target: ---
GNOME version: 2.21/2.22


Attachments
Work around broken MIME implementations by appending name field last on MIME headers (5.78 KB, patch)
2008-03-02 22:45 UTC, Michael Trausch
rejected Details | Review
the-hard-way.patch (5.65 KB, patch)
2008-05-22 15:09 UTC, Jeffrey Stedfast
rejected Details | Review
the-easy-way.patch (1002 bytes, patch)
2008-05-22 15:13 UTC, Jeffrey Stedfast
committed Details | Review

Description Michael Trausch 2008-02-23 21:46:09 UTC
First off, this is not a “bug” in EDS per sé, but it is an interoperability problem with servers that are broken and do not properly work with MIME headers.

To work around broken servers which assume that the name parameter should come last when there is a file name present in the Content-Type header (and the server does not use the Content-Disposition header), the name parameter should come last. For example:

Content-Disposition: attachment; filename=CheckpointTextPrintingProgramTrausch1.java
Content-Transfer-Encoding: base64
Content-Type: text/x-java; name=CheckpointTextPrintingProgramTrausch1.java; charset=UTF-8

Should really be:

Content-Disposition: attachment; filename=CheckpointTextPrintingProgramTrausch1.java
Content-Transfer-Encoding: base64
Content-Type: text/x-java; charset=UTF-8; name=CheckpointTextPrintingProgramTrausch1.java

In the case of this particular attachment, the message was processed improperly by the server and came back with:

Content-Type: text/x-java; name="CheckpointTextPrintingProgramTrausch1.java; charset=UTF-8"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="CheckpointTextPrintingProgramTrausch1.java; charset=UTF-8"

This is, in particular, to work around a bug in the NNTP gateway in the Jive Forums software. There may be other server-side implementations which make this dangerous assumption, though, as well, and it would be well to work around those, too.

I have determined that the modification that appears to be made (and mind, I could be wrong on this) is to camel-mime-utils.c in the camel_header_param_list_format_append() function, which should, if passed multiple parameters to format:

  * Process every parameter in the list, and determine if the filename is
    in the list,
  * If the filename is in the list, move it to the end of the list,
  * Then process the list as it already does.

Alternatively, it could loop through the list and if it finds the filename, cache that information, and then when it is finished looping through the list, put the filename at the end of the formatted output.

What had led me to believe that this is where the fix needs to be is that I modified widgets/misc/e-attachment-bar.c in Evolution, function attach_to_multipart, and removed the * for the check for text/* and made it check instead for text/plain.  This seems to skip the code here unless the Content-Type of the attachment is text/plain, which works for my very temporary needs, but is not a solution that is advisable for anyone else to use.

I have filed this as LP #194642 in Ubuntu[1], as well.  I did so while I was working on trying to figure out just what the issue was.

There is a remote chance that I might be able to fix this one, but I am not holding my breath; I have to learn more about how camel_header_param_list_format_append() and the functions/data types it relies on work in general, because I am not what you would consider to be an experienced C programmer by any stretch, and have not used GLib before, really, either---therefore any fix I could provide would likely not meet any sort of quality criteria for inclusion in EDS anyway.

https://bugs.edge.launchpad.net/ubuntu/+source/evolution-data-server/+bug/194642
Comment 1 Michael Trausch 2008-03-02 22:45:42 UTC
Created attachment 106436 [details] [review]
Work around broken MIME implementations by appending name field last on MIME headers

This patch modifies camel_header_param_list_format_append() to defer processing of the name field for a MIME header until last, and moves the processing logic into _camel_header_param_format_append() so that there is no code duplication in this deferred processing and checking.

This patch modifies camel/camel-mime-utils.h and camel-mime-utils.c to add the prototype for the new function _camel_header_param_format_append(), which starts with an underscore to indicate that the function is not intended to be publicly used, which seems to be the convention here.
Comment 2 C de-Avillez 2008-03-09 18:14:57 UTC
Comments from original Ubuntu bug that I think may help on this (trimmed):

> Also -- it might be a good idea to show the output of your change, and
> state if you checked it against "normal" servers. Upstream will
> certainly ask for this.

Indeed. This is what the header now looks like when re-arranged (I am
pasting these from a 40 MB message I just put together to showcase a
bunch of different types of files.

Content-Type: text/x-java; charset=UTF-8; name=WeeklyPay.java
Content-Type: text/x-csharp; charset=UTF-8; name=fsw.cs
Content-Type: application/x-ms-dos-executable; name=threading.exe
Content-Type: image/png; name=mbt
Content-Type: application/pgp-signature; name=signature.asc
Content-Type: application/pgp-signature; name=UbuntuCodeofConduct-1.0.1.txt.asc
Content-Type: audio/x-vorbis+ogg; name=TWiT0134.ogg

I have been using this patch for several days now and everything appears
to be working just fine; the broken server that I am working around now
silently drops charset MIME header parameters, but no other server I
have sent mail through has done that sort of thing, to my knowledge.
Comment 3 C de-Avillez 2008-03-09 18:15:39 UTC
also this comment:



It may also be prudent to reiterate that the changes made remain compliant with MIME Part One, RFC 2045, §5, ¶ 3, which states:

"The Content-Type header field specifies the nature of the data in the body of an entity by giving media type and subtype identifiers, and by providing auxiliary information that may be required for certain media types. After the media type and subtype names, the remainder of the header field is simply a set of parameters, specified in an attribute=value notation. The ordering of parameters is not significant."

Since the only change is the ordering to work around nasty parsers in existing software that breaks this portion of the standard, there _should_ be no additional side effects. To date, I have observed none.
Comment 4 Michael Trausch 2008-03-10 19:40:27 UTC
Bug is still present in 2.22.0; modifying the bug to indicate this.
Comment 5 Jeffrey Stedfast 2008-03-10 22:42:19 UTC
would be better to find the composer code that generates the params and make sure to set the filename/name params last.
Comment 6 Michael Trausch 2008-03-11 02:33:20 UTC
Might I ask why?  Since this is the one central location that does it, would it not be prudent to do so in all situations, and thus benefit all code that uses this routine?
Comment 7 Michael Trausch 2008-03-30 04:51:16 UTC
Is there an answer to my previous query?

Maybe I read the response to the patch wrong, so I will ask another (two):  Is there a _more_ central place to fix this, that more code paths would cross?  Or is the pushback on the patch a request to have it _not_ be centralized at all?
Comment 8 Jeffrey Stedfast 2008-05-22 15:09:04 UTC
Created attachment 111345 [details] [review]
the-hard-way.patch

the "hard way" approach
Comment 9 Jeffrey Stedfast 2008-05-22 15:13:09 UTC
Created attachment 111347 [details] [review]
the-easy-way.patch

an alternative approach which might also fix it, but in a much less obtrusive way.
Comment 10 Jeffrey Stedfast 2008-05-22 15:29:19 UTC
hmmm, seems to have lost my comments when I added the first patch.

judging from the message received back from the server (comment #1), it seems to me that the real problem is not param ordering at all, but rather that the server doesn't know how to parse non-quoted parameter values - you'll notice that it thinks that the following parameter was all part of the name value... likely because it kept scanning the line looking for an end-quote. When none appeared, it just gobbled the rest of the line and assumed it was part of the name value.

I bet that if we force all param values to be quoted, then the server will be happy.
Comment 11 Michael Trausch 2008-05-23 14:16:39 UTC
I'll build with the patch and see if that fixes it and report back.  It's going to be a few days, I am getting ready to head out of town at the moment, but I'll get back on this one ASAP.
Comment 12 Jeffrey Stedfast 2008-05-23 16:23:48 UTC
cool, no rush.
Comment 13 Michael Trausch 2008-05-27 21:02:08 UTC
Jeffrey,

The "easy way" patch appears to work to solve the problem; the server correctly handles the filename when it is quoted in double-quotes.  However, if the filename contains a double quote character itself, things break again, because the server doesn't understand it.  What goes out is:

--=-ZoI5ZTGTfAMN6QLLa4nJ
Content-Disposition: attachment; filename="test\"foo.java"
Content-Transfer-Encoding: base64
Content-Type: text/x-java; name="test\"foo.java"; charset="UTF-8"

What the server winds up doing, to it, though, is still not pretty:

--JivePart=_36310a3.zeB8CwDOH5aMJzMl
Content-Type: text/x-java; name="test\"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="test\"

This seems to happen anyway with the vanilla Evolution and this server, as well, since MIME requires that filenames with the " character in them be quoted with " characters, and the new " be escaped with a backslash.  So it is not a new problem (though I wonder if there is a legal way around it; is it legal to encode filenames using the quoted-printable format, as can be done for a Subject or From header?).

Regardless, this patch would seem to fix the reported bug's real issue.  Thanks!
Comment 14 Jeffrey Stedfast 2008-05-27 21:23:15 UTC
heh, looks like the authors of this nntp server software need to learn to read :p

it's technically not legal to use the rfc2047 encoding mechanism for header parameter values because rfc2047 explicitly forbids quoting those tokens, but according to rfc2045 (which defines parameter name/value pairs), chars like '=' must be quoted or encoded according to the rules of rfc2184 (superseded by rfc2231)

my guess is that this news server will also die a horrible death when given an rfc2231-encoded name/value pair...
Comment 15 Jeffrey Stedfast 2008-05-27 21:26:35 UTC
committed my fix
Comment 16 Michael Trausch 2008-05-30 16:40:16 UTC
Yeah, I would wager that they only glanced at the RFCs.  It seriously should not be hard for them to get "right", given that they have been around for forever.

Anyway, thank you for the fix!