GNOME Bugzilla – Bug 518312
Attachments’ Content-Type header have filename last.
Last modified: 2008-06-03 03:42:52 UTC
First off, this is not a “bug” in EDS per sé, but it is an interoperability problem with servers that are broken and do not properly work with MIME headers. To work around broken servers which assume that the name parameter should come last when there is a file name present in the Content-Type header (and the server does not use the Content-Disposition header), the name parameter should come last. For example: Content-Disposition: attachment; filename=CheckpointTextPrintingProgramTrausch1.java Content-Transfer-Encoding: base64 Content-Type: text/x-java; name=CheckpointTextPrintingProgramTrausch1.java; charset=UTF-8 Should really be: Content-Disposition: attachment; filename=CheckpointTextPrintingProgramTrausch1.java Content-Transfer-Encoding: base64 Content-Type: text/x-java; charset=UTF-8; name=CheckpointTextPrintingProgramTrausch1.java In the case of this particular attachment, the message was processed improperly by the server and came back with: Content-Type: text/x-java; name="CheckpointTextPrintingProgramTrausch1.java; charset=UTF-8" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="CheckpointTextPrintingProgramTrausch1.java; charset=UTF-8" This is, in particular, to work around a bug in the NNTP gateway in the Jive Forums software. There may be other server-side implementations which make this dangerous assumption, though, as well, and it would be well to work around those, too. I have determined that the modification that appears to be made (and mind, I could be wrong on this) is to camel-mime-utils.c in the camel_header_param_list_format_append() function, which should, if passed multiple parameters to format: * Process every parameter in the list, and determine if the filename is in the list, * If the filename is in the list, move it to the end of the list, * Then process the list as it already does. Alternatively, it could loop through the list and if it finds the filename, cache that information, and then when it is finished looping through the list, put the filename at the end of the formatted output. What had led me to believe that this is where the fix needs to be is that I modified widgets/misc/e-attachment-bar.c in Evolution, function attach_to_multipart, and removed the * for the check for text/* and made it check instead for text/plain. This seems to skip the code here unless the Content-Type of the attachment is text/plain, which works for my very temporary needs, but is not a solution that is advisable for anyone else to use. I have filed this as LP #194642 in Ubuntu[1], as well. I did so while I was working on trying to figure out just what the issue was. There is a remote chance that I might be able to fix this one, but I am not holding my breath; I have to learn more about how camel_header_param_list_format_append() and the functions/data types it relies on work in general, because I am not what you would consider to be an experienced C programmer by any stretch, and have not used GLib before, really, either---therefore any fix I could provide would likely not meet any sort of quality criteria for inclusion in EDS anyway. https://bugs.edge.launchpad.net/ubuntu/+source/evolution-data-server/+bug/194642
Created attachment 106436 [details] [review] Work around broken MIME implementations by appending name field last on MIME headers This patch modifies camel_header_param_list_format_append() to defer processing of the name field for a MIME header until last, and moves the processing logic into _camel_header_param_format_append() so that there is no code duplication in this deferred processing and checking. This patch modifies camel/camel-mime-utils.h and camel-mime-utils.c to add the prototype for the new function _camel_header_param_format_append(), which starts with an underscore to indicate that the function is not intended to be publicly used, which seems to be the convention here.
Comments from original Ubuntu bug that I think may help on this (trimmed): > Also -- it might be a good idea to show the output of your change, and > state if you checked it against "normal" servers. Upstream will > certainly ask for this. Indeed. This is what the header now looks like when re-arranged (I am pasting these from a 40 MB message I just put together to showcase a bunch of different types of files. Content-Type: text/x-java; charset=UTF-8; name=WeeklyPay.java Content-Type: text/x-csharp; charset=UTF-8; name=fsw.cs Content-Type: application/x-ms-dos-executable; name=threading.exe Content-Type: image/png; name=mbt Content-Type: application/pgp-signature; name=signature.asc Content-Type: application/pgp-signature; name=UbuntuCodeofConduct-1.0.1.txt.asc Content-Type: audio/x-vorbis+ogg; name=TWiT0134.ogg I have been using this patch for several days now and everything appears to be working just fine; the broken server that I am working around now silently drops charset MIME header parameters, but no other server I have sent mail through has done that sort of thing, to my knowledge.
also this comment: It may also be prudent to reiterate that the changes made remain compliant with MIME Part One, RFC 2045, §5, ¶ 3, which states: "The Content-Type header field specifies the nature of the data in the body of an entity by giving media type and subtype identifiers, and by providing auxiliary information that may be required for certain media types. After the media type and subtype names, the remainder of the header field is simply a set of parameters, specified in an attribute=value notation. The ordering of parameters is not significant." Since the only change is the ordering to work around nasty parsers in existing software that breaks this portion of the standard, there _should_ be no additional side effects. To date, I have observed none.
Bug is still present in 2.22.0; modifying the bug to indicate this.
would be better to find the composer code that generates the params and make sure to set the filename/name params last.
Might I ask why? Since this is the one central location that does it, would it not be prudent to do so in all situations, and thus benefit all code that uses this routine?
Is there an answer to my previous query? Maybe I read the response to the patch wrong, so I will ask another (two): Is there a _more_ central place to fix this, that more code paths would cross? Or is the pushback on the patch a request to have it _not_ be centralized at all?
Created attachment 111345 [details] [review] the-hard-way.patch the "hard way" approach
Created attachment 111347 [details] [review] the-easy-way.patch an alternative approach which might also fix it, but in a much less obtrusive way.
hmmm, seems to have lost my comments when I added the first patch. judging from the message received back from the server (comment #1), it seems to me that the real problem is not param ordering at all, but rather that the server doesn't know how to parse non-quoted parameter values - you'll notice that it thinks that the following parameter was all part of the name value... likely because it kept scanning the line looking for an end-quote. When none appeared, it just gobbled the rest of the line and assumed it was part of the name value. I bet that if we force all param values to be quoted, then the server will be happy.
I'll build with the patch and see if that fixes it and report back. It's going to be a few days, I am getting ready to head out of town at the moment, but I'll get back on this one ASAP.
cool, no rush.
Jeffrey, The "easy way" patch appears to work to solve the problem; the server correctly handles the filename when it is quoted in double-quotes. However, if the filename contains a double quote character itself, things break again, because the server doesn't understand it. What goes out is: --=-ZoI5ZTGTfAMN6QLLa4nJ Content-Disposition: attachment; filename="test\"foo.java" Content-Transfer-Encoding: base64 Content-Type: text/x-java; name="test\"foo.java"; charset="UTF-8" What the server winds up doing, to it, though, is still not pretty: --JivePart=_36310a3.zeB8CwDOH5aMJzMl Content-Type: text/x-java; name="test\" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="test\" This seems to happen anyway with the vanilla Evolution and this server, as well, since MIME requires that filenames with the " character in them be quoted with " characters, and the new " be escaped with a backslash. So it is not a new problem (though I wonder if there is a legal way around it; is it legal to encode filenames using the quoted-printable format, as can be done for a Subject or From header?). Regardless, this patch would seem to fix the reported bug's real issue. Thanks!
heh, looks like the authors of this nntp server software need to learn to read :p it's technically not legal to use the rfc2047 encoding mechanism for header parameter values because rfc2047 explicitly forbids quoting those tokens, but according to rfc2045 (which defines parameter name/value pairs), chars like '=' must be quoted or encoded according to the rules of rfc2184 (superseded by rfc2231) my guess is that this news server will also die a horrible death when given an rfc2231-encoded name/value pair...
committed my fix
Yeah, I would wager that they only glanced at the RFCs. It seriously should not be hard for them to get "right", given that they have been around for forever. Anyway, thank you for the fix!