looks dupe of bug 490237 , see bug 582991 also 

hm, i don't think so. In my case, it's not a matter of using the wrong SMTP server from all those configured. It's about talking to it properly, i.e. don't pretend we're someone we're not.

*** Bug 595048 has been marked as a duplicate of this bug. ***

No feedback from anyone? /me sad

In camel/providers/smtp/camel-smtp-transport.c, smtp_send_to() creates the FROM for the *envelope* from the 'from' parameter it received from camel_transport_send_to(), so it's coming from evolution itself, i.e. from the mail headers as shown here:

evolution-2.28.1/mail/mail-ops.c:

        /* Check for email sending */
        from = (CamelAddress *) camel_internet_address_new ();
        resent_from = camel_medium_get_header (CAMEL_MEDIUM (message), "Resent-From");
        if (resent_from) {
                camel_address_decode (from, resent_from);
        } else {
                iaddr = camel_mime_message_get_from (message);
                camel_address_copy (from, CAMEL_ADDRESS (iaddr));
        }
        ...
          camel_transport_send_to(xport, message, from, recipients, ex);


This is just called spoofing. This has nothing to do with multiple accounts. It's just that envelope spoofing won't work with servers blocking that (internal emails coming from the outside).

Mutt is wise to always use whoami@fqdn for the FROM in the envelope, while leaving the From in the headers untouched.

Now, i'm not sure if the bug is in evolution-data-server or in evolution, or both.

This is a bug in Evolution itself, it needs to provide Camel with the proper email address of the sender. I guess it probably shouldn't be getting it from the message headers, but rather from the account info.

Should be simple for someone to fix.

Created attachment 148917 [details] [review]
fix v0

I gave it a try and produced a patch that i've been using successfully for ~2 weeks now.
I don't consider it clean, but here is it anyway.

Does the domain name in the "MAIL FROM" address have to be fully-qualified, or could we get away with whatever gethostname() returns?

Here is what the RFC says about it:


         The first step in the procedure is the MAIL command.  The
         <reverse-path> contains the source mailbox.

            MAIL <SP> FROM:<reverse-path> <CRLF>

         This command tells the SMTP-receiver that a new mail
         transaction is starting and to reset all its state tables and
         buffers, including any recipients or mail data.  It gives the
         reverse-path which can be used to report errors.  If accepted,
         the receiver-SMTP returns a 250 OK reply.

         The <reverse-path> can contain more than just a mailbox.  The
         <reverse-path> is a reverse source routing list of hosts and
         source mailbox.  The first host in the <reverse-path> should be
         the host sending this command.

My understanding of this is that it says the 1st reverse-path should be the source mailbox + local machine which has to be routed, so if the remote speaker is not in the same domain, errors will never reach the source unless a fully-qualified reverse-path was used.

That makes sense.  Might be nice for Camel to make the FQDN available on it's own instead of having to dig it out of a Message-ID string, but I think your approach is fine for now.

I asked a GIO maintainer if he would consider adding some kind of "give me my FQDN" function to the library and his answer was that strictly speaking we should be extracting it from a connected socket.  Since this logic runs before a connection is ever made, perhaps it should be moved into Camel such that camel_transport_send_to() automatically replaces a NULL "from" address with "whoami@fqdn" (or I guess that would be up to each transport provider to do).

so we can commit the current patch here?

Because this is such a critical section of code I'm going to hold off on committing until 2.31.  If the patch fares well in 2.31 we can look at backporting it to a 2.30 update.

any update?

I'm now running 2.30.1.2 and i'm still suffering from this (i can't use evolution at/for work).

My patch no longer applies due to the refactoring that happened since i wrote it.
I'm not sure how to do it now that mail_config_get_default_account() is gone.

Thanks for the reminder, I'll take a look at this again.

The function got moved and renamed to e_get_default_account() in e-util/e-account-utils.h.

Patch committed to master in:
http://git.gnome.org/browse/evolution/commit/?id=a52f00a9c9c00ecf06ac7179742ceb15e9971c5d

Reopening, I reverted this patch in commit b3f8d47 because I cannot send message through my IMAP account with an error:
> Error while Sending message.
> MAIL FROM command failed: <mcrha@localhost>...
> Real domain name required for sender address

hm, in my case, it's always @my_fqdn, never @localhost.
Do you mean that all your message-ids are always @localhost too?

Mine are.  Turns out I have the same problem as Milan.

e.g.  Typical sent message:

    From: Matthew Barnes <mbarnes@redhat.com>
    ...
    Message-ID: <1275276671.26525.12.camel@localhost.localdomain>

In my situation I'm typically on my home network and accessing Red Hat's SMTP server remotely, and my home network has no registered domain name.  Pretty common scenario.

d'oh! this is bad.

maybe it should default to camel_mime_message_get_from(message) when hostname
(as extracted from camel_header_msgid_generate()) is found to be
"localhost.localdomain" or "localhost".

Might not be that easy, I'm afraid, as you can have your domain name set only locally, or even the real domain name, but the remote server may reject your request because it expects only a domain name same as your email address, to not allow you using the SMTP server for sending addresses from different domains.

note that this is exactly the reason i filed the bug initially.
My SMTP servers at work (a very big company) reject all my emails sent from outside because evolution pretends i'm from within, while I'm not. Hence the patch tries to stop pretending that (stop spoofing) and really say (in the SMTP envelope only) where i'm really writing from.

imho, localhost(.localdomain) is a broken setup for SMTP, even if it's unfortunately common.

what i propose looks like this:

msgid = camel_header_msgid_generate(); /* => xxx@fqdn or xxx@localhost.localdomain */
if (msgid matches '@localhost.localdomain') {
  addr = camel_mime_message_get_from(message);  /* as before */
  from = ...(addr)
}
else {
  account = e_get_default_account(); /* => username */
  from = compose_email_address(account, msg_id); /* => username@fqdn */
}

remember it's the "From" for the SMTP transaction, not the "From:" for the headers, which still comes from the profile.

if it's not acceptable, like if you want the default to stay the same as the email in the profile (even if it's not true and it's spoofed), the only option to solve my problem would be to allow me to specify a non-default value in the profile(s).. but i can't propose a patch for that, i have absolutely no idea how to touch to the profiles.

*** Bug 621585 has been marked as a duplicate of this bug. ***

Created attachment 217553 [details] [review]
fix v1

Here is the patch updated to fit 3.5.3.1 (after the EAccount transition).
It addresses the localhost.localdomain/localhost issue mentioned before by not doing anything when it happens (i.e. use the From).

Comment 21 sounds like a reasonable policy.

Only change I would make to the policy is: earlier on in that function we look for an "X-Evolution-Identity" header and extract some account details based on that header value if can.  We should also try to extract an email address based on that header value, and otherwise fall back to the default mail identity.

Also, this logic is getting complicated enough that I'd like it split off into a separate mail_session_get_sender_address() function, which returns a newly-allocated string containing the email address to use for the MAIL FROM command.

Created attachment 220279 [details] [review]
fix v2

Re-posting the patch with the tiny leaks (spotted by mcrha) fixed.

@mbarnes: about moving that to a separate mail_session_get_sender_address() function, I read the surrounding code carefully and I'm not really sure how to do it best.

I see 4 locations where it could be useful:
1/ where i put it in my patch (that's where it is useful for my use case)
2/ in the (similar) "resent" branch of the enclosing if/else.
3/ & 4/ as you mentioned, where the source is derived from the X-Evolution-Identity header, either early or via the extension.

But those 4 cases seem to require different parameters, the raw header or msg-id, the source or extension. ...
I'm not really sure how to do that properly.

Please advise.

@mbarnes: would you re-land my last patch if my logic was put behind a g_settings boolean, disabled by defaut?
if so, how should this key be named?

Matthew: Could you answer comment 26, please?

Review of attachment 220279 [details] [review]:

With respect of a GSettings option, let's add it into
   evo/data/org.gnome.evolution.mail.gschema.xml.in
as a boolean key 'send-with-hostname-from' with default to 'false'. Then, a helper function to get a hostname can create a new GSettings instance, read from this key, and if it's set to FALSE, then simply return NULL.

::: libemail-engine/e-mail-session-utils.c
@@ +773,3 @@
+		const gchar *extension_name;
+		gchar *who;
+		gchar *res;

define variables only there where needed

@@ +776,3 @@
+
+		/* Don't get the 'from' from the headers.
+		 * See https://bugzilla.gnome.org/show_bug.cgi?id=585577

we usually do not cite bug links/numbers in the code, the most for (temporary) workarounds.

@@ +782,3 @@
+		 * fallback to using the 'from'.
+		*/
+		hostname = strchr ((fake_msgid = camel_header_msgid_generate ()), '@');

while I'm against code duplication, the code behind this function is not that complicated, and as a side effect increases its internal counter, thus what about to extract relevant parts of that function into a helper function into this file, which will simply return either host name to be used or NULL?

@@ +790,3 @@
+			camel_address_copy (from, CAMEL_ADDRESS (addr));
+		}
+		else {

these two lines should be one line:
    } else {

@@ +805,3 @@
+			        *p = '\0';
+			}
+			res = g_strconcat (who, "@", hostname, NULL);

this one is still leaking

Created attachment 240800 [details] [review]
fix v3

Here is the patch updated, hopefully taking into account your previous review.

Review of attachment 240800 [details] [review]:

Thanks for the update. The patch looks good, I have only few really minor concerns about it, as indicated below.

::: libemail-engine/e-mail-session-utils.c
@@ +825,3 @@
+			return NULL;
+	}
+	return g_strdup (cached_hostname);

does it make sense to strdup here? It does not, from my point of view.

@@ +831,3 @@
+e_mail_session_set_smtp_return_path (CamelAddress *from,
+				     EMailSession *session,
+				     CamelMimeMessage *message)

what about naming the function:
e_mail_session_set_from_address (EMailSession *session,
	CamelMimeMessage *message,
	CamelAddress *from)

@@ +870,3 @@
+		        gchar *p;
+		        p = strchr (who, '@');
+		        *p = '\0';

just to be on a safe side, check if p != NULL

Created attachment 240943 [details] [review]
fix v4

I initially named the function with smtp_return_path to match the wording of the RFC snippet I posted above ("FROM <return-path>" in the SMTP transaction, vs "From:" header in the message). Anyway, I don't mind either way.

Here is the patch updated once again.

This approach is not correct. You must not use the local hostname in the SMTP 'MAIL FROM', otherwise known as the reverse-path. The address of the account was the correct thing to put there.

If you break your email by putting a bogus reverse-path on it, you'll break the reliability of the whole system because you'll never receive bounces when your messages don't get through. And in fact some recipients will notice that brokenness "up front" and refuse to accept your messages at all. Not that you'll notice...

The real problem here seems to be that either the reporter's mail server, or the reporter's installation of Evolution, is misconfigured. Either he's within the company "trusted" network and mail sent with SMTP should be accepted as being from within the company. Or he's *outside* the company network, and he ought to have authenticated before attempting to send the email.

Either way, this patch isn't a fix. It isn't even a viable workaround. Please do *not* apply this.

wrong, I beg to differ. It has nothing to do with authentication.

My patch introduces a feature to set the reverse path to the true (as in not-spoofed) and routable mailbox (assuming the local machine is properly configured), exactly what the venerable Mutt and its predecessors have been doing since.. well, forever.

I understand that there are many cases when the 2nd criteria is difficult to meet (NAT..), hence the feature being disabled by default. But I assume those users also have difficulties with Mutt (their getaddrinfo is most probably failing or bogus)

If the local machine is properly configured as a 'traditional' unix host would have been, with an MTA of its own, then you probably don't want to use SMTP at all. Just feed the message to /usr/sbin/sendmail instead. That's what mutt does by default. And Evolution does have a sendmail transport which would do just that, although I'm not sure how to make it get *used* for a given mail account.

However, it is the *wrong* thing to do in almost all circumstances. Yes, it's true that old terminal-based MUAs designed for traditional UNIX environments will do it by default. That doesn't make it right for Evolution.

The reverse-path used in SMTP is the address to which bounces will be sent if your outbound mail cannot be delivered. And if it's not a valid, routable mailbox, then many recipients may refuse to accept the message you're sending.

In all normal circumstances it should be the same as the From: header (or Resent-From: in some cases) on the message you are sending.

Your problem here really is a misconfiguration by your company mail servers. They are broken, because they're administered by someone who wrongly thinks that they'll never see a reverse-path of 'first.last@example.com' on a mail which is being received from outside the company. That's obviously bogus if you know the first thing about email. Just set up an outside email address (@linkedin.com or @alumni.youruniversity.edu or a vanity domain, or even certain restricted mailing/distribution lists) to point at someone's @example.com mailbox, and you'll see it happening.

The company email is wrongly and stupidly configured. And do they really not provide you with any way to send authenticated email through their servers while you're outside the company? No authenticated SMTP? No VPN? No ActiveSync? I'm not sure I believe it.

If you *really* have to work around this utter brokenness on the part of your idiotic employer, then don't infer the reverse-path automatically from whatever hostname happens to have been set by DHCP today. Use a configuration option with a hard-coded override for the reverse-path address, instead.

But seriously, since you're sending this through your *own* MTA and you're insisting that of course this is a valid thing to do because you do *have* your own MTA and it's set up to receive messages to the address you're using... why in $DEITY's name don't you just set the reverse-path as you with *in* the MTA instead, rather than hacking Evolution to do it?

To clarify: I object strongly to a boolean option which would use the local username at the local hostname as the SMTP reverse-path. That is almost never the right thing to do.

However, I'd tolerate a string option which is used to override the SMTP reverse-path for a given account with a specific user-provided string. Although it's rare that this would be useful, at least it's not actively encouraging users to do horribly broken things.

The *only* time the boolean option would be sane is when the hostname is actually constant anyway, because you're running on a host with a public IP address and a *constant* public DNS hostname (even if it's dyndns). So a constant string should be perfectly sufficient to work around the misconfiguration of fta's corporate email servers.

(In reply to comment #35)
> However, I'd tolerate a string option which is used to override the SMTP
> reverse-path for a given account with a specific user-provided string.

I consider the part "for a given account" crucial, thus the string property should not be just an email address, but an address which will replace user-configured account's email into a new value for the FROM address.
I think a string value of format
   <old-email>:<new-email>
and replacing only if "<old-email>" matches the currently used FROM and the "<new-email>" is not empty (and possibly some email format checking, like at least the '@' is included).

An example value would be:
   me@company.com:me@home.com
while my Account email address configured in evolution will be me@company.com.

David just notified my on IRC that my above comment #36 is not correct. I added it due to my misunderstanding of David's proposal. After a brief clarification on IRC the setting should be added into CamelSmtpSettings, and will be used in
   eds/camel/providers/smtp
only. This will be accessible in a corresponding .source file in
   ~/.config/evolution/sources

In fact it might be nice to set and use that setting in *all* cases, rather than grubbing around in the cosmetic Resent-From: and From: headers for it.

Closing as WONTFIX since the patch broke Evolution for both Milan and I, and David gave a pretty solid argument against it.

@mbarnes: I don't see how my last patch could break evolution for anyone as it's only a gsetting pref, disabled by default, and it's not even exposed in the UI.

I initially wanted to work on yet-another-version of a fix/workaround implementing David's proposal in comment #35, but soon after, comment #37 proposed yet another way to do it, and it exceeds my current knowledge of Camel & Evo's internals.

I don't think comment #37 was "another way to do it". It was more of a technical clarification of *how* one would implement what was suggested in comment #35.

If you're going to do an option that allows you to specify the SMTP reverse-path to use for a given account's SMTP sessions, it is the file in .config/evolution/sources that would contain it.