Hi, is there any update on the implementation of this feature?

I found today that Geary won't log in a Google account without enabling "Less secure apps" access while Evolution with an account configured through GOA works fine (see [1]), and I was wondering on whether adding support for GOA in Geary would help fix that situation (besides improving its integration with GNOME)

Comments?

[1] https://mail.gnome.org/archives/geary-list/2015-March/msg00025.html

No one is taking this on at the moment, to my knowledge.  You might look at some of the discussion on Ubuntu Online Accounts (UOA) for ideas of how this might work: bug #714877

Thanks Jim for your reply and for confirming this, that's very helpful info.

What is still not entirely clear to me at the moment, and sorry for the small detour, is whether it would possible to fix the problem I mentioned before[*] by fixing the  imple (non-GOA based) authentication mechanism in Geary, or if makes no sense at all (perhaps adding GOA support is the only good way to go?)

If you could comment on that, I certainly would appreciate it a lot too. GOA support would be a wonderful thing to have, but for now I'm trying to figure out what all the options are to fix the situation with the "Less secure apps" thing, since that's kind of the showstopper for me at the moment.

Thanks in advance!

[*] "Geary won't log in a Google account without enabling "Less secure apps" access"

You are correct, simply adding GOA support doesn't solve every issue.  What it does is solve the problem of generating and managing an OAuth2 token.  Geary will still need to use an alternate authentication scheme to login.

However, that's ticketed as well (bug #713860).  This ticket relies on that one (but not vice-versa, hence their separation).

Also see bug #713745 and bug #714877.

(In reply to Jim Nelson from comment #4)
> You are correct, simply adding GOA support doesn't solve every issue.  What
> it does is solve the problem of generating and managing an OAuth2 token. 
> Geary will still need to use an alternate authentication scheme to login.
>
I understand the mechanism that needs to be implemented in Geary is XOAuth2[1] then, isn't it? At least that's what I understand from this blog post:

http://googleappsdeveloper.blogspot.co.uk/2014/10/updates-on-authentication-for-gmail.html

I quickly grepped through Geary's code and I saw there's one abstract class Geary.Smtp.Authenticator with two direct subclasses: Geary.Smtp.PlainAuthenticator and Geary.Smtp.LoginAuthenticator.

Would this new XOAuth2 mechanism need to be implemented in a similar way? I guess that would probably not fit as a subclass of that one because OAUTH is not a SASL-based authentication mechanism (afaics), but still that would the place to look this new mechanism was to be implemented in Geary, right?

Sorry if it's a dumb question, my knowledge of email and auth mechanisms is very limited. I only know a bit how OAuth works because I had to implement it for frogr (although that was OAuth1.0), so there's a chance this is all nonsense :)

Just trying to figure out all the moving pieces before getting into it deeper.

> However, that's ticketed as well (bug #713860).  This ticket relies on that
> one (but not vice-versa, hence their separation).
> 
I see, thanks for the pointer. Btw, shouldn't we mark this bug as "Depending on" that one then? That way the relation would be clearer

> Also see bug #713745 and bug #714877.

Yes, I already saw those ones, but thanks for referencing them anyway.

[1] https://developers.google.com/gmail/xoauth2_protocol

> I understand the mechanism that needs to be implemented in Geary is
> XOAuth2[1] then, isn't it?

I don't know all the ins-and-outs of OAuth2, but I believe all that's required is for Geary to implement it as an alternate IMAP/SMTP login mechanism.  In other words, with GOA (or UOA), Geary doesn't have to worry about an out-of-band scheme for generating the OAuth2 token, it simply uses the one handed to it by GOA/UOA.  Then, when logging in to IMAP or SMTP, it negotiates the authentication using that token in some prescribed way.

> I quickly grepped through Geary's code and I saw there's one abstract class
> Geary.Smtp.Authenticator with two direct subclasses:
> Geary.Smtp.PlainAuthenticator and Geary.Smtp.LoginAuthenticator.
> 
> Would this new XOAuth2 mechanism need to be implemented in a similar way?

For SMTP, yes.  My presumption at this moment is that OAuth2 *could* be implemented by extending the abstract base class.

> I guess that would probably not fit as a subclass of that one because OAUTH is
> not a SASL-based authentication mechanism (afaics)

Actually, if you look at your link, it says it's a SASL specification.  The examples it gives look like SASL as well.  That's a Gmail document, obviously, but I believe other services will follow Google's lead.

Note that there's no similar abstract base class in the IMAP stack because at this moment we only support one authentication scheme: LOGIN.  I would want to refactor the IMAP code to use something similar (or identical) to the SMTP code, i.e. Geary.Imap.Authenticator, Geary.Imap.LoginAuthenticator, and Geary.Imap.OAuth2Authenticator.

If the code was *exactly* identical, we could imagine refactoring the original SMTP classes out into a separate unit, i.e. Geary.Sasl.Authenticator.  But that could come later, as there's a lot of work already just to make everything connect properly.

> I see, thanks for the pointer. Btw, shouldn't we mark this bug as "Depending
> on" that one then?

Yeah, I'll go ahead and do that.

> > I see, thanks for the pointer. Btw, shouldn't we mark this bug as "Depending
> > on" that one then?
> 
> Yeah, I'll go ahead and do that.

Actually, I'm not going to do that.  The other ticket is a catch-all ticket we crafted a long time ago as a placeholder for a body of work we knew would need to do at some point.  It was written at a time when we didn't know what schemes would need to be supported and which we could effectively ignore.  There's a twin ticket for the SMTP work: bug #713859.  I've closed both as too broad.

I've created a single ticket for adding OAuth2 support: bug #746705.  I've made this ticket dependent on that one.

(In reply to Jim Nelson from comment #6)
> > I understand the mechanism that needs to be implemented in Geary is
> > XOAuth2[1] then, isn't it?
> 
> I don't know all the ins-and-outs of OAuth2, but I believe all that's
> required is for Geary to implement it as an alternate IMAP/SMTP login
> mechanism.  In other words, with GOA (or UOA), Geary doesn't have to worry
> about an out-of-band scheme for generating the OAuth2 token, it simply uses
> the one handed to it by GOA/UOA.  Then, when logging in to IMAP or SMTP, it
> negotiates the authentication using that token in some prescribed way.
> 
> > I quickly grepped through Geary's code and I saw there's one abstract class
> > Geary.Smtp.Authenticator with two direct subclasses:
> > Geary.Smtp.PlainAuthenticator and Geary.Smtp.LoginAuthenticator.
> > 
> > Would this new XOAuth2 mechanism need to be implemented in a similar way?
> 
> For SMTP, yes.  My presumption at this moment is that OAuth2 *could* be
> implemented by extending the abstract base class.
> 
> > I guess that would probably not fit as a subclass of that one because OAUTH is
> > not a SASL-based authentication mechanism (afaics)
> 
> Actually, if you look at your link, it says it's a SASL specification.  The
> examples it gives look like SASL as well.  That's a Gmail document,
> obviously, but I believe other services will follow Google's lead.
> 
Oops! Right, sorry about that. Don't know what I was thinking of.

> Note that there's no similar abstract base class in the IMAP stack because
> at this moment we only support one authentication scheme: LOGIN.  I would
> want to refactor the IMAP code to use something similar (or identical) to
> the SMTP code, i.e. Geary.Imap.Authenticator, Geary.Imap.LoginAuthenticator,
> and Geary.Imap.OAuth2Authenticator.
> 
Understood. Thanks for clarifiying, though.

> If the code was *exactly* identical, we could imagine refactoring the
> original SMTP classes out into a separate unit, i.e.
> Geary.Sasl.Authenticator.  But that could come later, as there's a lot of
> work already just to make everything connect properly.
> 
Yes, I also think it's better to start by having something that works, then move onto a refactor like that one, if it ever makes sense.

(In reply to Jim Nelson from comment #7)
> [...]
> Actually, I'm not going to do that.  The other ticket is a catch-all ticket
> we crafted a long time ago as a placeholder for a body of work we knew would
> need to do at some point.  It was written at a time when we didn't know what
> schemes would need to be supported and which we could effectively ignore. 
> There's a twin ticket for the SMTP work: bug #713859.  I've closed both as
> too broad.
> 
> I've created a single ticket for adding OAuth2 support: bug #746705.  I've
> made this ticket dependent on that one.

Great, I think everything is clearer now and I think I have all the information I need to figure out how to move forward. Expect more feedback from my side in a few days.

Thanks!

One thing, though. GOA also offers simple password based accounts for generic (ie. not Google or Outlook.com) IMAP/SMTP email accounts. To find out what kind of authentication a GoaObject needs, you can look at the oauth-based, oauth2-based and password-based properties, or their corresponding getters. See: https://developer.gnome.org/goa/unstable/GoaObject.html#GoaObject--oauth-based

For more information about the GoaPasswordBased interface, see: https://developer.gnome.org/goa/unstable/GoaPasswordBased.html

So, if someone wants to start with the GOA integration before the XOAUTH2 SASL mechanism is implemented in Geary, then they can do so by filtering out the non-password-based accounts.

I've got a WIP branch up on https://github.com/tshikaboom/geary/tree/wip-goa.

This enumerates existing accounts and adds existing password-based email accounts to Geary. I've also tried to document some code here and there, so all that would make sense.

For some mysterious reason, it crashes when I remove a second GOA account somewhere in the sidebar and on startup, it crashes on the assert I commented out in client/sidebar/sidebar-branch.vala when Geary tries to load a/the second account..
It also crashes when you remove an account used by Geary in GOA, I'll have to find out how to do this gracefully.

I've had some trouble managing the password prompts though, i'm wondering how to get Geary off the assumption that passwords always have to be directly provided by the user; so it's a bit hackish in some ways. That would make it easier to add oauth2 support later on.

Note that this depends on the account identifier patches in bug 714643, having an unique id gave me the possibility of prefixing goa account folders with "goa_" so they would be easily recognizable without having to open geary.ini.

Hmm, starting off with password-based accounts does sound like a good idea. A few random thoughts about this:

Contrary to Jim's comment Bug 714877, I feel like Geary should automatically add and remove accounts from GOA without further user interaction. I'm not concerned about the cost of downloading email issue since it will only default to 2 weeks back anyway. This is, after all, the whole point of SSO.

GOA accounts should appear in the account list, and it should be obvious that they are GOA-managed. They should not be able to be removed directly, but the user will need a way to get to the GOA capplet easily to do that and to change settings there. This will probably need to be kept in mind with the design work going into Bug 714104.

GOA accounts will need to have editable account prefs since things like the signature, alternate addresses, download limits, etc still need to be able to be changed. Also, GOA doesn't have an on/off switch for password based email accounts, so Geary needs to provide one in for it in the account list or in the account's prefs - in case there is a GOA account that the user does not want enabled in Geary.

Since we want to use the the existing generic account implementation for GOA-managed password accounts, and in the future will want use the existing GMail account implementation for GOA-managed GMail accounts, we need to be able to instead distinguish between sources of auth-related information rather than say subclasses. We want to be able to support additional auth providers like UOA as well (Bug 714877).

All of this has a few implications:
 - GOA accounts need a geary id and geary.ini file anyway
 - We probably need to introduce an AuthProvider enum in the same vein as the ServiceProvider class, and hence "auth_provider" key in geary.ini.
 - All of the AccountInformation *_imap_* properties and their uses need to be replaced with a single class, in the same vein as the Endpoints classes, same for *_smtp_*. In fact, I think a few of those properties should just be obtained directly from the Endpoint classes anyway.

Oh, also, we need a way to flag accounts as being disabled in geary.ini, and in the engine.

(In reply to Michael Gratton from comment #11)
> Also, GOA doesn't have an on/off switch for password based
> email accounts, so Geary needs to provide one in for it in the account list
> or in the account's prefs - in case there is a GOA account that the user
> does not want enabled in Geary.

If there is a good use-case for it, then we can add the switch in the GOA panel itself.

The reason there is no switch at the moment is because I couldn't convince myself why an user would want to disable an email account. Note that email accounts are different from Google / Facebook accounts which offer multiple services in multiple applications and I can see why one would like some granularity there.

That said, Kerberos accounts also have a switch, so I don't have a strong objection to adding one for email. I am only curious about the use-case.

(In reply to Debarshi Ray from comment #13)
> (In reply to Michael Gratton from comment #11)
> > Also, GOA doesn't have an on/off switch for password based
> > email accounts, so Geary needs to provide one in for it in the account list
> > or in the account's prefs - in case there is a GOA account that the user
> > does not want enabled in Geary.
> 
> If there is a good use-case for it, then we can add the switch in the GOA
> panel itself.

Oh hey, I'm not complaining. In fact I'm not sure if such a use case exists either.

I /can/ imagine the inverse scenario, where a user needs to disable a GOA account in specific apps: C has a personal GMail account and a work Exchange account, and wants to access the GMail account from Geary for obvious reasons, but has to access the Exchange account from Evolution, because corporations. Here they would want to disable the GMail account in Evo, and the Exchange account in Geary - but both would be present in GOA.

For the purposes of GOA support in Geary, I'd be inclined to make disabling accounts possible in general and hence a separate bug anyway - it could be more broadly useful.

And to formalise this: Bug 768974 submitted for disabling accounts.

Further, the under-the-hood work required to get both pluggable endpoint configuration and authentication credentials is now covered at Bug 768975.

Thanks to Oskar Viljasaars work over in Bug 768975 and here, Geary now has support for GOA IMAP password-based accounts as of commit 25e2172e. This means that people who add IMAP accounts to GOA will see them automatically appear in Geary in current Git master.

This does not address adding support for OAuth2 (i.e. Google's proprietary auth extension), we'll do that over in Bug 746705, but it does lay the ground work for supporting OAuth2 nicely.

Also, I probably was a bit too keen to push this: The accounts UI still pretends to allow users to change their GOA password, which isn't supported. I'll cook up a fix for that in the next few days then close this.

Great to see all the work being done in this direction! :)

(In reply to Debarshi Ray from comment #18)
> Great to see all the work being done in this direction! :)

Finally, at least! :)

Support for OAuth2 GOA accounts just landed over in https://gitlab.gnome.org/GNOME/geary/merge_requests/8. There's still some UI work to be done for both GOA and OAuth2 the accounts dialog, but I'll take care of that in Bug 714104.

As such, I'll close this as fixed. Please file new tickets for any problems with the implementation that people might find.