Has there been any progress on this? Happy to contribute if it helps.

Currently not, due to lack of testing environment. I'll be glad to review patches, if you mean this kind of contribution, though I would still need the testing environment to verify that the code changes do what they are supposed to do.

Working on getting a set of testing credentials if that helps.

I tried to turn on a two-factor authentication, as suggested in a downstream
bug [1], but it wasn't enough to let the server offer me other than:
< WWW-Authenticate: Basic Realm=""

I expect to see there more WWW-Authenticate headers than just the one with Basic, for example something like OAuth or OAuth2. You can try it too, when running evolution form a terminal like this:

   $ EWS_DEBUG=2 evolution

and then watch the output from the server. The first response from the server should contain WWW-Authenticate header(s), eventually when being in account Properties, Receiving Email tab, clicking the "Check for supported types" in the Authentication Type section.

And, as kind of expected, you do not have the account added into Evolution, then it's a bit harder. While looking into the MSDN page from comment #0, they also use outlook.office365.com, the same server as I do, thus maybe they do support OAuth2, but they do not advertise it with the server. That's doable, but not nice of them. I mean, maybe my expectation about advertising all supported authentication methods is false.

I've been able not-to-login to the server when I had the two-factor authentication enabled, but the response didn't give me any hint that I should switch to another authentication type, it just failed with:

< HTTP/1.1 401 Unauthorized
< Soup-Debug-Timestamp: 1515669524
< Soup-Debug: ESoapMessage 1 (0x7f9584017710)
< Server: Microsoft-IIS/10.0
< request-id: ee7bcdfa-b4a2-432f-a885-2b7db7fb5871
< X-CalculatedFETarget: VI1PR06CU002.internal.outlook.com
< X-BackEndHttpStatus: 401
< Set-Cookie: exchangecookie=33408b9233b643b0af731368d726d071; expires=Fri, 11-Jan-2019 11:17:32 GMT; path=/; HttpOnly
< X-FEProxyInfo: VI1PR06CA0049.EURPRD06.PROD.OUTLOOK.COM
< X-CalculatedBETarget: VI1PR1001MB0992.EURPRD10.PROD.OUTLOOK.COM
< X-BackEndHttpStatus: 401
< X-DiagInfo: VI1PR1001MB0992
< X-BEServer: VI1PR1001MB0992
< X-FEServer: VI1PR06CA0049
< X-Powered-By: ASP.NET
< X-FEServer: VI1PR0802CA0045
< WWW-Authenticate: Basic Realm="",Basic Realm="",Basic Realm=""
< Date: Thu, 11 Jan 2018 11:17:36 GMT
< Content-Length: 0

The value of WWW-Authenticate header is kind of suspicious.

As PJ mentioned in the comment #0, their code sample is kind of crappy, with which I agree. I'd need pure communication log, to see what they sent and what they received, to know "how it works". It's known for Google, when using OAuth2, but not that much with office365.com. On the other hand, the OAuth2 protocol is well described and it's (sort of) standard, thus they should not diverge in any way.

I've got an idea to try with their IMAP server (imap-mail.outlook.com). I see they do advertise these authentication methods:
> AUTH=PLAIN AUTH=XOAUTH2
which makes me believe that my expectations about the WWW-Authenticate headers when accessing the EWS server are really wrong and the only thing I'd do is to enable two-factor authentication on the server and then use OAUth2 without receiving any hint from the EWS server.

I cannot promise any quick fix, it requires more changes in eds/evo/ews, but I'll try my best to have it available in 3.30.0.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1533254

A bit better documentation:
https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-protocols-oauth-code

Okay, I somehow managed to connect to outlook.com IMAP and SMTP using OAuth2, but not for EWS. The EWS is particularly more complicated, it even requires paid account (MSDN subscription) to be able to create an application [1][2]. It also requires the administrators of the EWS account (not respective users) to enable the 3rd-party application to connect to the server as the user, because the application (evolution-ews) should be a multi-tenant application [3]. I tried to create a new Azure AD application at [4] by picking "Azure Active Directory" in the list on the left, but I get there 

>  Access denied
>  You do not have access
>  Looks like you don't have access to this content. To get access, please
>  contact the owner.

I do not know whom the "owner" is, but I suspect that mu outlook.com account doesn't have enough privileges to manage Azure AD. I'm surely not going to pay them, every year, any fee to be able to provide an open source application to connect to the Office365 server using EWS protocol.

I tried to use token provided by outlook.com API, but they do not work for EWS, they are rejected with something like:

> X-BackEndHttpStatus: 401
> x-ms-diagnostics: 2000010;reason="ErrorCode: 'PP_E_RPS_REASON_OFFERACTIONS_INVALID'.
>   Message: 'All the offeractions povided in the property bag cannot be
>   validated for the token.%0d%0a'";error_category="invalid_msa_ticket"
> WWW-Authenticate: Bearer client_id="00000002-0000-0ff1-ce00-000000000000",
>   trusted_issuers="00000001-0000-0000-c000-000000000000@*",
>   token_types="app_asserted_user_v1 service_asserted_app_v1",
>   authorization_uri="https://login.windows.net/common/oauth2/authorize",
>   error="invalid_token",Basic Realm="",Basic Realm="",Basic Realm=""

which doesn't have many hits with Google.

There would be probably one option, let the company administrators configure their own application, in which case the evolution-ews would be a single-tenant application, having configured, in some way, the tenant itself and some other values, possibly through GSettings, but it a) doesn't fix the issue for regular (non-corporate/personal) accounts at outlook.office365.com; b) I cannot test, which I really should do before telling it's done.

I mean, I'm on hold at the moment, until I find anyone being able to register a multi-tenant application at [4], which I could eventually use for evolution-ews.

[1] https://msdn.microsoft.com/en-us/office/office365/howto/setup-development-environment
[2] https://msdn.microsoft.com/en-us/library/office/dn626019(v=exchg.150).aspx
[3] https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-authentication-scenarios#basics-of-registering-an-application-in-azure-ad
[4] https://portal.azure.com/

Couple other links I do not want to lost:
https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-devhowto-multi-tenant-overview
https://tsmatz.wordpress.com/2016/10/07/application-permission-with-v2-endpoint-and-microsoft-graph/

To add a bit for the tenant thing, using my account GUID as the tenant, I get an error response:

> AADSTS50020: User account 'my@microsoft-account' from identity provider
> 'live.com' does not exist in tenant 'Microsoft Services' and cannot access
> the application 'GUID of GNOME Evolution in live.com' in that tenant.
> The account needs to be added as an external user in the tenant first.
> Sign out and sign in again with a different Azure Active Directory user
> account.

Then I tried to add 'GUID of GNOME Evolution in live.com' (that's really a GUID, not this text in the above error), just in case, but it also fails, with:

> AADSTS90002: Tenant 'GUID of GNOME Evolution in live.com' not found. This
> may happen if there are no active subscriptions for the tenant. Check with
> your subscription administrator.

Created attachment 367381 [details] [review]
unfinished ews patch

Here is an unfinished change, which:
a) doesn't get proper OAuth2 token for outlook.office365.com and EWS protocol
b) doesn't use ESoupAuthBearer for "Office365" authentication method; how
   to actually use it properly shows ESoupSession (both in evolution-data-
   server), while ews may use it similarly as its own ESoupAuthNegotiate.
   The evo-ews should ideally use ESoupSession instead of SoupSession, but
   I'm unsure of (negative) side-effects of such change, not talking that
   EEwsConnection doesn't have access to the associated ESource, thus it might
   better copy relevant parts of the Bearer authenticator usage from eds.

One more link here:
https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-devhowto-multi-tenant-overview

I am more than happy to test patches and provide feedback,headers,etc. Subscribing to monitor for changes.

I just found these steps to generate application specific passwords which should be usable by evolution-ews. It depends whether your company administrator allowed to create such, but one can give it a try to be sure:
https://support.office.com/en-us/article/create-an-app-password-for-office-365-3e7c860f-bda4-4441-a618-b53953ee1183

I made some initial changes (commit 6b7aff3), which are part of 3.27.90, but the code is work in progress, only for testing at the moment, thus also disabled by default, but can be enabled in runtime with:
   export EWS_WITH_OAUTH2=1
then the OAuth2 is offered for outlook.office365.com account, where at least the Tenant and Application ID should be filled. Note that this requires development version 3.27.90 of evolution-data-server and evolution and it cannot be used with any earlier version.

Well done Milan!
I have tested version 3.27.90 and here are my findings.

If you use the scope as defined at this moment, it does not work. The error that shows up is:

Message: AADSTS70011: The provided value for the input parameter 'scope' is not valid.

Looking at the headers from the webmail, the scope at authorize time is defined as 'openid'. So I patched it by inserting 'openid' at the top, leaving offline_access and the URLs for mail and calendar read/writes.

This did not work.

Then I removed everything except 'openid' and it worked. Brought me to the company sign-on webpage. Then this redirected to the dual factor web page and then microsoft asked if I wanted to trust the application and proceed.

So far so good. OAuth2 worked. However, instead of closing the authorization window and starting to update emails in evolution, it just loaded the OWA page in the dialog window and evolution inbox remained disconnected.

It only prints twice the line below(prior of authentication attempts):
e-mail-engine-WARNING **: mail_folder_cache_note_store_thread: Failed to run initial setup for 'emailaddress': Office365 authentication failed

Thank you,
-N

(In reply to Nikolay Kichukov from comment #12)
> Message: AADSTS70011: The provided value for the input parameter 'scope' is
> not valid.

Right, I noticed it only after making the release, I deleted too much when cleaning up code from my test garbage:
https://git.gnome.org/browse/evolution-ews/commit/?id=6824028e0685e76cde2f2fbaabb6ad74c8e057cd

> It only prints twice the line below(prior of authentication attempts):
> e-mail-engine-WARNING **: mail_folder_cache_note_store_thread: Failed to run
> initial setup for 'emailaddress': Office365 authentication failed

That all depends on used tenant, application ID (if you used the application ID you found from elsewhere, then it belongs to "O365 Suite UX", which is web application, not a native application) and eventually actual OAuth entry points, which is kind of mess. The code currently "uses" v2.0 entry points, but it may probably work also without it. I do not know. I do not want to write here all my tests and theories, I spent quite a lot of time on this. I opened a ticket with Microsoft asking for some guidance. I'll update the sources and/or this bug when I know more.

I made some more changes in the sources (commit d96735f), which, with correct tenant and application ID in that tenant allow me to open an outlook.office365.com account in evolution-ews using OAuth2. There are still things to be tested and verified. My trial application ID doesn't work for free outlook.com accounts, but it seems to work for those in portal.azure.com. I'd like to collect necessary information to make this work (to obtain proper tenant and eventually create an Application in https://portal.azure.com , which may or may not be possible for regular users with company/school accounts), but I'll need help with this from someone having such advanced accounts.

Hi Milan,
For some reason I was unable to compile 'master' (cmake package dependencies?), so I had to switch back to 3.27.90 release and apply all commit patches on top.

Still the native entry point does not work. I will provide details in the chat.
Thanks for your great work on this!
-Nik

Final change done for this bug. It requires cooperation from the company admins. I tried to write a page for it at:
https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2
which is referenced also when trying to setup an OAuth2 for the EWS account, thus it's easy to find. I'll appreciate any feedback on the page content, especially when some parts might not make much sense. It's this complicated, because Microsoft needs it to be that way (my personal understanding of the matter), which has its pros and cons.

Created commit a487e14 in ews master (3.27.91+)

Hi Milan,

once again a big thank you for your dedication on this and the excellent job in fulfilling this enhancement request. I have gone through the documentation page and it is pretty good - easy to grasp and implement from there.

I will provide feedback if/when we have our corporate application approved.

Regards,
-Nikolay

Hi Milan,

I also want to thank you for working on this issue.  I'm not sure if this will work for my situation.  In our Office 365 environment, Office365 hands off authentication to our locally hosted ADFS servers.  We have MFA enabled on our local ADFS server using Duo Security.

I've documented the steps for login that we see on a web page with images:
http://staff.tamucc.edu/phale/office365-login-process/

Please let me know if this helps or if you need any other information.

Thanks,

Phil

(In reply to Phil Hale from comment #18)
> I'm not sure if this will work for my situation.

That's no problem. It's a common situation with OAuth2 when the company offers things like single sign on or anything like that. Companies can achieve the same with other OAuth2-capable servers too, like with Google.

As you can see, the login.microsoftonline.com redirects to your company site for actual authentication. Evolution-ews begins the OAuth2 at the same login.microsoftonline.com place. You'll go through exactly the same steps using evolution-ews as you did with Firefox.

Good deal!  Looking forward to testing this out.  Thanks again for working on this.  

Now if I can only find Evolution+Evolution-EWS devel release packaged up as a flatpak! ;-)

Phil

You can build it yourself [1], only instead of downloading
   org.gnome.Evolution-stable.json
use
   org.gnome.Evolution-master.json
and make similar changes in the cook book too (where is 'stable' use 'master'). Similar info can be found also here [2]. Using flathub is a problem for me, it's hosted on github, which is not a friend of me, I'm sorry. Maybe I'll figure out how to create my own flatpak repository one day, which will make things significantly easier for early testers.

Please note that the org.gnome.Evolution-master.json references development version, thus it's expected to face bugs (more than with stable version).

[1] https://wiki.gnome.org/Apps/Evolution/Flatpak
[2] https://git.gnome.org/browse/evolution/tree/flatpak/

Hello Milan,

Thanks again for all your help on this!

I've been successful in building a flatpak of the master branch and the auth seems to be working.  I had my Office365 administrator follow the directions and set up a "Evolution-EWS" application with the 15 or so permissions you recommend in the https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2.  Everything looks good on his side, but I continue to get the following error after successful login, "Need admin approval

Evolution-EWS needs permission to access resources in your organization that only an admin can grant.  Please ask an admin to grant permission to this app before you can use it."

Request Id: 141a56a1-d384-45b6-8ff7-376c011e0a00
Correlation Id: aa612265-2026-4a33-86f1-aea1398e3603
Timestamp: 2018-02-14T15:26:15Z
Message: AADSTS90094: The grant requires admin permission.
"

Oh so very close!  I've been rebuilding the master flatpak every few days, but I continue to see the same error.  My Office365 admin says he's granted all the right permissions and sent me the information I needed to plug in.  I get all the correct login prompts and MFA, but still hit this.  The funny thing is when he tests his account it works!  He's an Admin and I'm a normal user.  Any ideas?

Phil

Hello Milan,

We've determined that the directions offered on the page, https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2 only work for the Office365 admin who created the Evolution-EWS app in our tenant.  I can see the app in my Office365 control panel, but I never get the prompt to grant permissions to the app.  At this point the only account that is able to complete the setup per the wiki page is our Admin.  Something is not quite right in the App permissions for normal users.  We've checked the error message ID, "Message: AADSTS90094: The grant requires admin permission." on Google and made recommended changes, but it has not helped.  Any ideas you might have would be greatly appreciated.

Phil

Milan,

We have potentially found the issue.  The Evolution continues to prompt for "admin_consent" in the login string:

https://login.microsoftonline.com/34cbfaf1-67a6-4781-a9ca-514eb2550b66/oauth2/authorize?resource=https%3A%2F%2Foutlook.office.com&response_type=code&response_mode=query&login_hint=phil.hale%40tamucc.edu&prompt=admin_consent&client_id=dfd04dea-3123-4290-abb9-4ecc54587631&redirect_uri=https%3A%2F%2Flogin.microsoftonline.com%2Fcommon%2Foauth2%2Fnativeclient&scope=openid+offline_access+profile+Mail.ReadWrite+Mail.ReadWrite.Shared+Mail.Send+Mail.Send.Shared+Calendars.ReadWrite+Calendars.ReadWrite.Shared+Contacts.ReadWrite+Contacts.ReadWrite.Shared+Tasks.ReadWrite+Tasks.ReadWrite.Shared+MailboxSettings.ReadWrite+People.Read+User.ReadBasic.All

The above is from my debug session.  In the string, we see "prompt=admin_consent".  According to my Office365 Admin, this should only occur once, for him to grant permission.  The evolution client from then on needs to use "prompt=login".

Any way we can make this change in master?  I'd be happy to test.

Phil

Thank you for early testing.

I see. My trial Azure account also provides a mail account on which I tested the Wiki steps and the evo-ews code. The problem is that I am also an admin of that account, thus it behaves the same for me as for your admin.

I found somewhere that the admin_consent is requires for EWS. Without it it failed here; it's when I didn't use any 'prompt' argument. Using the 'login' as you suggested (there is also 'consent' value [1]) seems to work too, thus I made the change.

Created commit 820bd13 in ews master (3.27.91+)

[1] https://docs.microsoft.com/en-us/azure/active-directory/develop/active-directory-protocols-oauth-code

Hi,

Well done! Running with evolution-ews commit 820bd13 applied made it work for me too. It feels so nice to be able to do emails in a proper email client again!

Cheers,
-Nikolay

The most sad is that other APIs work without all that additional settings on the azure.portal, like here:
https://oauthplay.azurewebsites.net/
It seems only EWS with OAuth2 requires all that additional craft. I can open even my free outlook.com address on that above page, but not with EWS, where it claims something about wrong tenant for live.net addresses.

Hello Milan!

The updates have been working just fine for me.  Thank you for all your work on this.  My Office365 administrator has some notes/corrections to the document listed here: https://wiki.gnome.org/Apps/Evolution/EWS/OAuth2

6. c. iii. App permission that is only needed for O365 Exchange Online

    Access mailboxes as the signed-in user via Exchange Web Services

 

6. d.

To authorize the app for everybody:

If your setting for ‘Users can consent to apps accessing company data on their behalf’ is set to No (More secure), you (AAD Admin) can do the option below

(In AAD admin center – Users – User settings – Enterprise applications)

 

Copy the link below to a browser and fill the <TENANT ID HERE> and <APPLICATION ID HERE>

https://login.microsoftonline.com/<TENANT ID HERE>/oauth2/authorize?resource=https%3A%2F%2Foutlook.office.com&response_type=code&response_mode=query&prompt=admin_consent&client_id=<APPLICATION ID HERE>&redirect_uri=https%3A%2F%2Flogin.microsoftonline.com%2Fcommon%2Foauth2%2Fnativeclient

 

Prompt=admin_consent

If your setting for ‘Users can consent to apps accessing company data on their behalf’ is set to Yes (I think this is the default), users can do the option below

(In AAD admin center – Users – User settings – Enterprise applications)

 

--
 I'll forward you his e-mail as it includes some images.  

Phil

(In reply to Phil Hale from comment #28)
> 6. c. iii. App permission that is only needed for O365 Exchange Online
> 
>     Access mailboxes as the signed-in user via Exchange Web Services

Hmm, to be honest, I didn't try to limit the permissions to the smallest possible set, I expected that the scopes the evolution-ews asks for should match what the application on the Azure web page has checked. Having less options sounds interesting. I suppose you see all your calendars/books/tasks and you can create/modify/delete them too? That would be really interesting.

> --
>  I'll forward you his e-mail as it includes some images.  

Thanks a lot, I'll update the Wiki page accordingly.

Milan,

The permissions were based on what he was able to observe from the Apple iOS access for mobile mail/calendar/contacts along with what was in place for another cross-platform application we looked at called Hiri. As far as functionality goes, everything appears to work just fine.  E-mail, Calendars, Tasks, Memos all work as expected.

Phil

Thanks for the confirmation. I updated the Wiki page with the information from your mail, while I reworded it a bit and I didn't include the part when users can add applications, because the URL being used is the one which evolution-ews uses, thus the only reason why the users do not see a "confirm the application access" page is that they have already stored the confirmation made either earlier by them or made by the administrator on their behalf. Otherwise the users see the confirmation page and when they confirm they are logged in (and receive the OAuth2 token). I thought that the administrator can do the same when opening in portal.azure.com the Azure Active Directory->App Registrations->click the EWS application->Settings->Required permissions->and here at the top the "Grant Permissions" button, which opens a dialog asking:

   Do you want to grant the permissions below for APPLICATION for all accounts
   in current directory? This action will update any existing permissions this
   application already has to match what is listed below.

But I can be wrong. All these Azure Active Directory settings are pretty confusing for me.