GNOME Bugzilla – Bug 781590
2.58.0 fails to connect to EWS server: HTTP/1.1 1 Cancelled
Last modified: 2017-05-03 08:22:43 UTC
I recieve the following message evolution randomly throughout the day. This is happening for exchange address books. I see the following messages in console: (evolution:14032): evolution-util-WARNING **: name_selector_get_client_cb: Unable to connect to “[.*]”: Message recipient disconnected from message bus without replying I don't see any relevant logs with EWS_DEBUG=2.
Which kind of address books are configured in Evolution? Which kind of servers are used for remote address books?
These are all EWS address books. The server is Exchange Server 2016 Cumulative Update 4 (CU4).
Thanks for a bug report. It looks like evolution-addressbook-factory-subprocess with "--factory ews" is stuck in something. I suspect libsoup or ntlm_auth, but it needs more debugging. Could you install debuginfo packages for evolution-data-server, evolution , evolution-ews and libsoup and then reproduce the issue (this particular one comes from an entry in a mail message composer (File->New->Mail Message). To get logging from EWS and eventually see what happened to the background process run the main factory process from a terminal as this: $ EWS_DEBUG=2 /usr/libexec/evolution-addressbook-factory -w before you run evolution. Give it 3-5 seconds to be registered in D-Bus, then run evolution and open the composer window. By the way, that “[.*]”, is it a verbatim copy of the message, or you replaced the name of the address book by this? I do not recall seeing address book name of such name.
I forgot to mention, the debuginfo packages are good for better backtrace of the stuck evolution-addressbook-factory-subprocess, where the backtrace can caught by a command like this: $ gdb --batch --ex "t a a bt" -pid=PID &>bt.txt Please check the bt.txt for any private information, like passwords, email address, server addresses,... I usually search for "pass" at least (quotes for clarity only). The PID in the command should be replaced with a process ID of the evolution-addressbook-factory-subprocess process whose --factory argument is followed by "ews" string. You can see them for example with: $ ps ax | grep evolution
The [.*] was me. Their are about 10 different address books that will show that message. I'll have to rebuild with debug symbols. Once I have something meaningful I'll update the ticket. Thanks for the help.
I just tried with a fresh install of Fedora 26 Alpha, which also contains evolution* 3.24.1. I configured a new EWS account in GNOME Online Accounts and it worked fine, I could reach my address books and lookup for addresses there when composing a new mail. I tried also after restart, still fine. I hope the backtrace will show something useful. Or at least the EWS_DEBUG log.
Created attachment 350437 [details] ews debug
Created attachment 350438 [details] backtrace
EWS debug and backtrace attached. Hopefully I have captured something useful.
Thanks for the update. The ews debug log shows a use-after-free in libsoup. It also claims some issue (see below). What is your libsoup version, please? Could you re-run the factory like this, please? Install 'valgrind, if you've not yet: $ G_SLICE=always-malloc valgrind --trace-children=yes \ /usr/libexec/evolution-addressbook-factory -w Wait a bit longer, the process will be significantly slower due to memory checking. It's possible it'll not crash, but valgrind would still notify about the memory issues on the console, thus, please, keep the log somewhere. By the way, the log shows that all the requests had been cancelled. That's unusual. How do you reproduce this precisely, please? And the way you configured this EWS account is? Directly in evolution, or with GNOME/Ubuntu Online Accounts? > (evolution-addressbook-factory-subprocess:3079): libsoup-CRITICAL **: > soup_message_queue_remove: assertion '!item->removed' failed > > (evolution-addressbook-factory-subprocess:3079): GLib-GObject-WARNING **: > gsignal.c:2641: instance '0x7fc98c021020' has no handler with id '876' > > *** Error in `/usr/lib/evolution-data-server/evolution-addressbook- > factory-subprocess': double free or corruption (fasttop): 0x00007fc914002e20 *** >======= Backtrace: ========= >/usr/lib/libc.so.6(+0x722ab)[0x7fc9a02af2ab] >/usr/lib/libc.so.6(+0x7890e)[0x7fc9a02b590e] >/usr/lib/libc.so.6(+0x7911e)[0x7fc9a02b611e] >/usr/lib/libsoup-2.4.so.1(+0x8211e)[0x7fc99cebf11e] >/usr/lib/libsoup-2.4.so.1(+0x82a72)[0x7fc99cebfa72] >/usr/lib/libsoup-2.4.so.1(+0x82b26)[0x7fc99cebfb26] >/usr/lib/libglib-2.0.so.0(g_main_context_dispatch+0x15a)[0x7fc9a0ab266a] >/usr/lib/libglib-2.0.so.0(+0x4aa20)[0x7fc9a0ab2a20] >/usr/lib/libglib-2.0.so.0(g_main_loop_run+0xc2)[0x7fc9a0ab2d42] >/usr/lib/evolution-ews/libevolution-ews.so(+0x1c29e)[0x7fc97f57129e] >/usr/lib/libglib-2.0.so.0(+0x72295)[0x7fc9a0ada295] >/usr/lib/libpthread.so.0(+0x72e7)[0x7fc99f84c2e7] >/usr/lib/libc.so.6(clone+0x3f)[0x7fc9a032954f]
Created attachment 350469 [details] valgrind output with samba/ntlm_auth installed
Created attachment 350470 [details] valgrind output with samba/ntlm_auth removed
With samba installed which provides ntlm_auth I was getting different results then without it installed. I've attached both. I'm using libsoup 2.58.0.
These are the steps I took in each case. 1. Start Evolution. 2. Click the New icon in the toolbar. 3. Try to autocomplete and address by typing in a known name.
Thanks for a quick update. Do you use locally stored Global Address List (GAL) or the other one? You can find out in Edit->Preferences->Mail Accounts-><ews account>->Edit->Receiving Options tab. It's at the bottom.
(In reply to Milan Crha from comment #15) > Do you use locally stored Global Address List (GAL) or the other one? I have cache offline address book enabled for the GAL.
(In reply to William Shuman from comment #13) > I'm using libsoup 2.58.0. That's it. When I installed 2.58.0 release from the tarball, it behaved oddly, I receive many "No response: Cancelled", but when I use 2.56.0, then it behaves properly, I see my contacts from an EWS server. I wasn't able to reproduce that use-after-free, thus I'm renaming this bug report and I'm moving it to libsoup.
(In reply to Milan Crha from comment #17) > (In reply to William Shuman from comment #13) > > I'm using libsoup 2.58.0. > > That's it. When I installed 2.58.0 release from the tarball, it behaved > oddly, I receive many "No response: Cancelled", but when I use 2.56.0, then > it behaves properly, I see my contacts from an EWS server. > > I wasn't able to reproduce that use-after-free, thus I'm renaming this bug > report and I'm moving it to libsoup. Quickly bisected. Looks like it is this commit. https://git.gnome.org/browse/libsoup/commit/?id=28849060f5b8d8829bc4fc6d208d22f15e869f46
True, I just realized the same, it's after the change from bug #777326. Going before that change fixes the issue.
I managed to reproduce that use-after-free (see comment #10) and filled bug #781777 for it.
Why are you setting a new proxy resolver if you're not modifying proxy settings...?
(In reply to Michael Catanzaro from comment #21) > Why are you setting a new proxy resolver if you're not modifying proxy > settings...? Because I do not know whether the proxy settings are in effect. It's all automated and transparent, as simple as a single call of: e_binding_bind_property ( cbews, "proxy-resolver", oab_cnc, "proxy-resolver", G_BINDING_SYNC_CREATE); where e_binding_bind_property() is just a thread-safe wrapper of g_binding_bind_property(). This call is even done right after the connection is created, far before the connection is established. It's much simpler this way, than trying to check on each single place whether proxy is required and made appropriate changes only if it changes in the parent object (cbews above). That's the GBinding for, to make things simpler and to avoid code duplication. Maybe your change (in the other bug) blindly calls soup_session_abort() without checking that something actually changed. You know, glib is able to call fake property change notifications (fake as when the property didn't change at all), which is one of the reasons why evolution* code: a) verifies that value changed before calling g_object_notify() itself b) has other wrappers which mute notify propagation when the value did not change If I recall correctly, calling g_object_set() can also invoke g_object_notify() on its own, which is rather wrong, thus it had been changed in some glib where one can add some flag or something to avoid this fake-notify-call behaviour, but this is just from my memory, which is rather poor.
I'm reverting the change as that's the simplest fix
Just for a record, it had been commit 8d337030b. It would be nice to do 2.58.1 release, thus distros will pick it up easily and quickly (this is quite sever regression from my point of view). I'm afraid of many reports coming up due to evolution-data-server using libsoup this way (comment #22) and apparently also evolution-ews, which is the main advantage for corporate users connecting to Microsoft Exchange servers.