After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 689549 - Start-up performance very bad for large Google (EDS) contact lists
Start-up performance very bad for large Google (EDS) contact lists
Status: RESOLVED OBSOLETE
Product: folks
Classification: Platform
Component: e-d-s backend
git master
Other Linux
: Normal major
: Unset
Assigned To: folks-maint
folks-maint
Depends on:
Blocks:
 
 
Reported: 2012-12-03 17:53 UTC by Travis Reitter
Modified: 2018-09-21 16:00 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
Callgrind run on folks-inspect with an EDS/Google account containing 6,000 contacts. (357.08 KB, application/octet-stream)
2012-12-03 17:53 UTC, Travis Reitter
Details
vCard file of 6000 generated contacts (578.71 KB, text/vcard)
2013-02-28 05:46 UTC, Travis Reitter
Details

Description Travis Reitter 2012-12-03 17:53:07 UTC
Created attachment 230547 [details]
Callgrind run on folks-inspect with an EDS/Google account containing 6,000 contacts.

I've heard a few people complain poor Folks performance for a while, but hadn't been able to reproduce it myself for a while.

There is a huge set-up time (for IndividualAggregator.prepare(), I assume) for large contact lists. It takes folks-inspect about 2 minutes to settle with a single 6,000-contact Google account (set up in Evolution Contacts). The entire time, folks-inspect burns 100% CPU.

There's a lot of investigation still to do. Unfortunately, callgrind doesn't yield any easy answers. Most of the actual time is constructing and destructing GObjects (at least in large part, Gee objects), emitting and handling signals. But in the recorded run, g_object_constructor is called 11.9 million (!) times. So, there's got to be a better way to re-use our objects (especially if it turns out we're creating and destroying Gee objects unnecessary). This run took 56 minutes within callgrind, so I'm going to have to create a scaled-down test account while whittling down this excess work.

There is a slight hint in the callgrind output that a lot of this comes down to the actual aggregation/linking, which probably shouldn't be a surprise.
Comment 1 Travis Reitter 2013-01-22 05:14:38 UTC
We should also see how much of this may be due to EDS itself. It looks like some work that recently landed (and may land in the future) has significant speed-ups for the main address book, though it's not clear how much it would affect large Google remote address books:

http://blogs.gnome.org/tvb/2013/01/15/getting-your-contacts-right-now/

This bug is probably still mostly on our side though (due to the aggregation work).
Comment 2 Travis Reitter 2013-02-28 05:46:19 UTC
Created attachment 237578 [details]
vCard file of 6000 generated contacts

I imported this list of contacts (through Evolution) into a Google account for testing out this bug
Comment 3 Simon McVittie 2013-03-04 20:58:54 UTC
So far, I haven't been able to reproduce anything this slow so far by writing Folks test-cases: the worst I've encountered is that IndividualAggregator quiescence takes about 5 seconds per 10K contacts (scaling linearly).
Comment 4 Simon McVittie 2013-03-04 21:24:45 UTC
The trick to getting any contacts at all out of the Google address book in my test environment seems to be to run ${libexecdir}/evolution-data-server/evolution-addressbook-factory explicitly. If I just let it run automatically in a fresh login session, preparation of the Google backend times out and I don't get any Google contacts at all. Perhaps some sort of environment problem?

With 2049 Google contacts, "time folks-inspect quit" (or "time folks-inspect persona-stores", or any other trivial command) takes about 2.5 seconds, which seems roughly consistent with a simple keyfile taking 5 seconds per 10K contacts - a factor of 2.5 for communication with e-d-s, and the fact that there are more contact fields involved, seems acceptable.

If you can reproduce worse timings, then I think something other than the aggregator is causing them, perhaps related to the initial-retrieval problems I'm seeing - perhaps something is not right in the Google e-d-s backend?
Comment 5 Philip Withnall 2013-03-04 23:14:01 UTC
(In reply to comment #4)
> perhaps something is not right in the Google e-d-s backend?

You can get debug information out of the Google EDS backend by running evolution-addressbook-factory with GOOGLE_DEBUG=1 G_MESSAGES_DEBUG=all LIBGDATA_DEBUG=2, which will print output for every HTTP message sent/received. That should allow you to see if it’s stalled anywhere.
Comment 6 Simon McVittie 2013-03-05 18:06:44 UTC
(This is all with 2049 Google contacts; cached, unless otherwise stated.)

Findings so far:

* Verbose debugging, even into a file, has a somewhat significant cost (7.5s vs 3s to prepare the EDS backend and count individuals, with G_MESSAGES_DEBUG=all or G_MESSAGES_DEBUG=).

* The Telepathy backend doesn't cope well with inability to autolaunch a D-Bus session bus, but that's a separate bug.

I've observed the behaviour of a Folks client eating up all the memory and CPU, but only once in a couple of days' experimenting. I wonder whether there's some special situation (refreshing the list from Google?) that triggers it?
Comment 7 Philip Withnall 2013-03-05 23:20:08 UTC
(In reply to comment #6)
> refreshing the list from Google?

You can simulate that by deleting your ~/.cache/evolution/addressbook/google_*/cache.xml file, or by editing the refresh period in the address book configuration in Evolution (e.g. to every 1 minute).
Comment 8 GNOME Infrastructure Team 2018-09-21 16:00:07 UTC
-- GitLab Migration Automatic Message --

This bug has been migrated to GNOME's GitLab instance and has been closed from further activity.

You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/folks/issues/50.