After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 733642 - gdata: Query the service in batches
gdata: Query the service in batches
Status: RESOLVED DUPLICATE of bug 770656
Product: gnome-online-miners
Classification: Applications
Component: general
unspecified
Other All
: Normal normal
: ---
Assigned To: GNOME Online Miners maintainer(s)
GNOME Online Miners maintainer(s)
Depends on:
Blocks:
 
 
Reported: 2014-07-23 23:50 UTC by Debarshi Ray
Modified: 2016-08-31 16:32 UTC
See Also:
GNOME target: ---
GNOME version: ---


Description Debarshi Ray 2014-07-23 23:50:54 UTC 
In the past we have had performance issues when starting with a clean DB.

One cause of this was that the number of items shown by GtkIconView would keep growing in the application. It gets very slow as the thumbnails get created and the GdkPixbufs are updated in the model. We have reduced this by limiting the number of items that appear in the view as much as possible. See bug 721402

The other issue was that the miner would consume a lot of CPU and I/O bandwidth when mining a big online storage. Throttling has helped (bug 722278), but I think we can do better. One option is to try to query the service in smaller batches instead of asking for everything and then inserting all the metadata one shot. The Google and libgdata APIs make it very easy to do this, and it is worth exploring if this has a positive effect or not.
Comment 1 Debarshi Ray 2015-04-14 11:12:43 UTC 
From bug 741472 :

I have heard reports that when a Drive has lots of items in it, some of them do not show up in gnome-documents. The problem is that when we try to get the entire contents of the Drive in one shot, we exceed the maximum items the server can return in a single query/feed.

We should use pagination, instead.
Comment 2 Debarshi Ray 2015-04-14 11:16:58 UTC 
*** Bug 741472 has been marked as a duplicate of this bug. ***
Comment 3 Debarshi Ray 2016-08-31 16:32:12 UTC 

*** This bug has been marked as a duplicate of bug 770656 ***