GNOME Bugzilla – Bug 71968
Inserting rows into a TreeModel involves a lot of overhead
Last modified: 2011-02-04 16:11:56 UTC
I've begun porting Pan to gtk2 and am having trouble with CPU abuse from the TreeView. In the newsgroup list of Pan, the user can select "all" to populate the list with all the newsgroups on a server, for filtering. In gtk+ 1.2 using a clist, inserting 30,000 groups takes 5 seconds. In gtk+ 1.3.14 using a treeview, it takes 5 minutes, 20 seconds, pegs the CPU, and I can actually watch rows being inserted into the view in what looks like a handful at a time. From stack trace snapshots it appears that the cycles are being taken up in computing the treeview's size requisition: gtk_size_group_compute_requisition -> gtk_cell_renderer_get_size -> pango* A couple of these snapshots are attached. My apologies in advance if this is a user error. :)
Created attachment 6783 [details] pstack of TextView laughing at my CPU
Created attachment 6784 [details] pstack of TextView laughing at my CPU
Created attachment 6785 [details] one more pstack, just in case it's helpful
A few questions: 1) Which version of GTK are you using? 2) Which model are you using, and how are you inserting them? 3) What sizing mode are you setting the column to?
* gtk 1.3.14 * I've tried to remove outside code and reduced it to this: store = gtk_list_store_new (6, G_TYPE_POINTER, G_TYPE_BOOLEAN, G_TYPE_STRING, G_TYPE_INT, G_TYPE_INT, G_TYPE_STRING); for (i=0; i<qty; ++i) { const Group * group = g[i]; GtkTreeIter iter; (void) memset (&iter, 0, sizeof(GtkTreeIter)); gtk_list_store_insert (store, &iter, 0); gtk_list_store_set (store, &iter, 0, group, 1, group_is_subscribed(group), 2, group_get_readable_name(group), 3, group->article_read_qty, 4, group->article_qty, 5, group->description, -1); } gtk_tree_view_set_model (GTK_TREE_VIEW(Pan.group_tree), GTK_TREE_MODEL(store)); * All the columns are set with gtk_tree_view_column_set_sizing (column, GTK_TREE_VIEW_COLUMN_FIXED); to turn off the 'find best width' calculations. Is there a way to insert batches of rows (or subtrees) that I should be using?
Created attachment 6815 [details] times how long it takes to populate & render a 30,000-item clist
Created attachment 6816 [details] time-treeview.c: times how long it takes to populate & render a 30,000-item treeview
The two attachments, time-clist.c and time-treeview.c, are test cases to time how long it takes to populate & render a 30,000-item clist and treeview, respectively. results on my meager p450, which was otherwise idle: (18:34:12)(charles ip68-12-64-81)(~/tmp): ./time-clist Message: clist inserted 30000 rows in 0.8 seconds (39253 rows/sec) (18:34:17)(charles ip68-12-64-81)(~/tmp): ./time-clist Message: clist inserted 30000 rows in 0.8 seconds (37153 rows/sec) (18:34:20)(charles ip68-12-64-81)(~/tmp): ./time-clist Message: clist inserted 30000 rows in 0.8 seconds (39436 rows/sec) (18:34:44)(charles ip68-12-64-81)(~/tmp): ./time-treeview Message: added 30000 rows in 35.2 seconds (851 rows/sec) (18:36:09)(charles ip68-12-64-81)(~/tmp): ./time-treeview Message: added 30000 rows in 36.2 seconds (829 rows/sec) (18:37:31)(charles ip68-12-64-81)(~/tmp): ./time-treeview Message: added 30000 rows in 34.6 seconds (866 rows/sec)
Hmm, my machine gives me for time-treeview: Message: added 30000 rows in 8.9 seconds (3372 rows/sec) If I comment out the call to gtk_list_store_set, the treeview window shows up a _lot_ faster. I think gtk_list_store_set is the real bottleneck here. Though it still takes a while before the treeview is completely populated -- that's the incremental reflow at work. Maybe it's an idea to be able to 'throttle' the reflow, ie speeding it up when a lot of rows are 'waiting' to be added. Maybe completely populating the model before showing the treeview isn't the right way for big quantities of rows. I played around with a 'row_add_handler' (timeout thing). This way the treeview shows up at once, but it takes a while before the model and view are completely populated. I think we definitely need to solve this soon. Setting milestone to 2.0.0 for now, priority high.
> Hmm, my machine gives me for time-treeview: > Message: added 30000 rows in 8.9 seconds (3372 rows/sec) Well... I'm on a p450 here. ;) > If I comment out the call to gtk_list_store_set, the > treeview window shows up a _lot_ faster. I think > gtk_list_store_set is the real bottleneck here. It's part of the bottleneck because of the linked lists in list-store. So I built a new model in Pan based on a GPtrArray but the reflow problem is still severe. > Though it still takes a while before the treeview is > completely populated -- that's the incremental reflow > at work. Maybe it's an idea to be able to 'throttle' > the reflow, ie speeding it up when a lot of rows are > 'waiting' to be added. Attached find a time-treeview2.c, which times the store insert time and the reflow time separately. I've changed the list_store code to prepend, rather than append, to omit unnecessary list traversal. Again on my p450: Message: gtk_list_store appended 30000 items in 4.9 secs (6168 rows/sec) Message: 37.9 seconds until idle The latter is the time between gtk_main() and the idle func being called. > Maybe completely populating the model before showing the treeview > isn't the right way for big quantities of rows. I played > around with a 'row_add_handler' (timeout thing). This way the > treeview shows up at once, but it takes a while before the model > and view are completely populated. I hope this isn't the solution you pursue. This doesn't fix the problem; it just staggers it out over a longer period of time, and makes programmers jump through hoops in order to use treeview. The CPU will still be spiked, and users will still have to wait before scrolling down to 'z'. I haven't read the treeview code, so these may be wildly wrong: 1) Is there a way I can set the height of a row, such that recalculating the height is just (noncollapsed_rows * row_height)? 2) Is there a way to insert a batch of rows, rather than inserting them one-by-one, so that the model only needs to fire one event instead of 30,000?
Created attachment 6826 [details] time-treeview2.c: times the list_store insertion & reflow times separately
I think I might've been unclear on that last question... what I'm asking about is something like swing's fireTreeStructureChanged where you can notify listeners of changes to an entire subtree with one event. IIRC this was done in Swing instead of fire-for-each-change because of the performance hit of the latter.
No time for 2.0.1. One of the big goals for 2.0.1 will be to optimize the treeview.
Moving remaining 2.0.1 bugs to 2.0.2 milestone.
It now guesses the height of the tree, so even if it's not done measuring the height, it looks like it is to the user. Can you tell me if this works better for you?
Reassigning bugs to new component owner. Sorry for the flood.
Sorry for going AWOL on this bug report. Somehow I missed it getting NEEDINFOed. As of 2.0.6 the gtk2 widget is still 20x slower than gtk1's widget. Using the same two test apps as listed above: % pkg-config --modversion gtk+-2.0 2.0.6 % ./time-clist ** Message: clist inserted 30000 rows in 1.3 seconds (22722 rows/sec) % ./time-treeview ** Message: added 30000 rows in 26.4 seconds (1138 rows/sec)
We are still working on it. I have a couple of ideas/patches myself. Though we are trying to think about more ideas to shave more seconds off. But keep in mind that it is absolutely not possible to get treeview as fast as clist.
Moving remaining bugs to 2.2.1.
*** This bug has been marked as a duplicate of 80868 ***