GNOME Bugzilla – Bug 526936
gda_connection_execute_select_command() slow even when using ITER_MODEL_ONLY
Last modified: 2008-11-17 10:16:29 UTC
I'm using a copy of the musicbrainz database on my PostgreSQL server on localhost, just to test this. I'm using the "artist" table, which has almost 600,000 records. As the attached libgda_test_bigtable.c test case shows, it takes about 1 minute (on my 3Mhz PC with 1.5Gb RAM) to return from gda_connection_execute_select_command() and uses about 300Mb of memory (as seen in System Monitor). postgres_test_bigtable.c shows that the equivalent (I think) with libpq takes no more than 10 seconds and uses about 35Mb of memory. That's not great but it's much better than libgda.
Created attachment 108855 [details] libgda_test_bigtable.c
Created attachment 108856 [details] postgres_test_bigtable.c
Created attachment 108887 [details] libgda4_test_importtest.c The same thing with libgda-4.0. I don't know what you did, but both speed and memory seem not much more than when using libpq directly. Well don.
This is strange... In the last attachment, the returned data model is a random access one: the ITER_MODEL_ONLY option does not exist anymore in V4 (this should be in the doc about migration, I'll mention it), you need to use gda_connection_statement_execute_select_full() and pass GDA_STATEMENT_MODEL_CURSOR as the model_usage argument to have a cusron based data model. The params argument of gda_connection_statement_execute_select() is not needed in your case as the statement does not have any variable. So in that last attachment, libpq transfers all the data from the postgres server (it is all stored in memory), but compared to V3, the data is not converted to GValue values until actually requested. This is why it is so close to what you'd get with libq. Reading the contents of the data model will make memory usage increase.
Created attachment 108969 [details] libgda4_test_bigtable_gettingvalues.c It's wonderfully fast when I get values too, and when I correctly request a cursor/iter-based model.
Closing because people should now just use libgda-4.0 instead.