After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 679835 - gvariant format string parsing (and assertions) are expensive for high-performance applications
gvariant format string parsing (and assertions) are expensive for high-perfor...
Status: RESOLVED FIXED
Product: glib
Classification: Platform
Component: general
unspecified
Other Linux
: Normal normal
: ---
Assigned To: gtkdev
gtkdev
Depends on:
Blocks:
 
 
Reported: 2012-07-13 03:13 UTC by Colin Walters
Modified: 2012-08-27 20:53 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
perf report (387.72 KB, text/plain)
2012-07-13 11:45 UTC, Colin Walters
Details

Description Colin Walters 2012-07-13 03:13:49 UTC
So for better or worse (mostly better, but this bug is about the worse), I chose to use GVariant for implementing a userspace filesystem.  For example:

http://git.gnome.org/browse/ostree/tree/src/libostree/ostree-core.h?id=95921bcbf0a6b710dedd107fbacb41090af74ed9#n47

It turns out that the slowest part of my program (currently, after I've fixed various other things), is g_variant_type_string_scan().  Which in turn is called mostly from g_variant_get().  Which in turn is mostly hit when I do this:

http://git.gnome.org/browse/ostree/tree/src/libostree/ostree-core.c?id=95921bcbf0a6b710dedd107fbacb41090af74ed9#n1011

That code gets run *a lot*, because, well, I have a lot of files.  Now, I could probably avoid turning my file objects into GFileInfo, but if I want to use GVariant, I'm not aware of a faster way to parse them.

I think what I'd like is something like:

GVariantReader *g_variant_reader_new ("(uuuu&s@a(ayay))");
void g_variant_reader_get (reader, variant, &uid, &gid, &mode, &rdev,
                 &symlink_target, &ret_xattrs);

Possibly even nicer would be a way to build a relationship between C structures and certain GVariant formats.  For example:

typedef struct {
  guint32 uid;
  guint32 gid;
  ...
  GVariant *xattrs;
} MyFileObject;

MyFileObject mystruct;

g_variant_reader_new ("(uuuu&s@a(ayay))", G_VARIANT_READER_STRUCTURE_BASIC);
g_variant_reader_get (reader, variant, &mystruct);
Comment 1 Colin Walters 2012-07-13 11:45:14 UTC
Created attachment 218703 [details]
perf report
Comment 2 Allison Karlitskaya (desrt) 2012-07-15 02:55:01 UTC
It might be faster to use a GVariantIter or g_variant_get_child_value() followed by direct access with the appropriate getter -- you get to avoid format strings that way.

See bug 544026 for some possibly interesting ideas with regards to fixing this issue in GLib...