GNOME Bugzilla – Bug 78055
gdk-pixbuf small reads?
Last modified: 2017-06-21 18:14:12 UTC
In the thread starting at http://lists.eazel.com/pipermail/nautilus-list/2001-August/004978.html it was pointed out that gdk-pixbuf loaders seem to be obsessed with small (4k in fact) reads which kill performance when done over eg NFS. Looking closer, the ico, bmp, ras, wbmp and pnm loaders seem to implement non-incremental loading by feeding 4k chunks to the incremental loader. I don't quite see where the jpeg loader is doing 4k reads unless libjpeg does them, but it also sends chunks of at most 4k down to libjpeg in the incremental case. I would like to propose that the fake-non-incremental-loading should be removed from the individual loaders and gdk-pixbuf-loader should have a default implementation for load which would just feed the whole file to the incremental loader, probably in larger chunks (64k have been proposed in the thread above).
Looking at the libjpeg source, its default stdio source manager does indeed also do the 4k fread thing, and the buffer size is hardwired at compile time. Thus to fix this for jpeg will require to create our own stdio source manager (which is easy). I also discovered that it will not be so easy to create a default load implementation, since there is no `this` pointer in the load signature, thus the default implementation would have no way to get at the progressive loading functions which it needs to fake a non-incremental load. We could still tread module->load == NULL as a special case in gdk_pixbuf_loader_write and do the default implementation there.
For an implementation of the generic nonincremental loader, see the patch attached to #71266. With that patch, there is a single #define for cranking up the read buffer size.
- the loader interfaces are completely private currently can add this pointers if we want. - It seems that in the original thread, the 4k reads were probably a red herring. (After all, they are the default jpeg size.) It might be interesting to do some testing, but 4k reads shouldn't cause things to be dramatically slow.
The generic_image_load is in now, unfortunately I don't have the necessary setup at home to do any real performance tests with different buffer sizes.
Not worth keeping this open, really.
*** Bug 784054 has been marked as a duplicate of this bug. ***