GNOME Bugzilla – Bug 142428
Thumbnailing certain TIF files eats all RAM
Last modified: 2011-12-16 23:21:28 UTC
Description of Problem: Navigating to a folder containing certain TIF image files causes Nautilus to grind and eat hundreds of megs of RAM while it attempts to create a thumbnail. The TIF format can produce small files i.e. 200k, that make Nautilus use 600 or 700 megabytes of RAM. The result is that the machine swaps like mad, rendering the desktop useless until a terminal can be opened and killall nautilus executed. I notice that Eye of Gnome also uses hundreds of megs when opening these TIF files; so the problem is likely in some library that Nautilus is accessing. Turning off previewing alleviates the problem, but is not really a solution. Steps to reproduce the problem: 1. Navigate to a folder containing TIF files with very large bitmaps. One such TIF is at http://www.trigby.com/sfstreets.tif and I will try and attach that file to this bugreport. Actual Results: Watch your mem usage go thru the roof. Be sure to have a cmdwindow open so you can issue killall nautilus. Expected Results: Nautilus should either skip previewing this file, or be much more efficient at creating the thumbnail so the entire desktop doesn't grind/swap to a halt. How often does this happen? Every time Additional Information: Sample TIF file is at http://www.trigby.com/sfstreets.tif
Created attachment 27651 [details] TIF image file that causes trouble
still here with 2.8
not a nautilus bug but probably a gtk+ or a libtiff one. It happens with nautilus but also when try to load the file with gdk_pixbuf in a small test program. Top of the stack when it hangs: (gdb) bt
+ Trace 51162
This is a libtiff problem, non-GTK apps like "convert" suffer of this too.
It is probably "just" the fact that tiff is sometimes used to store bw images with large dimensions. If a 2000x2000 bw image is stored as a bitmap, it costs ~ 4M on disk, but when it is loaded into a pixbuf which uses 32 bit/pixel, the size is blown up by a factor of 32, resulting in 128M. Compression makes this ratio even worse. The example image attached above is 11248x14484. Handling images of this size requires a more sophisticated approach to the handling of image memory than gdk-pixbufs flat 32bpp approach. gimp's tiling approach should be able to handle images of this size, I wonder how well the gimp tiff loader handles the job of converting tiffs of this size into tiles...
GIMP seems to take about 80Mb of RAM to open this test file on my machine. Firefox uses about 60Mb to do so.
11248x14484 = 162,916,032 pixels. Even if an application could display this using only one bit per pixel (since it is a bitonal image), it would take 20 Mb to display. An application such as vips (http://www.vips.ecs.soton.ac.uk) which is designed to work with very large images handles this image just fine. Applications that read the image a line at a time and don't load the whole image into memory also work fine. For example: tifftopnm sfstreets.tif| pnmscale .1 | pnmtotiff > scaled.tif works without using up much RAM at all, and the resluting image is easily viewed. In other words, this isn't a bug in any library -- it's just a question of the image being very large, as a previous poster already pointed out. (These examples just show cases of applications that are able to handle the image fine and use the same libraries.)
In other words, the application in question here should have a more efficient way of creating thumbnails -- it should read the image and scale it as it reads rather than reading the image and then scaling it. (I may be off base here -- I became aware of this bug only because someone posted a reference to it in a bug against the debian libtiff package. I don't really know anything about what Nautilus does, so please accept my apologies if I'm out of line ;-])
Would need a rewrite of the tiff loader to use more of the libtiff api, instead of just using the RGBAImage api.
Matthias, Why re-invent the wheel? Can't we just define: /desktop/gnome/thumbnailers/image@tiff to be something like: tifftopnm -byrow %i | pnmscale -width 128 -height 128 > %o If it doesn't work, it falls back to the existing gdk-pixbuf implementation. - Mike
Making the gdk-pixbuf loader handle huge tiffs better would benefit everybody, not just the thumbnailers. But sure, if that workaround works, then go for it.
Some follow-up: the tifftopnm / pnmscale approach mentioned in comment 10 is great for saving memory, but the tiff orientation tag is ignored, which is very bad. The thumbnail may be oriented differently than the main image... so I don't recommend using that approach anywhere. gThumb used the tifftopnm / pnmscale approach for a while (svn revs 1488-1648), but I've reverted to the standard gdk-pixbuf loader for tiff thumbnailing. - Mike
I'm seeing this in Fedora 7 and 8. Many businesses have thousands of tiff scans and I'm finding that performance viewing directories containing tiff files is absolutely atrocious, even with low numbers of tiff files, eg less than 200. It sits there for 30 seconds until it shown anything (list view) and once it has shown something, the scrollbar performance is intolerable. In my view, this bug should be seriously bumped up to in priority!!
Perhaps a problem could be that with multi-page tiff files, the thumbnailing process is looking at the ENTIRE image, not just the first page??? Just a guess.
There are completely two separate issues here. The tiff loader currently loads the ENTIRE tiff image into a pixbuf, even if you only want a 128x128 thumbnail. The gdk-pixbuf scaling routines then scale down the pixbuf. The problems are: 1) The tiff loader should not load the whole image. For certain tiff files (the common ones), it is possible to read scanlines in one-at-a-time. We could use the libtiff scanline interface to skip every N scanlines, to implement a prescaling step. This would save a lot of time and memory. See the discussion here, for example: http://www.asmail.be/msg0054977953.html Something similar could probably be done for tiled images (which don't use scanlines). We would need to fall back to the whole image for certain types of tiff files, however. 2) The pixbuf scaling routines collapse at high-scaling ratios. They really, really, need to be fixed. See bug 80925. Please, would a linear algebra savant take a look at that? This also applies to the png loader. The jpeg loader is slightly less susceptible, because it implements some prescaling (by 2, 4, or 8 I think). Bumping version, priority, and severity. This is a performance and a security issue. It's not hard to use this as an exploit (i.e., tiff or png in an evolution email). Adding keywords. - Mike
*** Bug 521721 has been marked as a duplicate of this bug. ***
*** This bug has been marked as a duplicate of bug 666391 ***