GNOME Bugzilla – Bug 142428
Thumbnailing certain TIF files eats all RAM
Last modified: 2011-12-16 23:21:28 UTC
Description of Problem:
Navigating to a folder containing certain TIF
image files causes Nautilus to grind and eat
hundreds of megs of RAM while it attempts to
create a thumbnail. The TIF format can produce
small files i.e. 200k, that make Nautilus use 600
or 700 megabytes of RAM.
The result is that the machine swaps like mad,
rendering the desktop useless until a terminal can
be opened and killall nautilus executed.
I notice that Eye of Gnome also uses hundreds of
megs when opening these TIF files; so the problem
is likely in some library that Nautilus is
Turning off previewing alleviates the problem, but
is not really a solution.
Steps to reproduce the problem:
1. Navigate to a folder containing TIF files with
very large bitmaps. One such TIF is at
http://www.trigby.com/sfstreets.tif and I will try
and attach that file to this bugreport.
Watch your mem usage go thru the roof. Be sure to
have a cmdwindow open so you can issue killall
Nautilus should either skip previewing this file,
or be much more efficient at creating the
thumbnail so the entire desktop doesn't grind/swap
to a halt.
How often does this happen?
Sample TIF file is at
Created attachment 27651 [details]
TIF image file that causes trouble
still here with 2.8
not a nautilus bug but probably a gtk+ or a libtiff one. It happens with
nautilus but also when try to load the file with gdk_pixbuf in a small test program.
Top of the stack when it hangs:
This is a libtiff problem, non-GTK apps like "convert" suffer of this too.
It is probably "just" the fact that tiff is sometimes used to store bw images with
large dimensions. If a 2000x2000 bw image is stored as a bitmap, it costs ~ 4M
on disk, but when it is loaded into a pixbuf which uses 32 bit/pixel, the size
is blown up by a factor of 32, resulting in 128M. Compression makes this ratio
The example image attached above is 11248x14484. Handling images of this size
requires a more sophisticated approach to the handling of image memory than
gdk-pixbufs flat 32bpp approach. gimp's tiling approach should be able to handle
images of this size, I wonder how well the gimp tiff loader handles the job of
converting tiffs of this size into tiles...
GIMP seems to take about 80Mb of RAM to open this test file on my machine.
Firefox uses about 60Mb to do so.
11248x14484 = 162,916,032 pixels. Even if an application could display this
using only one bit per pixel (since it is a bitonal image), it would take 20 Mb
to display. An application such as vips (http://www.vips.ecs.soton.ac.uk) which
is designed to work with very large images handles this image just fine.
Applications that read the image a line at a time and don't load the whole image
into memory also work fine. For example:
tifftopnm sfstreets.tif| pnmscale .1 | pnmtotiff > scaled.tif
works without using up much RAM at all, and the resluting image is easily
viewed. In other words, this isn't a bug in any library -- it's just a question
of the image being very large, as a previous poster already pointed out. (These
examples just show cases of applications that are able to handle the image fine
and use the same libraries.)
In other words, the application in question here should have a more efficient
way of creating thumbnails -- it should read the image and scale it as it reads
rather than reading the image and then scaling it. (I may be off base here -- I
became aware of this bug only because someone posted a reference to it in a bug
against the debian libtiff package. I don't really know anything about what
Nautilus does, so please accept my apologies if I'm out of line ;-])
Would need a rewrite of the tiff loader to use more of the libtiff api, instead
of just using the RGBAImage api.
Why re-invent the wheel? Can't we just define:
to be something like:
tifftopnm -byrow %i | pnmscale -width 128 -height 128 > %o
If it doesn't work, it falls back to the existing gdk-pixbuf implementation.
Making the gdk-pixbuf loader handle huge tiffs better would benefit everybody, not just the thumbnailers. But sure, if that workaround works, then go for it.
Some follow-up: the tifftopnm / pnmscale approach mentioned in comment 10 is great for saving memory, but the tiff orientation tag is ignored, which is very bad. The thumbnail may be oriented differently than the main image... so I don't recommend using that approach anywhere.
gThumb used the tifftopnm / pnmscale approach for a while (svn revs 1488-1648), but I've reverted to the standard gdk-pixbuf loader for tiff thumbnailing.
I'm seeing this in Fedora 7 and 8.
Many businesses have thousands of tiff scans and I'm finding that performance viewing directories containing tiff files is absolutely atrocious, even with low numbers of tiff files, eg less than 200. It sits there for 30 seconds until it shown anything (list view) and once it has shown something, the scrollbar performance is intolerable.
In my view, this bug should be seriously bumped up to in priority!!
Perhaps a problem could be that with multi-page tiff files, the thumbnailing process is looking at the ENTIRE image, not just the first page??? Just a guess.
There are completely two separate issues here. The tiff loader currently loads the ENTIRE tiff image into a pixbuf, even if you only want a 128x128 thumbnail. The gdk-pixbuf scaling routines then scale down the pixbuf.
The problems are:
1) The tiff loader should not load the whole image. For certain tiff files (the common ones), it is possible to read scanlines in one-at-a-time. We could use the libtiff scanline interface to skip every N scanlines, to implement a prescaling step. This would save a lot of time and memory. See the discussion here, for example:
Something similar could probably be done for tiled images (which don't use scanlines).
We would need to fall back to the whole image for certain types of tiff files, however.
2) The pixbuf scaling routines collapse at high-scaling ratios. They really, really, need to be fixed. See bug 80925. Please, would a linear algebra savant take a look at that?
This also applies to the png loader.
The jpeg loader is slightly less susceptible, because it implements some prescaling (by 2, 4, or 8 I think).
Bumping version, priority, and severity. This is a performance and a security issue. It's not hard to use this as an exploit (i.e., tiff or png in an evolution email). Adding keywords.
*** Bug 521721 has been marked as a duplicate of this bug. ***
*** This bug has been marked as a duplicate of bug 666391 ***