GNOME Bugzilla – Bug 311932
Crash when opening very large files
Last modified: 2009-02-10 17:30:10 UTC
Version details: win gimp Distribution/Version: xp - sp2 - up to date Simply wanted to open satelite maps of the world. The Jpeg or Tif files weight beetween 200 MB and 400 MB. Open Gimp, then choose one of the files. (see - warning, 240 MB file- http://visibleearth.nasa.gov/images/2433/land_shallow_topo_west.tif or -warning 400 MB fle- http://visibleearth.nasa.gov/images/2433/land_shallow_topo_east.tif) I expected the images to show. But Gimp will crash every time, saying that it coudn't allocate enough memory. My system has 1GB Memory, 4GB swap for windows, and the Gimp paths are directed toward a Hardrive that has over 60 GB free ; photoshop succeeds in opening these files so my system is not overloaded...
How much memory was it trying to allocate in total?
It said it couldn't allocate 12000 Bytes or so (I'll try this evening when I come back from office). But I tried to reboot, free memory and so on, it still failed to allocate some memory on opening the image(s). In fact, it didn't fill windows's swap as in task manager I had a swap filled with 1.7 GB.... so there was still some memory to use (even if it was swap, since I had 4GB swap and 1GB Ram)
ok, tried again to open the 240 MB tiff : windows's swap defined : 3000MB < swap < 4000MB total RAM : 1 GB (didn't move much ^^) According to task manager => windows total mem used : 2433056 Mb (i.e. about 2400 MB) windows total max mem used from session start : 2439244 (i.e. about 2400 MB) windows max mem : 4022336Mb (i.e. about 4000 MB) watching gimp's memory use, it grows up to 900 MB then falls to 3MB where it finaly displays this error message : Glib-ERROR **:gmem.c:141: failed to allocate 16384 bytes aborting... clicking on the "ok" button frees memory and closes gimp :(
re opening after giving more informations
How large are these images (in pixels)? It is very likely that you need a 64bit machine in order to handle the amount of virtual memory that is needed.
To answer my own question, the images are 21600 x 21600 pixels RGB. That makes up for 466,5 million pixels. GIMP will need 4GB of tile memory to load this file. Should actually still work on a 32bit machine but I'd say this is not worth trying on a machine that has less than 2GB of physical RAM. Sure, the handling of large images could be improved and it is a known fact that PS does this better. But given the current architecture, I'd say it is pointless to try to edit these images in GIMP unless you are willing to throw more hardware onto it. Not sure what should be best done about this bug report. It might make sense to have an enhancement request to improve handling of large images but such a request should also include a proposal on how that should be achieved.
I would suggest that you allow gimp to open with options, and then you can tell GIMP to open only a portion of the file (perhaps a small preview image where you can box the area of interest. Just a possibility
*** Bug 340656 has been marked as a duplicate of this bug. ***
Memory issues with GIMP (Win32): These are issues on a machine which is able to handle images with 10 GPixel with Photoshop. Loading of 10 GPixels takes a while (5 min), but it is possible to work with Photoshop without any aches. All drives are NTFS partitions with partially files larger than 25 GByte. Issue 1: * GIMP uses much more memory than needed. When loading 1, 4, 8, 24, 48 bit images, a performant design program should use 1, 4, 8, 24 or 48 bit per pixel, i.e. a X * Y image should use X * Y * (1/8, 1/2, 1, 3, 6) Bytes of RAM and Tile Cache. This can be increased when you start to manipulate the image. GIMP uses around 6.25 Bytes per Pixel for a 0.5 Bytes per Pixel image. This is more than a factor of 10! Issue 2: * A tile based program in a world of computer with 1...3 GByte RAM should use more than 2 GByte Tile Cache. When using 32 bit offsets, 4 GByte should be possible by using OpenFile() instead of fopen() to open the Tile Cache. Better idea is to open multiple swap files.
On reasonable operating systems fopen() can handle files larger than 2GB. Issue 2 is only relevant on Win32. We would accept patches to fix this. Issue 1 is well known but rather hard to address given the current design of GIMP.
Trying to reproduce another "a large file opened in GIMP kills my X session" issue discovered by a user (in a forum) I opened a 8bit/channel TIFF file which is 3088 X 2056px large and tried to resize it to 10000 X 6658px. Soon after I clicked "Resize" GIMP stopped responding on my Centrino 1,6 GHz/ 512 Mbyte RAM even with 2 Gbyte of tile cache set in Preferences.
I wonder if Tor's changes to fix bug #398311 are relevant for this one as well: 2007-01-26 Tor Lillqvist <tml@novell.com> Fix #398311 in GIMP until corresponding abstraction has been added to GLib: * app/base/tile-private.h: Use gint64 instead of off_t. (I assume the configury makes sure GIMP is always compiled as large-file-aware on Unix with an off_t of at least 64 bits?) Introduce wrapper macros LARGE_SEEK() and LARGE_TRUNCATE(). On Win32 LARGE_SEEK() calls _lseeki64() in the Microsoft C library, and LARGE_TRUNCATE calls a new internal function gimp_win32_large_truncate(). On Unix they call lseek() and ftruncate(). * app/base/tile-swap.c: Use gint64 instead of off_t. Use LARGE_SEEK() and LARGE_TRUNCATE() instead of lseek() and ftruncate(). (gimp_win32_large_truncate): New function. Calls LARGE_SEEK() and SetEndOfFile(). * app/xcf/xcf-load.c (xcf_swap_func): Use LARGE_SEEK() instead of lseek().
Hmm, maybe fixed? Cannot find so big picture to test it and these provided ones aren't there anymore.
The files are available from ftp://veftp.gsfc.nasa.gov/bluemarble/
I can load the land_shallow_topo_21600.tif without crashes in 2.6.4. It's not exactly fun to work on such images, of course ;) Resolving as FIXED.