After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 730211 - Extra zeroes appear in XCF files (was: after saving and closing .xcf files, they open empty - please help!)
Extra zeroes appear in XCF files (was: after saving and closing .xcf files, t...
Status: RESOLVED FIXED
Product: GIMP
Classification: Other
Component: General
2.8.10
Other All
: Urgent critical
: 2.8
Assigned To: GIMP Bugs
GIMP Bugs
: 721233 730787 744035 (view as bug list)
Depends on:
Blocks:
 
 
Reported: 2014-05-15 18:06 UTC by Shelbie
Modified: 2017-02-10 13:21 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
this didn't attach to bug report - it's 2.1MB but opens empty (536.35 KB, application/zip)
2014-05-15 18:09 UTC, Shelbie
  Details
jpg export of gimp file that is opening empty (75.41 KB, image/jpeg)
2014-05-15 19:02 UTC, Shelbie
  Details
layers recovered (376.62 KB, image/x-compressed-xcf)
2014-05-16 07:49 UTC, Massimo
  Details
proposed patch (11.28 KB, patch)
2014-05-31 09:42 UTC, Massimo
committed Details | Review
patch to load particularly damaged files (2.72 KB, patch)
2014-06-15 05:34 UTC, Massimo
committed Details | Review
Result of the simpler test procedure in comment 25 . (2.33 KB, application/octet-stream)
2014-06-17 13:00 UTC, ajmcrae
  Details
The simplest test case. (2.32 KB, application/octet-stream)
2014-06-17 13:26 UTC, ajmcrae
  Details

Description Shelbie 2014-05-15 18:06:35 UTC
I've never done anything like this, so I hope I'm reporting this right.  (I was encouraged to report this by OFNUTS in a gimp forum.)   I've been using gimp and loving it for several years (though I am self-taught and no pro, and know nothing about code and bugs). Until a couple of months ago, I had no problems - then a large file suddenly opened up with no image and the bottom of the blank canvas said "Can't work on an empty image, add a layer first," even though my computer said the file was several MB. I assumed the problem was because the file was too big - nope. I recently started creating several smaller graphics for a season poster, all reasonably-sized files, and they are all opening empty, too...is all my work lost?!? The computer still says the files have size, (2.1MB, 4MB, 8.1MB, etc.), but they open in gimp totally blank. I'm so scared to open other files I've recently worked on!  I'm using version 2.8.10 on macs (this happened on my laptop and desktop), and I've deleted gimp and reinstalled, but the same thing keeps happening. Please help! 

When I reported it to the forum, I attached one of the smaller 2.1MB files that I've also attached to this, and two people said "Looking at the file contents with a binary editor, there are more metadata, and it seems there are also layer data, but there is a large chunk filled with zeroes, and this normally doesn't happen."

Do you have any idea what it putting in all those zeros and how I can stop it?

FYI: I do usually have several layers in my files, and love to use different fonts, filters, gradients, etc (that's why I love gimp so much!), but now I feel that I can't trust it at all.  OFNUTS also suggested saving a copy and working on that, but I had already tried that and it still opened blank/empty. 

Any advice?
Comment 1 Shelbie 2014-05-15 18:09:26 UTC
Created attachment 276616 [details]
this didn't attach to bug report - it's 2.1MB but opens empty

I tried to zip it so I could attach it.  Otherwise, I don't know how I can send it, though you can find it with my post at the help forum: http://gimpforums.com/thread-empty-files
Comment 2 Ed J 2014-05-15 18:36:59 UTC
Opens empty on my Win 7 GIMP 2.8.10 too. Sorry, nothing else to add.
Comment 3 Ed J 2014-05-15 18:38:10 UTC
Can you give us steps to reproduce this from scratch, with a new file?
Comment 4 Shelbie 2014-05-15 19:02:46 UTC
Created attachment 276621 [details]
jpg export of gimp file that is opening empty
Comment 5 Michael Schumacher 2014-05-15 20:15:08 UTC
Maybe the same as bug 721233.
Comment 6 Massimo 2014-05-16 07:49:51 UTC
Created attachment 276654 [details]
layers recovered

Unfortunately I don't know what's causing it.

Conflicts with osx automatic versioning of files?
Encripted disks? Closing the GUI while a save is in progress
thinking to hide the main window and instead terminating GIMP?

The attached xcf.bz2 is what I recovered using
the following program + vim and xxd.

#include <glib.h>
#include <stdio.h>
#include <stdint.h>

#define BUFSIZE 4 * 1024 * 1024
uint8_t buffer[BUFSIZE + 8];

int32_t
to_int32 (const uint8_t *buf)
{
  return (((((buf[0] << 8) | buf[1]) << 8) | buf[2]) << 8) | buf[3];
}

static void
filter (const uint8_t *buffer, uint32_t offset)
{
  int32_t width    = to_int32 (buffer);
  int32_t height   = to_int32 (buffer + 4);
  int32_t type     = to_int32 (buffer + 8);
  int32_t len      = to_int32 (buffer + 12);
  const char *name = (const gchar*) buffer + 16;

  if (0 <= type && type < 6 && 0 < width && width < 100000 &&
      0 < height && height < 100000 && 0 < len && len < 16*1024*1024 &&
      name[0] != '\0' && g_utf8_validate (name, len - 1, NULL))
    g_print ("offset: %08x width: %d height: %d type: %d name: '%s'\n",
             offset, width, height, type, name);
}

int main (int argc, char *argv[])
{
  if (argc == 2)
    {
      FILE *fp = fopen (argv[1], "rb");
      if (fp)
        {
          uint32_t j = 0;
          int r = fread (buffer, 1, BUFSIZE, fp), i;

          for (i = 0; i < r; ++i, ++j)
            filter (&buffer[i], j);
          fclose (fp);
        }
    }
  return 0;
}
Comment 7 Shelbie 2014-05-16 13:29:16 UTC
Wow - you are amazing!  I'm grateful to not have to recreate at least one of my files.  Thank you:)  If anything ever comes up where y'all figure out what I can do to avoid this happening in the future, please let me know.  I'm just so upset about potentially having to recreate all my graphics on a regular basis, but my family has a joke about being cursed: we manage to have mysterious crap like this happen all the time.  :/  Thanks again for all your time and help!
~S
Comment 8 Massimo 2014-05-16 14:06:42 UTC
Well, it seems that all the data is there, after writing the image grid
parasite GIMP has to write the layers and it seems that it jumps
to the wrong place leaving a gap filled with zeros. When reading,
those zeros are misinterpreted as no layer and no channel.

Are you using the official version of GIMP or perhaps that
from lisanet or partha or something else?

Unfortunately on linux this does not happen and I can't
see what could be wrong.

when building GIMP's Mac version are there compiler (clang?)
warnings related to xcf-save.c or xcf-write.c that could help
fix this issue?
Comment 9 Massimo 2014-05-17 04:46:32 UTC
Judgind from:

> https://www.securecoding.cert.org/confluence/display/seccode/FIO19-C.+Do+not+use+fseek%28%29+and+ftell%28%29+to+compute+the+size+of+a+regular+file

In addition, footnote 268 of subclause 7.21.3 says:

    Setting the file position indicator to end-of-file, 
    as with fseek(file, 0, SEEK_END), has undefined behavior for a 
    binary stream (because of possible trailing null characters) ....

and 

https://git.gnome.org/browse/gimp/tree/app/xcf/xcf.c?h=gimp-2-8#n250

https://git.gnome.org/browse/gimp/tree/app/xcf/xcf-seek.c?h=gimp-2-8#n57

GIMP (gimp-2-8) code is invoking undefined behavior.
Comment 10 Michael Natterer 2014-05-17 07:32:05 UTC
That sounds like OSX is not behaving like a POSIX system, are
we sure that this is the cause of the problem? It surely sounds
very likely tho.
Comment 11 Massimo 2014-05-19 04:44:24 UTC
(In reply to comment #10)
> That sounds like OSX is not behaving like a POSIX system, are
> we sure that this is the cause of the problem? It surely sounds
> very likely tho.

I'm not sure. Nonetheless I think GIMP should be tolerant in
what it loads and strict in what it saves. So if it finds that
after the grid parasite there are few zeros before the PROP_END
marker it should skip them and it should not call

fseek(... SEEK_END);
Comment 12 Michael Schumacher 2014-05-27 09:04:21 UTC
*** Bug 730787 has been marked as a duplicate of this bug. ***
Comment 13 Michael Schumacher 2014-05-27 14:28:06 UTC
*** Bug 721233 has been marked as a duplicate of this bug. ***
Comment 14 Massimo 2014-05-31 09:42:12 UTC
Created attachment 277612 [details] [review]
proposed patch

This is a patch that avoids seeking to the end of file, when saving.
    
Each function after which there was an xcf_seek_end call now
leaves the info->cp at the last byte written so the caller
does not need to seek.

It also writes the layer/channel/mask/tile offset before
writing the layer/channel/mask/tile, so if a power outage
interrupts the save when all but one tiles are written
it should be possible to load all but one tile.

It is an important part of the code so it requires extensive
review and testing.
Comment 15 Max Mustermann 2014-06-14 16:10:17 UTC
I've integrated your patch into the osx-build branch as basis for testing and improving.
GIMP can be built with the patch and a simple test to save and load "layers recovered.xcf" passes. 
Needs further review and testing as proposed by Massimo in comment 14.
Comment 16 ajmcrae 2014-06-14 19:43:57 UTC
Just responding to some comments made so far...
This bug is definitely not specific to Mac. In previous years I was using the version of GIMP that was in Ubuntu 11.04 which I think was 2.6. I have recently switched to Linux Mint 15 which distributes GIMP 2.8. I have now found (to my horror) that some XCF files saved in 2.6 show as empty with no layers when opened in 2.8. It's only some files though, as some multilayer XCF files will open okay.

I do not think this was anything to do with GIMP crashing or being closed abnormally as I am sure I would have noticed this in the 3 years that I was using GIMP 2.6. I have never seen this error occur before.

The files of mine which don't show any layers also have a heap of zeros in them just after the GIMP header and before a bit of nonzero (picture??) data. I wondered if this was due to the GIMP 2.6 writing a block length equal to the uncompressed image size, but the math doesn't quite add up to the image dimensions. In one image it is 31kB of zeros, in others it's 530kB or 962kB, no consistent relation to the image data size (W*H*bpp).

I have not re-tested whether GIMP 2.6 can open these files. If I can build a VM with Ubuntu 11 and GIMP 2.6 in the next few days I will try that out.
Comment 17 Massimo 2014-06-15 05:34:36 UTC
Created attachment 278467 [details] [review]
patch to load particularly damaged files

This is a patch to possibly load files that have unexpected zeros after
the grid parasite.

It adds code that is only executed if the normal way does not find any
layer nor channel. So the current successful execution is basically
unaffected.

It is based on gimp-2-8, but does not use directly the C function fgetc
etc, so it is possible to use it in master that has been already ported
to GIO.

Don't know if it makes sense to include it in GIMP, but could be useful
to recover damaged files.
Comment 18 Max Mustermann 2014-06-16 18:05:10 UTC
Just some short notes in between: 

1.) I could not reproduce it with GIMP 2.8.10, neither on OS X 10.9.3 (Clayton's build, 64 bit), Windows 7 (32 bit) nor Linux (32 bit). 
Attempt 1:
To reproduce it I opened Massimo's attachment 'layers recovered' and saved it.

Attempt 2:
I also created a new image: 1600x1200 px, RGB, two layers: bottom layer plain white, top layer ca. 60% opacity, transparent, with some scribbles.

Both images were also saved to SMB shares like reported in bug 730787 and could then be opened without loss.

Are there other steps to reproduce this error reliably?

2.) Ubuntu 11.04 ships with GIMP 2.6.11, released on 03.10.2010. No changes have been made to the whole /app/xcf directory since 29.05.2009, i.e. the xcf code is the same in versions 2.6.11 and 2.8.10. Therefore I'm a bit surprised about comment 16 (however, thank you for trying to shed some light on the issue).
If the bug is really platform independent it is probably better fixed in a separate branch than osx-build.

3.) The XCF format specification was created in 2006 and is still in draft (sic!) state (see https://git.gnome.org/browse/gimp/log/devel-docs/xcf.txt?h=gimp-2-8), which makes it less usable to solve XCF related bugs like this. What of this document is mandatory now? Can we remove the draft remarks or solve issues in question?
Comment 19 Max Mustermann 2014-06-16 18:07:04 UTC
Just some short notes in between: 

1.) I could not reproduce it with GIMP 2.8.10, neither on OS X 10.9.3 (Clayton's build, 64 bit), Windows 7 (32 bit) nor Linux (32 bit). 
Attempt 1:
To reproduce it I opened Massimo's attachment 'layers recovered' and saved it.

Attempt 2:
I also created a new image: 1600x1200 px, RGB, two layers: bottom layer plain white, top layer ca. 60% opacity, transparent, with some scribbles.

Both images were also saved to SMB shares like reported in bug 730787 and could then be opened without loss.

Are there other steps to reproduce this error reliably?

2.) Ubuntu 11.04 ships with GIMP 2.6.11, released on 03.10.2010. No changes have been made to the whole /app/xcf directory since 29.05.2009, i.e. the xcf code is the same in versions 2.6.11 and 2.8.10. Therefore I'm a bit surprised about comment 16 (however, thank you for trying to shed some light on the issue).
If the bug is really platform independent it is probably better fixed in a separate branch than osx-build.

3.) The XCF format specification was created in 2006 and is still in draft (sic!) state (see https://git.gnome.org/browse/gimp/log/devel-docs/xcf.txt?h=gimp-2-8), which makes it less usable to solve XCF related bugs like this. What of this document is mandatory now? Can we remove the draft remarks or solve issues in question?
Comment 20 Max Mustermann 2014-06-16 18:07:39 UTC
Sorry for double posting, Bugzilla had issues.
Comment 21 Max Mustermann 2014-06-16 19:38:22 UTC
Shelbie, can you please tell us which version of OS X you are running?
You find this information by clicking on the Apple icon in the menu bar and then choosing 'About this Mac'. Thank you in advance.
Comment 22 Daniel Savi 2014-06-16 20:03:43 UTC
(In reply to comment #21)
> Shelbie, can you please tell us which version of OS X you are running?

I was reporting bug 730787. It was the most recent OS X (is it called mavericks?) with all updates installed and the current GIMP for OS X. I've got the same troubles as you have Sven to reporduce the bug. So far, I could not create a file with the bug on purpose, but had it twice. Layers have been transparent and with alpha channel in both cases. I do also remember, that I've created and deleted at least one additional layer in both images before the bug happened. Maybe, that could be another condition to be tested?
Comment 23 Daniel Savi 2014-06-17 06:40:54 UTC
Exact Versions:
OS X 10.9.3
Gimp 2.8.10
Comment 24 ajmcrae 2014-06-17 12:07:17 UTC
If comment 16 was merely surprising then prepare to be shocked. :)
I have just seen the bug occur in GIMP 2.8.4 on my very first attempt to reproduce the bug. This was running under Linux Mint 15 cinnamon (which is based on Ubuntu 13.04). The NAS is a QNAP brand TS-209II from 2008.

The steps were:
----
1. Mount a public SMB share with write access (eg using Nemo / Nautilus).

2. Create a symlink in user's home directory to the share under the GVFS mount point so that drag'n'drop of files works in the file manager, eg a folder called NP links to a share called Public: 
<code> ln -s /run/user/<USER>/gvfs/smb-share:server=nas1,share=public NP </code>

3. Start GIMP 2.8.4 . Create new 1600*1200 RGB image. Leave background plain white. Create new layer on top, same size, transparent. Scribble black on the top layer.

4. Save the XCF to the SMB share via the user's symlink. The bug is not yet triggered. Close GIMP.

5. Restart GIMP, open the previously saved file from the share via the symlink.

6. With the "Background" (bottom) layer selected, Add a new layer, smaller size than the image (eg 64*64), transparent, and it becomes the middle layer ("Layer #1" by default). Scribble some other colour on the new middle layer.

7. Save the XCF to the share. The bug is now triggered. Close the GIMP.

Verification:
8. Open the XCF in a hex viewer, eg GHex, and look at offset 0x199, which will be the start of a whole heap of nulls, 225kB of zeros in my example.

9. Try opening the XCF again in GIMP 2.8.4 and there will be no layers in the image, plus the message in the status bar "Can't work on an empty image, add a layer first."

----
I have tried the above procedure twice, the bug occurred both times, so it seems reproducible... on my system at least.

Trying the above method but saving to a local directory on the PC (eg ~/Pictures) did NOT produce the error, so it is probably related only to saving on volumes that are not directly supported by the main filesystem.

In fact, trying to save from the GIMP onto the SMB share directly, by picking the share name in the "Places" list of the Save dialog, fails with the message "Could not open 'smb://nas1/public/Technology/Electronics/Computers/vanishinglayers4.xcf' for writing: No such file or directory". 
Instead you must browse to the gvfs emulation of the file tree by going to /run/<user>/gvfs/<smb share>/etcetera and saving to that location.
In that case, skipping the symlink and saving to the gvfs mount, the bug is still triggered, and in this test run it inserted 449kB of zeros instead of 225kB and 244kB in the previous tests.

[ Side note: Seems GNU/Linux for a long time doesn't support any equivalent of UNC on MS Windows, so a valid path for a SMB location must be given to an SMB-specific filesystem interface, the root filesystem will not resolve it. Thus the reason for the hack workaround that I have been using for the last couple of years under linux by symlinking the share into my home directory (via gvfs in step 2). Linux is such a mess that even the location of the gvfs path is different between Ubuntu 11 and Mint 15, it used to be under the user's home directory (~/.gvfs/), but now it is (more usefully) under /run/<user>/gvfs/. ]

The bug really looks like a problem with saving to SMB shares, and perhaps shares mounted by GVFS in particular. 

As you say, the XCF code in GIMP hasn't changed since 2009, so the bug has been there at least that long. I wonder why we haven't seen this bug reported earlier. It's not my NAS because I haven't seen this type of error in other applications saving to it, like LibreOffice or just copying files.

I can only guess that most people who use GIMP store all their files on the PC, not on a NAS SMB mount. People who used a NAS for more reliability and backups are more likely to be part of an organisation that would pay for Photoshop (instead of GIMP). This bug may be occurring more often now that NASes have become so cheap and easy to set up that more people are using them at home. The consumer versions of most NASes only support SMB or FTP (eg in the QNAP range you have to pay extra for a Pro version that supports DFS mounts for linux) so saving to a "Windows share" is going to be the popular configuration.
Comment 25 ajmcrae 2014-06-17 12:45:58 UTC
Further to the above, I have found the test can be made much simpler.

Leaving the background all white is not necessary, the bug will also occur with some scribble added on the Background. 
Putting scribble on the top layer is not necessary, the bug will also occur with the new layer empty.
Having three layers is not necessary, the bug occurs with only two.
Using a large image size is not necessary, the bug occurs at the default 640*480.
Most surprising: Closing GIMP before adding the final layer is not necessary, the bug occurs if the file is resaved in the same instance of GIMP that saved the single layer file.

Simpler test for 730211:
----
1. Mount and symlink SMB share as previously described. This is done once and the mount is reused across as many test runs as you want.
2. Start GIMP 2.8, create an image, 640*480, white. 
3. Save as XCF to the SMB symlink. Leave GIMP running.
4. Add a new layer, same size, transparent. Layer content is irrelevant.
5. Click Save to re-save as XCF to the SMB symlink. Quit GIMP.
6. Open the XCF by double-clicking file or opening it via GIMP.
No layers are displayed and the error message appears in status bar.
A hex view of the XCF shows about 1400 zeros inserted after the header.
----
Much easier to test it that way. Has happened in 2 out of 2 attempts.
Comment 26 ajmcrae 2014-06-17 13:00:11 UTC
Created attachment 278597 [details]
Result of the simpler test procedure in comment 25 .

Performing the test of comment 25 on the SMB share should result in a XCF file byte-for-byte identical to this one.
Comment 27 Michael Schumacher 2014-06-17 13:14:13 UTC
So is saving on an SMB share a necessary condition, or does it make reproducing this much easier?
Comment 28 ajmcrae 2014-06-17 13:26:21 UTC
Created attachment 278600 [details]
The simplest test case.

I'm sorry everyone, this must seem like spam, but I have found the test can be even simpler. You hardly have to do a thing for this bug to occur.
Same environment as before, Linux Mint 15, QNAP TS209II NAS, Gimp 2.8.4, the SMB share is symlinked from gvfs into the user's home directory.

1. Create a new image, any size, white.
2. Click "Save As" and save it to an SMB share.
3. Click "Save" to re-save the same image to the same file. The bug occurs.
4. Quit GIMP, now re-open the XCF and there are no layers displayed.

The image content and dimensions are totally irrelevant. No editing or layer alteration required. Just resaving anything to SMB shows the bug.

Re-saving to a local drive has never shown the bug.

Saving to a new file on SMB each time you save works just fine. But the moment you re-save over the same file via SMB, the file is corrupted.
Comment 29 Michael Schumacher 2014-06-17 13:41:39 UTC
People, are we sure that everyone who experienced the original bug was saving to SMB shares?

And if so, that we are not hunting a different bug then, e.g. bug 632296 (which is a bit more generic that just Nautilus)?
Comment 30 Daniel Savi 2014-06-17 17:05:23 UTC
(In reply to comment #28)

> 
> 1. Create a new image, any size, white.
> 2. Click "Save As" and save it to an SMB share.
> 3. Click "Save" to re-save the same image to the same file. The bug occurs.
> 4. Quit GIMP, now re-open the XCF and there are no layers displayed.
> 

Testing this recipe on my Mac with saving to an SMB share (QNAP network drive) did not result in a corrupted file. I can open it again and the background layer is still there.

Exact Versions:
OS X 10.9.3
Gimp 2.8.10
Comment 31 Shelbie 2014-06-17 18:46:52 UTC
I'm off for the summer - I will probably stop by my office at some point, but I don't know when that will be (since it's 25 minutes away).  I will check to answer your question then, but for now I'd guess the version of OS X on my desktop is likely the same as my laptop, which is 10.8.5.  Thank you.

(In reply to comment #21)
> Shelbie, can you please tell us which version of OS X you are running?
> You find this information by clicking on the Apple icon in the menu bar and
> then choosing 'About this Mac'. Thank you in advance.
Comment 32 ajmcrae 2014-06-19 01:40:02 UTC
(In reply to comment #29)
> People, are we sure that everyone who experienced the original bug was saving
> to SMB shares?
> 
> And if so, that we are not hunting a different bug then, e.g. bug 632296 (which
> is a bit more generic that just Nautilus)?

That is an interesting possibility, especially if it were due to a buggy version of samba on the NAS server, rather than a bug in the SMB client or GVFS.
However there is still the fact that I've never seen this corruption happen with any application other than GIMP.
For example, if saving a text file to the SMB share corrupted text files as reliably as that other ticket 632296 says it does, I and everyone else using Linux-based PCs or Linux-based NASes would be seeing that error all the time.

I agree it's important to narrow down whether this only happens on SMB shares, or whether it is any filesystem. When I said "Re-saving to a local drive has never shown the bug" I was only talking from my own experience. 
I don't think Shelbie ever told us whether they were saving to a Windows share.

Seems there have been 9 configurations tested in the above comments...
<pre><code>
_Config__________________________ #Bug_ #NoBug_
OSX 10.9.3 + Gimp 2.8.10 + Local |  0  |  1   | comment 19
OSX 10.9.3 + Gimp 2.8.10 + SMB.  |  0  |  2   | comment 19 + comment 30.
OSX 10.8.5 + Gimp 2.8.10 + ??    |  1  |  0   | comment 0 dest unknown
Mint 15 + Gimp 2.8.4 + SMB.      |  1  |  0   | comment 28
Mint 15 + Gimp 2.8.4 + Local.    |  0  |  1   | comment 28
'Linux' + Gimp 2.8.10 + SMB.     |  0  |  1   | comment 19
'Linux' + Gimp 2.8.10 + Local.   |  0  |  1   | comment 19
Windows7 + Gimp 2.8.10 + SMB.    |  0  |  2   | comment 19 + this.
Windows7 + Gimp 2.8.10 + Local.  |  0  |  1   | comment 19
</code></pre>

Do people agree that the above is a summary of test results so far?

Massimo says it doesn't happen on Linux, well yes it does, but his configuration worked okay, we just don't know what that config was, therefore what is the difference between his config and my config, it's not as simple as Linux or Not Linux.

I have now tested GIMP 2.8.10 on Windows 7, saving to the same SMB share via UNC path, and the bug does not occur for me.

Do people agree that the tests so far indicate that the defect is either in some lower level library/service which GIMP depends on in OSX and Linux but not in Windows, or in the way GIMP uses that library?

There's also the issue of whether this is due to an old version of Samba running on my old 2008-era NAS, but then Shelbie has not said s/he was saving to an old NAS, and neither has Daniel said what model his QNAP NAS was, and I have not had the problem in any other application.
Once we throw particular vendor versions of particular Samba shares into the mix the number of combinations goes up. Can we just check the parts over which GIMP devs have a bit more control first? ie- GIMP and the file libraries it uses?
Comment 33 Massimo 2014-06-19 05:03:18 UTC
(In reply to comment #32)
> 
> Massimo says it doesn't happen on Linux, well yes it does, but his
> configuration worked okay, we just don't know what that config was, therefore
> what is the difference between his config and my config, it's not as simple as
> Linux or Not Linux.
> 

I said it doesn't happen on Linux, when it wasn't known that
SMB shares were involved. I don't have SMB shares to test. I
was only saving locally or to a pen drive and have never seen
extraneous zeros mid file.

The interesting question is: does a GIMP patched with the patch
from comment #14 fixes the problem on at least a system where it
was observed?

We'll see always and anyway corruptions caused by faulty disk or
network drivers etc.
Comment 34 Max Mustermann 2014-07-16 04:12:01 UTC
I tested Massimo's patch to load broken files. It passes when opening the broken file from the original poster. It fails when trying to open the test case from comment 28. Massimo, can you please make your patch also work with that?
Comment 35 Massimo 2014-07-16 04:45:36 UTC
(In reply to comment #34)
> I tested Massimo's patch to load broken files. It passes when opening the
> broken file from the original poster. It fails when trying to open the test
> case from comment 28. Massimo, can you please make your patch also work with
> that?

No, that file is broken differently IIRC, I added code that considers
only one possible occurrence of a failed seek to the end because I've seen
several files damaged in the same way.

I attached https://bugzilla.gnome.org/attachment.cgi?id=278467 here
only to help people recover their files, not much to be included in
GIMPs tree and files damaged in many places would need a special analysis
probably better placed in a recovery tool, than in GIMP itself.

Plus I think we are seeing the effects of many bugs, not only in GIMP.
Comment 36 Max Mustermann 2014-07-22 18:39:51 UTC
Review of attachment 277612 [details] [review]:

Thank you. I added some comments to the code and committed it to GIMP 2.8. Commit to GIMP master follows.

commit a57e49b1bbd318b06c4f8d578557b63256a1e66d
Author: Massimo Valentini <mvalentini@src.gnome.org>
Date:   Sat Jun 14 17:57:11 2014 +0200

    Bug 730211 - Extra zeroes appear in XCF files (draft)
    
    Patch to solve bug 730211. GIMP can be built with it and a simple
    test to save and load "layers recovered.xcf" passes.
    Needs further review and extensive testing as proposed by Massimo
    in comment 14 of the bug report.

 app/xcf/xcf-save.c | 184 ++++++++++++++++++++++++++++++++++++++++++++++++++--------------------------------------------------------------------------------------------------------------------------------------
 1 file changed, 50 insertions(+), 134 deletions(-)
Comment 37 Max Mustermann 2014-07-22 18:41:10 UTC
Review of attachment 278467 [details] [review]:

Thank you. I did some minor changes and committed it to GIMP 2.8. GIMP master will follow soon. 

commit e1e8e19237937766f44c7fdf8c184b64dfe8492b
Author: Massimo Valentini <mvalentini@src.gnome.org>
Date:   Tue Jul 15 19:55:37 2014 +0200

    Bug 730211-Load XCF files with shifted layer offset table
    
    Load xcf files that have extra NULL characters between image properties
    and the layer offset table.
    
    Improvements to Massimo's patch:
    - Replace goto by while loop.
    - Add comments.

 app/xcf/xcf-load.c | 292 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------------------------------------------------
 1 file changed, 184 insertions(+), 108 deletions(-)
Comment 38 Max Mustermann 2014-07-22 18:53:29 UTC
The original problem is solved and the bugfix will be in the next release GIMP 2.8.12.
I also rechecked saving on SMB network shares and had no problems - neither with the official GIMP 2.8.10 build from Clayton Walker nor with my updated version.
So that problem must be different to the originally posted. Let's handle it in bug #632296.
Comment 39 Ross Lagerwall 2014-07-22 21:43:00 UTC
I can reproduce this using the recipe from comment #28.  The Linux issues are caused by the behavior of the fuse fallback code provided by gvfs for programs which don't use GIO (see my comments in bug 632296).

However, this will only reveal itself using certain GVFS network backends on Linux *and* when the file save plugin does not support GIO or you force it to use the fuse fallback code (as the symlink hack does). Any file corruption on Windows, OS X (even if saving to an smb backend), or for a local file is certainly a different issue.

Cheers
Comment 40 ajmcrae 2014-07-23 09:33:59 UTC
Well I guess I will have to take my woes to bug 632296.

> this will only reveal itself using certain GVFS network backends on 
Linux *and* when the file save plugin does not support GIO or you force it to
use the fuse fallback code (as the symlink hack does).

Trying to "Save As" to the SMB share from the connected shares listed on the left side of the Save Image dialog will simply cause the error:
> Saving 'smb://nas1/public/Technology/Electronics/Computers/Untitled.xcf' failed:
> Could not open 'smb://nas1/public/Technology/Electronics/Computers/Untitled.xcf' for writing: No such file or directory

It seems there is no way to save an image to a SMB share in GIMP without using the symlink hack. 

Only by saving to the connected share via the home directory symlink will the "Save As" even save anything at all. And of course it saves okay the first time.
It's the re-save over the same file that corrupts the file, as the victims of 632296 seem to have found.
Comment 41 Ross Lagerwall 2014-07-23 12:08:08 UTC
(In reply to comment #40)
> Well I guess I will have to take my woes to bug 632296.
> 
> > this will only reveal itself using certain GVFS network backends on 
> Linux *and* when the file save plugin does not support GIO or you force it to
> use the fuse fallback code (as the symlink hack does).
> 
> Trying to "Save As" to the SMB share from the connected shares listed on the
> left side of the Save Image dialog will simply cause the error:
> > Saving 'smb://nas1/public/Technology/Electronics/Computers/Untitled.xcf' failed:
> > Could not open 'smb://nas1/public/Technology/Electronics/Computers/Untitled.xcf' for writing: No such file or directory

GIMP is clearly doing something wrong here but I believe Mitch is in the process of fixing this so it works properly like other GNOME applications.

> 
> It seems there is no way to save an image to a SMB share in GIMP without using
> the symlink hack. 
> 
> Only by saving to the connected share via the home directory symlink will the
> "Save As" even save anything at all. And of course it saves okay the first
> time.
> It's the re-save over the same file that corrupts the file, as the victims of
> 632296 seem to have found.

I realize what the problem is and should hopefully be able to fix it soon... my previous post was to point out that the original corruption on OS X is clearly not related.
Comment 42 Michael Natterer 2014-07-23 19:44:29 UTC
Ross, I just pushed the outcome of our IRC discussions to git master,
thanks again for the help! :)

I *believe* this should all work properly now in master, but naturally
I have no way of testing all the possible remote file cases.
Comment 43 Ross Lagerwall 2014-07-23 21:18:37 UTC
(In reply to comment #42)
> Ross, I just pushed the outcome of our IRC discussions to git master,
> thanks again for the help! :)

No problem!

> 
> I *believe* this should all work properly now in master, but naturally
> I have no way of testing all the possible remote file cases.

You'll just have to wait for the bugs to start flowing :-)
Comment 44 Michael Natterer 2014-10-12 01:05:36 UTC
This bug should not have been closed, the fixes were pushed only
to 2-8.
Comment 45 Michael Natterer 2014-10-15 21:20:10 UTC
Pushed a modified (commented) saving fix to master:

commit 1368f73354070a40c0a73e92c5075164b3a07848
Author: Massimo Valentini <mvalentini@src.gnome.org>
Date:   Wed Oct 15 23:15:18 2014 +0200

    Bug 730211 - Extra zeroes appear in XCF files
    
    Don't use xcf_seek_end() because that seems to be broken on certain
    file systems / operating systems / FUSE mounts / whatever. Instead,
    seek to explicitly calculated file offsets.
    
    Ported Massimo's patch to master and added comments --Mitch
    
    The same commit in gimp-2-8 is a57e49b1bbd318b06c4f8d578557b63256a1e66d

 app/xcf/xcf-save.c | 235 +++++++++++++++++++++--------------------------------
 1 file changed, 93 insertions(+), 142 deletions(-)
Comment 46 Michael Natterer 2015-02-15 13:55:05 UTC
Massimo, I lost track here. What is missing to close this bug?
Comment 47 Carl 2015-04-04 04:34:18 UTC
After a day's work meticulously retouching a large scan of an old humidity-damaged photograph, I have lost all my work due to this bug. I did most of the work on a Windows machine running GIMP 2.8.10 and then had to switch to a Linux Mint 17.1 machine with more memory to finish off. The current version for Linux Mint is still GIMP 2.8.10 and I opened the XCF file using an SMB share, did the final steps, and saved. Reopening the file found it to be empty. Desperation had me examining Massimo's C-code in comment 6 in the hopes I could use it to recover my day's work, but then I looked at a hex dump of my XCF file. Out of 44MiB I have 27KiB that is non-null - those numbers are not typos. My topmost layer was a trivial small one, so that's probably all that's left. All the significant image layers are gone. Utter annihilation with no hope of recovery.

The title for this bug report is therefore misleading. It may be that in some cases the zeroes are "extra", but in some cases valid image data is simply overwritten with zeroes.

I don't understand why this "urgent critical" bug is still open after so much time has passed, but if there is a stable GIMP version in which this has actually been fixed, is there something that can be done to pressure mainstream repositories to upgrade? Right now GIMP is a major hazard, at least for Ubuntu and Linux Mint users.
Comment 48 Michael Schumacher 2015-04-04 09:51:24 UTC
Carl, in order to get patches like this into a Linux distro, you should open a bug in their bug tracker.
Comment 49 Michael Natterer 2015-05-25 23:24:50 UTC
Massimo, ping?
Comment 50 Michael Schumacher 2015-06-06 09:33:03 UTC
BTW, we do not think that this is OSX specific like the platform setting of this bug implies, do we?
Comment 51 Michael Natterer 2015-06-06 14:27:12 UTC
Nope.
Comment 52 Michael Schumacher 2015-08-10 08:08:58 UTC
So for all that we know, this issue should be fixed by the change described in comment 45. This would mean that we shouldn't receive any reports of the problem happening for user who are running either a current (i.e. post-2.8.14) gimp-2-8 build or a current git master build.

Is there any reason why we keep the bug report open?
Comment 53 Michael Schumacher 2016-05-03 12:38:17 UTC
*** Bug 744035 has been marked as a duplicate of this bug. ***
Comment 54 Michael Schumacher 2016-10-10 10:05:48 UTC
So... I don't recall any recent reports of this happening with current versions (2.8.16 and newer), at all. I'd like to have it in the list of changes for 2.8.20, finally - it's been missing from many release announcements already.

Any objections to resolving it as fixed? Speak up until 2016-10-27.
Comment 55 Michael Natterer 2016-11-07 18:41:33 UTC
No objections, closing as FIXED.