GNOME Bugzilla – Bug 627567
Data loss when writing files
Last modified: 2010-11-12 15:48:54 UTC
Some files are getting truncated when I write them to my wife's computer over the sftp backend to gvfs. I opened a photo from her computer using the GIMP on my computer, changed the colors a bit, and, when I wrote it back, it was a zero-length file. I don't get the same problem using gedit. The problem is reproducible. There's a Fedora bug at https://bugzilla.redhat.com/show_bug.cgi?id=605308.
Actually every editor: emacs, pico/nano, komodo, that is not using gio, suffers from this regression. It seems that gvfs=1.6.1 (as in ubuntu 10.04) is the latest version that does not have this bug (also see http://forums.fedoraforum.org/showthread.php?t=250636).
(In reply to comment #1) > It seems that gvfs=1.6.1 (as in ubuntu 10.04) is the latest version that does > not have this bug (also see > http://forums.fedoraforum.org/showthread.php?t=250636). That's weird, gvfs is developing slowly these days, I don't see any related commits since 1.6.1. Appending to the file gives me > $ echo xxxx >> ahoj.txt > bash: echo: write error: Operation not supported so I wonder how happened the file got truncated. However nano produces truncated file.
(In reply to comment #2) > (In reply to comment #1) > > It seems that gvfs=1.6.1 (as in ubuntu 10.04) is the latest version that does > > not have this bug (also see > > http://forums.fedoraforum.org/showthread.php?t=250636). > That's weird, gvfs is developing slowly these days, I don't see any related > commits since 1.6.1. It was just a wild guess. Based on those facts that editing files on gvfs (sftp) mount was reliable on ubuntu 10.04 with gvfs=1.6.1-0ubuntu1build1, always a loss on the first save on ubuntu 10.10 with gvfs=1.6.3-0ubuntu2 and seems unreliable on fedora with gvfs=1.6.2. > Appending to the file gives me > > $ echo xxxx >> ahoj.txt > > bash: echo: write error: Operation not supported > so I wonder how happened the file got truncated. > > However nano produces truncated file. strace on nano shows that file update is done by truncating destination file and writing over entire new content through mmapped file view: open("/<>/.gvfs/sftp on <>/<>/_test", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 umask(022) = 022 fcntl(3, F_GETFL) = 0x8001 (flags O_WRONLY|O_LARGEFILE) fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f285128c000 lseek(3, 0, SEEK_CUR) = 0 write(3, "200\n", 4) = 4 close(3) = 0 munmap(0x7f285128c000, 4096) = 0 stat("/<>/.gvfs/sftp on <>/<>/_test", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 and it seems that that mmap writing just doesn't work and leave file in truncated state.
Sorry, discard poke at mmap, I wasn't attentive enough, it obviously has nothing to do with the problem.
Happening also with other backends, such as localtest. It's clearly a fuse daemon issue. Looking at the logs, there's a nonsense series of operations: 1. append_to -> handle 1 2. replace -> handle 2 3. close handle 2 4. write handle 1 5. close handle 1 This way the data are written to the old file handle which gets unlinked by replace in step 2. This particularly happens on > open("/home/tbzatek/.gvfs/sftp on localhost/tmp/ahoj/ahoj2.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
It seems it is the same bug as https://bugzilla.gnome.org/show_bug.cgi?id=631403 Please some solution :S
Well, given that vfs_truncate() is called instead of vfs_ftruncate(), the behaviour from the gvfs-fuse-daemon seems correct to me. Just like POSIX truncate(), it can be called separately with no need to have a file open. So that explains two handles. vfs_ftruncate() properly closes the old handle and replaces it with a new one pointing to a truncated file. So I think this is a FUSE fault after all. The docs say > "If an application specifies O_TRUNC, fuse first calls truncate() and then open()." but in this case the order is exactly opposite (first open, then truncate).
For the record, strace of nano leading to events described above looks like this: > open("/home/tbzatek/.gvfs/sftp on localhost/tmp/ahoj2.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3 > umask(02) = 02 > fcntl(3, F_GETFL) = 0x8001 (flags O_WRONLY|O_LARGEFILE) > fstat(3, {st_mode=S_IFREG|0600, st_size=5, ...}) = 0 > mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f67e7159000 > lseek(3, 0, SEEK_CUR) = 0 > write(3, "ddd\naaaaa\n\n", 11) = 11 > close(3) = 0 > munmap(0x7f67e7159000, 4096) = 0
Created attachment 172274 [details] [review] a patch adding O_TRUNC support for open() This patch adds support for O_TRUNC flag in open(). This means we can truncate the file directly, without calling (f-)truncate. This should fix all above issues. Please test. Still, the previously described FUSE issue persists, this patch only removes the need for the truncate() call.
The patch has solved for me this problem. Tested on Ubuntu 10.10
Same here, problem is gone after applying patch to gvfs=1.6.4-0ubuntu1 (also ubuntu maverick).
this bug is very dangerous for a web developer or a sys admin. i don't really know how to apply the patch. please help! :(
Tomas, thank you for working around this. I noticed you pushed an update with this patch for F15; do you mind backporting to both F13 and F14?
(In reply to comment #13) > Tomas, thank you for working around this. I noticed you pushed an update with > this patch for F15; do you mind backporting to both F13 and F14? I wanted to give it some testing before pushing to master and other branches. I'm also waiting for answer from fuse folks to clarify the machinery. (In reply to comment #12) > this bug is very dangerous for a web developer or a sys admin. > i don't really know how to apply the patch. > please help! :( This really depends on your distribution, I can only provide Fedora packages.
thanks thomas, i'm using ubuntu 10.10. i will use bluefish for now, looks it's the only editor [with gedit] that doesn't have this issue [manly because it does not use gio libraries?] i hope in a quick updato to fix the problem. thanx for your amazing work.
(In reply to comment #15) > thanks thomas, i'm using ubuntu 10.10. > > i will use bluefish for now, looks it's the only editor [with gedit] that > doesn't have this issue [manly because it does not use gio libraries?] > > i hope in a quick updato to fix the problem. > > thanx for your amazing work. I've created applied the patch to the source and created some new .deb packages for 64bit Ubuntu 10.10 http://christiaan.baartse.nl/gvfs_1.6.4-0ubuntu1_amd64_fixed.tar.gz What I basically did was in my home dir mkdir tmpfuse cd tmpfuse apt-get source gvfs Go in the created gvfs Apply the patch to the given files by hand as it was so small dpkg-buildpackage -us -uc -b
christian, amazing work!!! anyway if i try to reinstall those packages i get a creepy message... "if you reinstall gvfs packaes the next updates will not comprend the elements of ubuntu desktop. do you really want to proceed?" is it strange... or not?
(In reply to comment #17) > christian, amazing work!!! > > anyway if i try to reinstall those packages i get a creepy message... "if you > reinstall gvfs packaes the next updates will not comprend the elements of > ubuntu desktop. do you really want to proceed?" > > is it strange... or not? I got the same messages but just ignored them and installed the .deb files. Maybe I should've bumped the version number or somthing. When a new official version comes out I guess I'll just reinstall gvfs via apt-get.
original Ubuntu bug: https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/658069
(In reply to comment #19) > original Ubuntu bug: https://bugs.launchpad.net/ubuntu/+source/gvfs/+bug/658069 Thanks, I'm eager to see the test results.
solved with christiaan baartse rebuilds. i installed his versions and the package manager told there is an update. after the update all i working fine.
Pushed to master and gnome-2-32 stable branch: commit 84f66425ea74f2657f83d00347acf7fc0a638f99 Author: Tomas Bzatek <tbzatek@redhat.com> Date: Fri Nov 12 16:42:49 2010 +0100 fuse: Add O_TRUNC support for open() This requires kernel version 2.6.24 or later. See bug 627567 for details.