GNOME Bugzilla – Bug 309592
problem with xfer with REMOVESOURCE when source and target are on different file systems
Last modified: 2008-09-06 19:10:32 UTC
Drag some text into CVS nautilus on a machine where /tmp and $HOME are on different file systems. You get "Error "Not on the same file system" creating new document." Probably because nautilus creates a tmp file in /tmp and tries to move it with gnome_vfs_move_uri
*** Bug 309594 has been marked as a duplicate of this bug. ***
*** Bug 309699 has been marked as a duplicate of this bug. ***
Hrm I don't see what we do wrong here: We use GnomeVFS Xfer means [1] to move the file from /tmp to its destination, which should work like a charm in theory. Christian Kellner aka GnomeVFS code monkey also isn't sure about why the XFer fails. Any ideas? [1] http://cvs.gnome.org/viewcvs/nautilus/libnautilus-private/nautilus-file-operations.c?r1=1.191&r2=1.192
FWIW, I also get this message when dropping a file from an archive using file-roller.
this is with a g_print of "progress_info->phase" and of the different case options: phase: 0 GNOME_VFS_XFER_PROGRESS_STATUS_OK phase: 1 GNOME_VFS_XFER_PROGRESS_STATUS_OK phase: 2 GNOME_VFS_XFER_PROGRESS_STATUS_OK phase: 2 GNOME_VFS_XFER_PROGRESS_STATUS_OK phase: 3 GNOME_VFS_XFER_PROGRESS_STATUS_OK phase: 3 GNOME_VFS_XFER_PROGRESS_STATUS_VFSERROR: 36 phase: 16
bug to fix for 2.12
*** Bug 310206 has been marked as a duplicate of this bug. ***
My /tmp isn't on a different partition as my $HOME, but the destination of where I wanted to create the file is. I guess the summary is wrong?
Right, doesn't matter really. But thanks for pointing this out.
*** Bug 310898 has been marked as a duplicate of this bug. ***
My /tmp and /home are on different partitions but on the same filesystem (reiser).
*** Bug 311396 has been marked as a duplicate of this bug. ***
*** Bug 312145 has been marked as a duplicate of this bug. ***
*** Bug 312768 has been marked as a duplicate of this bug. ***
*** Bug 313188 has been marked as a duplicate of this bug. ***
simple gnomevfs-mv test shows that actually that bug belongs to gnome-vfs. I think that because that is where we will be moving/copying to. */ static GnomeVFSResult do_check_same_fs (GnomeVFSMethod *method, GnomeVFSURI *source_uri, GnomeVFSURI *target_uri, gboolean *same_fs_return, GnomeVFSContext *context) in file-method of gnome-vfs is broken.
I even found why it fails - it checks destination file! but it doesn't exists befor moving, so just compare of stat info is wrong. See strace log setresgid32(-1, 500, -1) = 0 stat64("/home/shmyrev/t/b", 0xbfb5c944) = -1 ENOENT (No such file or directory) rename("/home/shmyrev/a", "/home/shmyrev/t/b") = -1 EXDEV (Invalid cross-device link) write(2, "move `file:///home/shmyrev/a\' `f"..., 86move `file:///home/shmyrev/a' `file:///home/shmyrev/t/b': Not on the same file system ) = 86
Ups, the two last comments was wrong.
Well, I've created new filesystem in $HOME/t with mount --bind and what I get with test-xfer from gnome-vfs tests of current cvs. [shmyrev@gnome test]$ ./test-xfer -d file:/home/shmyrev/t/b file:/home/shmyrev/.Trash/abd Warning: Removing source files. Status: OK Initial phase Status: OK Unexpected phase 1 Status: OK Collecting file list Status: OK Collecting file list Status: OK Ready to go! VFS Error: Not on the same file system [shmyrev@gnome test]$
lstat64("/home/shmyrev/.Trash/abd", 0xbfef15cc) = -1 ENOENT (No such file or dirgettimeofday({1123921193, 439950}, NULL) = 0 stat64("/home/shmyrev/.Trash/abd", 0xbfef15c4) = -1 ENOENT (No such file or direrename("/home/shmyrev/t/b", "/home/shmyrev/.Trash/abd") = -1 EXDEV (Invalid crosgettimeofday({1123921193, 440145}, NULL) = 0 Corresponding strace.
*** Bug 151257 has been marked as a duplicate of this bug. ***
Well, after digging on that problem I can state the following: rename can fail by various reasons. In can be error with binded filesystem, where actual device number will be the same, it can be NFS error, it can be simple race condition, since we first check for same device and only then do rename. Here is useful for my opinion quote from coreutils mv code: /* This may mean SOURCE and DEST referred to different devices. It may also conceivably mean that even though they referred to the same device, rename wasn't implemented for that device. E.g., (from Joel N. Weber), [...] there might someday be cases where you can't rename but you can copy where the device name is the same, especially on Hurd. Consider an ftpfs with a primitive ftp server that supports uploading, downloading and deleting, but not renaming. Also, note that comparing device numbers is not a reliable check for `can-rename'. Some systems can be set up so that files from many different physical devices all have the same st_dev field. This is a feature of some NFS mounting configurations. We reach this point if SOURCE has been successfully copied to DEST. Now we have to remove SOURCE. This function used to resort to copying only when rename failed and set errno to EXDEV. */ So I think the correct solution in this situation is fallback to copy in gnome_vfs_xfer in the case rename failed. This will make file management similar to one in other file management systems. Of course, we can update gnome_vfs_move, but it will break API in my understanding.
Created attachment 50648 [details] [review] Proposed patch
Thanks for your work on this. Reassigning the bug to gnome-vfs.
Uhh a so big change in xfer so short before a release. Ewks! So form a very quick look you are suggesting to just do the move and if the result is NOT_SAME_FS then switch it over to a copy in do that instead right? Let me have a closer look on that after breakfast!
Nautilus crashed 2 times while creating 5-6 new documents with this patch: Backtrace was generated from '/usr/bin/nautilus' Using host libthread_db library "/lib/tls/i686/cmov/libthread_db.so.1". `system-supplied DSO at 0xffffe000' has disappeared; keeping its symbols. [Thread debugging using libthread_db enabled] [New Thread -1226205504 (LWP 27280)] [New Thread -1241154640 (LWP 27287)] [New Thread -1240888400 (LWP 27286)] [New Thread -1240622160 (LWP 27285)] [New Thread -1240355920 (LWP 27284)] [New Thread -1240089680 (LWP 27283)] [New Thread -1239823440 (LWP 27282)] [New Thread -1229071440 (LWP 27281)] 0xffffe410 in __kernel_vsyscall ()
+ Trace 62437
Thread 8 (Thread -1229071440 (LWP 27281))
As reported by Seb, this patch makes nautilus crash. Something with the uri list is wrong (havent look to close at it for now).
*** Bug 313715 has been marked as a duplicate of this bug. ***
Well. Missed latest comments, so adding myself as CC. Really patch is not so clean. Probably we should postpone it after the release. Re #25: Christian, you are right. The suggestion is to fallback to copy when rename fails.
Btw, it seems that "fallback to copy when NOT_SAME_FS" is what mv does, and it is well explained in the man page, extracted from "man mv": When both source and destination are on the same filesystem, they are the same file (just the name is changed; owner, mode, timestamps remain unchanged). When they are on different filesystems, the source file is copied and then deleted. mv will copy modification time, access time, user and group ID, and mode if possible. When copying user and/or group ID fails, the setuid and setgid bits are cleared in the copy.
Created attachment 51027 [details] [review] Updated patch Crash should be fixed with this patch. Sorry, there was misunderstanding on how refcounting is going. Please test. Actually, this patch only looks so big. There is not much changes in it. Fortunately, there also was support for falling back to copy. It just use this support and move function to calculate copy size a bit lower, so it will also calculate size of newly added to copy list files. This new source is almost copy-pasted from old place.
I have gnome-vfs 2.10.1 with this patch applied since two days and it's playing fine, I've created several new documents, moving several files, folders, working with several windows and so on and haven't encountered any problem with it. Btw, looking for corner cases I manage to found a bug in copy mode of nautilus, for curious is bug 310061.
I have been testing the patch on 2.11.90 since release and it works well. This is a showstopper in my opinion, as it breaks something as basic as creating a new file on the desktop, and should be fixed in cvs asap. Whoever needs to review the patch, please do, and then let's get this commited. Gnome is fast becoming a really usable, great desktop. But these types of bugs, especially since it is an easy fix, really should get patched quickly. Thanks Nickolay for the patch.
I have encountered that if you try to move a file to which you have copy permission but not delete permission, a dialog error saying it cannot be moved appear, that dialog have a "Cancel" and "Omit" button, whichever button you press the destination file that have just being copied is not deleted, so the final result is like a copy operation not a move operation. Imho this is another bug that is not directly related to Nickolay's patch but who better can know that is Nickolay and people confidence to the code. But until I file/search a bug for this, here is a testcase: nelson@gnelson ~ $ mkdir temp nelson@gnelson ~ $ mkdir mounted nelson@gnelson ~ $ echo "hola" > temp/hola.txt nelson@gnelson ~ $ su Password: gnelson nelson # mkdir /mnt/mounted gnelson nelson # mount -o bind /home/nelson/mounted /mnt/mounted gnelson nelson # chown root:root temp gnelson nelson # exit exit nelson@gnelson ~ $ nautilus /mnt/mounted nelson@gnelson ~ $ nautilus temp (NOW DRAG-N-DROP 'hola.txt' from temp to mounted, press whichever button you want in error dialog, result is 'hola.txt' is not deleted from mounted as expected)
Also, it seems that gnomevfs-mv is not getting advantage of Nickolay's patch: nelson@gnelson ~ $ gnomevfs-mv /home/nelson/normal_file.txt /mnt/mounted/normal_file.txt move `file:///home/nelson/normal_file.txt' `file:///mnt/mounted/normal_file.txt': Not on the same file system nelson@gnelson ~ $ ls -l /mnt/mounted total 0 nelson@gnelson ~ $
Nelson: This is another issue, filed as bug 314247. After applying the patch of that bug report, Nickolay's patch should work.
After doing what Christian says in comment 36 , I could check it with gnomevfs-mv and the bug is there, so now it's officially a gnomevfs thing ;) : nelson@gnelson ~ $ gnomevfs-mv temp/hola.txt /mnt/mounted/ move `file:///home/nelson/temp/hola.txt' `file:///mnt/mounted/': Access denied nelson@gnelson ~ $ ls temp hola.txt nelson@gnelson ~ $ ls /mnt/mounted/ hola.txt nelson@gnelson ~ $
Also, as for my comment 34 , I've not been able to reproduce it in copy operations, that's because when a copy operation fails in the middle it's fair to leave in place the files that have already been copied (in fact that's what happens when you press "Cancel" or "Omit") but here we are using a copy operation to simulate a move operation which imho it's not fair to leave the file in place if the operation fails. So afaict this is not a separate bug but a corner case of this bug. Also I don't hesitate if Nickolay's patch gets committed and this corner case is left to address it in future releases.
Nelson: Why is that a bug? Isn't the "access denied" error supposed to be returned, since you're not allowed to remove the source.
Acutally when looking a bit more to the gnome-vfs-xfer function I see there is logic in place to check for same_fs and transmogrify move into copy if that fails. Strange enough that is embedded into a if ((xfer_options & GNOME_VFS_XFER_USE_UNIQUE_NAMES) == 0) { } statment. So if you dont have that set everything should be fine. I wonder why this is here. I dont wanna rush into wrong things here now.
Christian: the "access denied" error is not the bug, the bug is that after execution of gnomevfs-mv command you ended up with two files: nelson@gnelson ~ $ ls temp hola.txt nelson@gnelson ~ $ ls /mnt/mounted/ hola.txt nelson@gnelson ~ $ So this is an *inconsistent* state (for a move operation), exactly as when you cancel a copy of a large file at middle, nautilus will delete the partially copied file to put you in *consistent* state. I guess gnomevfs-mv internal code (gnome-vfs-xfer.c) should remove destination file in case it failed to remove source file, so it reverts to original *consistent* state. Also an error message could be echoed telling the user the "move operation" could not be done, for example with the text: "the move operation could not be performed because could not delete source file when moving file across different filesystem".
Hrm, so, everyone cc'd: should this be a 2.12 stopper? Thoughts? I'm leaning towards 'no' but it does seem to break the feature completely for large deployments with nfs/afs/whatever homedirs, I assume.
*** Bug 312150 has been marked as a duplicate of this bug. ***
From Bug 312150: If GNOME_VFS_XFER_REMOVESOURCE is passed to gnome_vfs_async_xfer, the move sometimes fails if the source and the target are on different file systems, since it calls gnome_vfs_move_uri, which checks for check_same_fs_in_uri in gnome-vfs-cancellable-ops.c:gnome_vfs_move_uri_cancellable. gnome_vfs_async_xfer should rather copy the files and clean up the source afterwards if the files are on different FSes. gnome_vfs_xfer_uri_internal seems to do the right thing if GNOME_VFS_XFER_USE_UNIQUE_NAMES is TRUE and gnome_vfs_check_same_fs_uris returns GNOME_VFS_OK which - from what I can see in file-method.c - may not always be the case. Maybe we have to a) remove the _USE_UNIQUE_NAMES branch and always do the same_fs voodoo b) also check for GNOME_VFS_ERROR_INVALID_URI when deciding whether to break in the same_fs loop.
it happens also with local partitions. I'm using two hard disks: /dev/hda2 75346652 25884632 45634536 37% / tmpfs 517000 0 517000 0% /dev/shm /dev/hda1 256666 14497 228917 6% /boot /dev/hdb1 76890340 64353144 8631396 89% /home no NFS or whatever, so I'd say it is a showstopper, and a regression, since this used to work for me, on the same setup.
Some comments from an initial readthrough of this bug: * gnome_vfs_move() is meant to only work if check_same_fs returns true. This has been the semantics always, and will not change. * gnome_fs_xfer() really should try to do a copy if the move fails (which it can for various reasons stated above). However, this is a bit delicate if the removal fails. What do we do then? remove the copied file? What if that copy replaced a file, it will now be gone. I guess there really isn't a solution to this, so we have to just figure out the least bad approach and do that. * I'm very hesitant to do any changes in gnome_vfs_xfer at this late time in the cycle. This is very core code and bugs here can easily cause major problems. * The original bug is due to a change in how template files are copied in Nautilus. This was a quite minor change. Before that template creation worked everywhere, even though this bug existed. I think the right fix for 2.12 is to just fix nautilus to work like it used to.
I fixed the problem in Nautilus by manually removing the tempfile. Leaving this bug open for the future.
Thanks, Alex. Removing from the 2.12 showstopper list then, unless anyone objects.
The fix in nautilus works for me
Well, Nickolay's patch also fixed the common case of people just moving files between two harddisks/partitions, but it's understandable we are very close to a release to change core code. Also, as for comment 46 , I think the "least bad approach" is to remove copied file when it's not an overwrite, when it is an overwrite we leave copied file (better to have two repeated files than a possible lost file).
*** Bug 322379 has been marked as a duplicate of this bug. ***
Other Ubuntu bug about this: http://bugzilla.ubuntu.com/show_bug.cgi?id=20541 "It's possible to create a symlink in nautilus that references an object on another filesystem than where the link is housed. However, nautilus refuses move such links to the Trash. To reproduce: 1) Open two nautilus windows, showing folders on separate filesystems, where you have write access to at least one of the folders 2) Middle-click and drag an object from one window to the other, choose "Link here" 3) Remove the link with the "Del" key or Right-click -> "Move to Trash" Nautilus replies with a dialog: Error "not on the same file system" while deleting "/path/to/link". Would you like to continue? [Cancel] [Retry] Retrying will repeat the error. 3b) Using direct "Delete" (if it's enabled in prefs) deletes the link though. I don't think this is the same as Bug #16884 , altough similar. ... > Thanks for your bug. What version of Ubuntu do you use? ... Breezy, with latest updates" With the current dapper packages it doesn't open a dialog but doesn't move the file neither
*** Bug 323638 has been marked as a duplicate of this bug. ***
Other ubuntu bug about that: https://launchpad.net/distros/ubuntu/+source/nautilus/+bug/6312
In nautilus 2.13.92 the problem persists. I have a symlink on /dev/hda3 (root) filesystem, that points to a file on /dev/hda5. I also receive '''Error "Not on the same file system" while deleting "/home/eric...dise Bonus".''' Interesting: right-click & move to wastebasked: FAIL select & hit delete: FAIL drag to Trash-applet in panel: SUCCESS!
Other Ubuntu bug about that: https://launchpad.net/distros/ubuntu/+source/nautilus/+bug/33210 "I have my trash on a separate file system from my desktop. When I select an icon on the desktop and try to delete it with the delete key, I get an error msg about "not on the same filesystem" and am asked if I want to continue. The options presented are Cancel and Retry. If I delete the file by dragging it to the wastebasket, it is all fine. So, - as a minor issue, if in fact moving files across file systems is something that users shouldn't be able to do implicitly, this should probably be a warning, not an error (I'm filing a related bug on the question of whether this can be an implicit action) - more importantly, it asks if I would like to continue. Suppose I do want to continue. The best I can do is select Retry, which tries to do the same thing and generates the same error message. If I acknowledge the error message and explicitly say I want to continue, it should allow the operation to continue, but currently does not" I figured than "Doing a "ln -s some_path_on_other_partition .Trash" gives the same message you described"
Other Ubuntu bug about that: https://launchpad.net/distros/ubuntu/+source/nautilus/+bug/43237 "Inconsistent Error Message regarding Link Deletion Step by step description: 1. I want to delete a link on HD1 to a directory on HD2 by choosing the link and pressing [Delete]. 2. I receive the following error message: Error "Not on the same file system" while deleting [path]". Would you like to continue? [Cancel] [Retry] [Cancel] is OK, however [Retry] is inconsistent since even after a million retries, the link will not be deleted. I would expect that I can delete the link, in particiular logged in as root."
Other Ubuntu bug about that: https://launchpad.net/distros/ubuntu/+source/nautilus/+bug/45709 "I've got my data partition mounted as ~/data. if I do "ln -s ~/data ~/data-ln" and then try to delete data-ln with nautlus I get an "not on the same filesystem error" whilst removing it from console works fine. So i guess this is an nautilus error..."
*** Bug 345296 has been marked as a duplicate of this bug. ***
The failure to handle the case where rename() returns EXDEV breaks the ability to move files across volumes in AFS, which is a serious problem for us. A brief test of the updated patch (falling back to copy when rename fails), backported to gnome-vfs 2.8.4, appears to fix the problem. Is there any chance of getting this patch (or whatever is deemed to be a more appropriate fix) committed?
Patch review from nautilus-list: Is it really right to move the size recalculation down to after the move has happened? I mean, then we will be sending the wrong size during the whole operation, and then at the end send the right one. The progress dialog will look very weird. Why did you do this? Otherwise it makes sense imho.
Hm, are you sure you need another behaviour. Current one in the patch seems natural for me - while we are moving files on the same filesystem we show only non-recursive sizes and it's correct since moves are recursive. Move is quite fast and shouldn't worry user, while during copy we'll show the size of files we are copying. And we have to check for free space twice. If so, I can update patch to calculate size before move.
Created attachment 82092 [details] [review] Another patch This one calculate sizes twice. Probably this one is better, but I am not sure.
*** Bug 337202 has been marked as a duplicate of this bug. ***
gnome-vfs has been deprecated and superseded by gio/gvfs since GNOME 2.22, hence mass-closing many of the gnome-vfs requests/bug reports. This means that gnome-vfs is NOT actively maintained anymore, however patches are still welcome. If your reported issue is still valid for gio/gvfs, please feel free to file a bug report against glib/gio or gvfs. @Bugzilla mail recipients: query for gnome-vfs-mass-close to get rid of these notification emails all together. General further information: http://en.wikipedia.org/wiki/GVFS Reasons behind this decision are listed at http://www.mail-archive.com/gnome-vfs-list@gnome.org/msg00899.html