GNOME Bugzilla – Bug 779569
Skipped files count for transfer rate
Last modified: 2017-03-06 12:55:30 UTC
When copying data and selecting "skip already existing files" the file sizes of the skipped files influence the transfer rate shown in this small popup. But only actual transferred data should count. In my case of copying a folder, a lot of files were already present, so just few files still had to merged/transferred. I was shown a (with time decreasing) rate of ~10 GB/s instead of the real ~20 MB/s. I'm running Nautilus 3.22.2 on Debian Testing.
Created attachment 347202 [details] [review] file-operations: change skip behavior Currently, when skipping files, they are added to the transfer counts, which results in inflated progress information. This commit reverses the behavior to remove from source counts.
Oh that was quick, thanks a lot! :)
I recognized a further bug in the exact same (still enduring) operation: File count in the same popup now shows me 1124/1081 files, the pie symbol is almost full and no transfer rate is shown – but still many files are missing (transfer still continues). So I assume that skipped files somehow lead to this overflow. Can you ore someone else take a look at this, please?
(In reply to Frank from comment #3) > I recognized a further bug in the exact same (still enduring) operation: > > File count in the same popup now shows me 1124/1081 files, the pie symbol is > almost full and no transfer rate is shown – but still many files are missing > (transfer still continues). So I assume that skipped files somehow lead to > this overflow. Can you ore someone else take a look at this, please? With how it’s handled currently, I can only think of a scenario, where it would be 1081/1124 with all the operations finished (oops), but not an overflow like you describe. I think it’s a specific configuration of files and folders that are skipped or merged that leads to this, but I can’t work it out yet.
My folder contains 1081 binary data files. After skipping ~800 already copied files, the merge operation jumped to ~801/1081 at which the first file was actually transferred, but it counted beyond 1081. It finished at 1177/1081. So I have the feeling that maybe the jump to 801 was too far, or some files were counted wrongly (maybe because already existing files, or maybe folders were counted on the left side, while on the right side the 1081 files just describe data files)?
(In reply to Frank from comment #5) > My folder contains 1081 binary data files. After skipping ~800 already > copied files, the merge operation jumped to ~801/1081 at which the first > file was actually transferred, but it counted beyond 1081. It could be that somehow the file count in the folder is off (unless you know that the number is correct). Sounds like a separate issue. *shrug* > It finished at 1177/1081. So I have the feeling that maybe the jump to 801 was > too far, or some files were counted wrongly (maybe because already existing > files, or maybe folders were counted on the left side, while on the right side > the 1081 files just describe data files)? No, folders just add +1 to the count on both sides, IIRC. No promises yet, but I’ll give it another try.
Should I open another bug report for this issue? The overall folder contains 902 Objects (as Ctrl + I tells me), the missing 179 files seem to be folders.
(In reply to Frank from comment #7) > Should I open another bug report for this issue? > > The overall folder contains 902 Objects (as Ctrl + I tells me), the missing > 179 files seem to be folders. Let’s keep it here for now, since we’re not sure what’s causing this.
Aha! I’m able to reproduce it as well.
If you merge directories, but skip all conflicting files, the code starts skipping the files that aren’t conflicting as well.
Cool! Aaand I just discovered a third bug with skipping files at copying… :P I copied several folders from one drive to another, but one of the folders was there already. At being asked to skip or re-copy, I chose skip and now the progress remains at 3.859/4.396 with the bar being at the corresponding percentage – but there is no abort button, but instead the checkmark for accomplished transfers, because the transfer is complete. So the 537 Objects of the skipped folder aren't counted, and secondly, this behaviour is contradictorily and should be prohibited.
Not quite sure if this is due to the same behaviour you described, but anyways, the checkmark must not coexist with a non-complete progress, if you ask me.
And there is no way to get rid of this status popup…
Created attachment 347264 [details] [review] file-operations: unset skipped_file on copy/move copy_move_file () does not set skipped_file to FALSE explicitly, which results in broken progress information due to callers not resetting the variable.
Created attachment 347265 [details] [review] file-operations: change skip behavior Currently, when skipping files, they are added to the transfer counts, which results in inflated progress information. This commit reverses the behavior to remove from source counts.
(In reply to Frank from comment #11) > Cool! Aaand I just discovered a third bug with skipping files at copying… :P > > I copied several folders from one drive to another, but one of the folders > was there already. At being asked to skip or re-copy, I chose skip and now > the progress remains at 3.859/4.396 with the bar being at the corresponding > percentage – but there is no abort button, but instead the checkmark for > accomplished transfers, because the transfer is complete. > > So the 537 Objects of the skipped folder aren't counted, and secondly, this > behaviour is contradictorily and should be prohibited. Yup, this is what I mentioned before. The two patches I’ve attached should fix all of the issues described here.
Many thanks Ernestas!
Review of attachment 347264 [details] [review]: Heya, thanks for the patch! silly mistake, I wonder when I introduced it :/ Don't use variable names and C code in commit messages, use English: "When copying a file we weren't setting properly whether it was skipped or not"
Review of attachment 347265 [details] [review]: Thanks for the patch! The UX of that is meh, but it's a right fix for now. Good thing to keep in the TODO list for the GSoC project about operations. In the commit message, if you mention "reverse" you should mention from where (commit id in this case). Here you can just "This commit subtract them from the total operation instead of adding to the transfer count" Feel free to commit after changing this to any of the cases.
Review of attachment 347264 [details] [review]: This one is also feel free to commit after changing the message.
Created attachment 347306 [details] [review] file-operations: reset skipped status of files on copy/move When copying or moving a file, the file’s “skipped” status is not reset, resulting in cases where the code starts skipping all files. In such cases the progress popover will show the finished operation count higher than the total amount of files.
Created attachment 347307 [details] [review] file-operations: change skip behavior Currently, when skipping files, they are added to the transfer counts, which results in inflated progress information. This commit makes the code subtract from the total counts, instead.
Attachment 347306 [details] pushed as b5ebb70 - file-operations: reset skipped status of files on copy/move Attachment 347307 [details] pushed as f80af50 - file-operations: change skip behavior
*** Bug 652218 has been marked as a duplicate of this bug. ***