GNOME Bugzilla – Bug 768455
Nautilus doesn't always update the number of files count
Last modified: 2021-06-18 15:52:34 UTC
Hello, Nautilus, in some cases, doesn't update the number of files count in the "size" column in the list view. Here is the use case that I'm aware that show it : - Open Nautilus in your /home, create a file with "touch" in a terminal in the "Downloads" folder, the count is not updated for the "Downloads" folder. - Take the previous exemple, go to a directory, like "Documents", go back to your /home, the count is still not updated for the "Downloads" folder. That can lead to some problem. The most annoying is obviously deleting a folder thinking it's empty whereas it's absolutely not empty. Thanks !
Created attachment 331421 [details] [review] files-view: monitor item count for subdirectories In Nautilus, only the directory currently displayed is monitored for changes in number of items. Therefore, Nautilus is not aware if subdirectories have files added or removed from them and can display information that is not synchronised with the file system. In order to fix this, monitor subdirectories for changes in the item count attribute.
Thank you Razvan, that was fast ! Will the patch work if, let say, I'm in a directory like "Documents", I create a file with "touch" in the "Downloads" folder and I go back to ~/ ?
(In reply to jeremy9856 from comment #2) > Thank you Razvan, that was fast ! > > Will the patch work if, let say, I'm in a directory like "Documents", I > create a file with "touch" in the "Downloads" folder and I go back to ~/ ? Hello! Yeah, just tested it and it works. Thanks for reporting the bug :)
Awesome ! I'm eager to test this.
Review of attachment 331421 [details] [review]: Hey Razvan, thanks a lot for the patch! I have two objections to the overall patch, one of them big: 1- I wonder if this should be part of the directory itself instead of the view. Probably not, so this is fine. 2- The important one... with this patch we are going to quickly reach the 1024 monitor limit, and that can prevent more important things than the count of files to work properly. So I'm pondering to wontfix this bug. ::: src/nautilus-files-view.c @@ -3962,2 @@ view = NAUTILUS_FILES_VIEW (callback_data); - unrelated change @@ +4030,3 @@ static void +monitor_subdirectories_item_count (NautilusDirectory *directory, + NautilusFilesView *view) can you flip the order of the parameters? @@ +4178,3 @@ + + g_hash_table_remove (view->details->monitored_subdirectories, directory); + g_list_foreach (subdirectories, why no disconnect here? You could have connected to the done-loading signal no?
Cosimo, I'm afraid of monitoring subdirectories of the current directory and reach the monitor limit, preventing more important things to work than the count of files. What do you think?
I really hope that won't be a 'wontfix'. Since others do it well there is a way to do it in Nautilus too. What about instead of monitoring a lot of folders just refresh the count of the visible folders when the user change directory ?
A nice thing to add too is the ability to autorefresh the count of visible folders when a file is created with an other app. Let say Nautilus is opened in ~/, you create a file in "Documents" with gedit, the count should be incremented automatically without the need, like now, to manually trigger the refresh feature.
"Since others do it well there is a way to do it in Nautilus too." We take care about all kind of setups, even those that has more than 8000 folders per directory. Nautilus is the most used in enterprise setups, we need to supports those. "A nice thing to add too is the ability to autorefresh the count of visible folders when a file is created with an other app." This is not how the file system and the kernel works. There is only one way, monitors, and the kernel has a limit.
(In reply to Carlos Soriano from comment #9) > We take care about all kind of setups, even those that has more than 8000 > folders per directory. Nautilus is the most used in enterprise setups, we > need to supports those. Of course, but I'm pretty sure that Dolphin can handle these cases without hassle. We just need to find a way to make it work even for those cases. It's a pretty important problem that can lead to data loss. Can you imagine that, your file manager, that you trust, says that this folder contain 0 item whereas it's totally false, you delete it and you just have lost a lot of work ! I'm not sure the enterprise users will appreciate, let alone the normal users. > "A nice thing to add too is the ability to autorefresh the count of visible > folders when a file is created with an other app." > This is not how the file system and the kernel works. > > There is only one way, monitors, and the kernel has a limit. I heard about inotify, does this can work for autorefreshing the count of visible folders when a file is created ? At least we should refresh the count of the visible folders when the user change directory.
(In reply to jeremy9856 from comment #10) > (In reply to Carlos Soriano from comment #9) > > We take care about all kind of setups, even those that has more than 8000 > > folders per directory. Nautilus is the most used in enterprise setups, we > > need to supports those. > > Of course, but I'm pretty sure that Dolphin can handle these cases without > hassle. We just need to find a way to make it work even for those cases. > It's a pretty important problem that can lead to data loss. Can you imagine > that, your file manager, that you trust, says that this folder contain 0 > item whereas it's totally false, you delete it and you just have lost a lot > of work ! I'm not sure the enterprise users will appreciate, let alone the > normal users. I might be wrong, but Dolphin will fail in this case too if they monitor all sub directories. Really there is a limit on the kernel, we can do nothing about that. There is no point on discussing how much we have to trust our file manager or these kind of arguments. What we want is to not fail in any important case because of the limit of monitors, and the count of files is less important than the current directory or other directories in the filesystem. You can check the code and documentation and report back if you have a factual idea on how to do it. > > > "A nice thing to add too is the ability to autorefresh the count of visible > > folders when a file is created with an other app." > > This is not how the file system and the kernel works. > > > > There is only one way, monitors, and the kernel has a limit. > > I heard about inotify, does this can work for autorefreshing the count of > visible folders when a file is created ? At least we should refresh the > count of the visible folders when the user change directory. Monitors use inotify. That's where the limit is. Refreshing count of visible folders when changing the directory is possible, and it might be an acceptable workaround.
Carlos, you're right, monitoring every subdirectory would quickly push us over the FD limit. When using inotify there are no alternatives to setting up recursive watches, to the best of my knowledge. However, we could possibly investigate using the fanotify API from the kernel for this use case, which, while not being a complete replacement for inotify (yet?) does not have that limitation. I would not be comfortable with a patch that simply always adds the recursive inotify watches without further investigation.
(In reply to Carlos Soriano from comment #11) > Really there is a limit on the kernel, we can do nothing > about that. I'm totally ok with that. > There is no point on discussing how much we have to trust our > file manager or these kind of arguments. I don't want to continue on this but since I almost loss data because of this I can say that the item count can't be trusted for now. That the point of the bug report. Nautilus must not report false infos. > What we want is to not fail in any important case because of the limit of > monitors, and the count of files is less important than the current > directory or other directories in the filesystem. > You can check the code and documentation and report back if you have a > factual idea on how to do it. The important thing and the whole point of the bug report in fact is not to be always accurate but it is to not report 0 when it's not the case. Maybe we can add a limit to the monitoring and simply report something like "100+" when there is too much item that can't be all counted and report something like "n/a" when there is too much subfolder that can't be all counted ? And maybe dynamically replace "n/a" while the user scroll ? > Refreshing count of visible folders when changing the directory is possible, > and it might be an acceptable workaround. That feature will be great and almost fix the problem ! The autorefresh is important too. Since Nautilus use inotify it's possible to have autorefresh. Coupled with the limits we can implement to avoid any problem. It's better to have something like "n/a" for some cases rather than a false info like 0.
(In reply to Cosimo Cecchi from comment #12) > Carlos, you're right, monitoring every subdirectory would quickly push us > over the FD limit. When using inotify there are no alternatives to setting > up recursive watches, to the best of my knowledge. > However, we could possibly investigate using the fanotify API from the > kernel for this use case, which, while not being a complete replacement for > inotify (yet?) does not have that limitation. > > I would not be comfortable with a patch that simply always adds the > recursive inotify watches without further investigation. Thanks Cosimo for the answer! I will take a deeper look at fanotify, looks interesting. I agree with you on not allowing a patch without taking care of this.
(In reply to jeremy9856 from comment #13) > (In reply to Carlos Soriano from comment #11) > > Really there is a limit on the kernel, we can do nothing > > about that. > > I'm totally ok with that. > > > There is no point on discussing how much we have to trust our > > file manager or these kind of arguments. > > I don't want to continue on this but since I almost loss data because of > this I can say that the item count can't be trusted for now. That the point > of the bug report. Nautilus must not report false infos. Ok fair enough, I'm fine removing the count information altogether until we have a more reliable way to keep them updated. But I'm not sure people will like it, it was implemented even with those limitations for a reason. But I guess that's not a excuse to not remove it now until we have something better.
(In reply to Carlos Soriano from comment #14) > Thanks Cosimo for the answer! I will take a deeper look at fanotify, looks > interesting. > I agree with you on not allowing a patch without taking care of this. From the fanotify man page: "Adding, removing, or changing children of a marked directory does not create events for the monitored directory itself." I guess we are not lucky with this either.
I don't know if removing the feature is a good idea, even if it's temporarily. The majority of people won't know that it's temporarily and will again complain that, I quote, "Gnome devs keep removing features !!". The feature is still useful if you know that you sometimes have to manually refresh the view to update the count. That's what I do for now. I prefer to keep it that way until it's improved but it's your call :D
(In reply to Razvan Chitu from comment #16) > (In reply to Carlos Soriano from comment #14) > > Thanks Cosimo for the answer! I will take a deeper look at fanotify, looks > > interesting. > > I agree with you on not allowing a patch without taking care of this. > > From the fanotify man page: > "Adding, removing, or changing children of a marked directory does not > create events for the monitored directory itself." > > I guess we are not lucky with this either. ugh :( I don't get it then...
(In reply to jeremy9856 from comment #17) > I don't know if removing the feature is a good idea, even if it's > temporarily. The majority of people won't know that it's temporarily and > will again complain that, I quote, "Gnome devs keep removing features !!". But you convinced me in the first place it was too dangerous to display wrong information :/ I'm confused. > The feature is still useful if you know that you sometimes have to manually > refresh the view to update the count. That's what I do for now. > > I prefer to keep it that way until it's improved but it's your call :D Let's keep it for now.
(In reply to Carlos Soriano from comment #19) > Let's keep it for now. I think it's the right decision. Thanks ! > But you convinced me in the first place it was too dangerous to display > wrong information :/ > I'm confused. That's right, it's dangerous to display wrong information but the feature has been here for years and people don't seem to complain that much. If you remove it, you will hear them quite well I think. A great ending will be a "quick" fix, so nobody will notice there was a problem :D
*** Bug 773416 has been marked as a duplicate of this bug. ***
Hello Carlos, Do you have some news about this issue ? Thanks !
See your comment 20, quoted "I think it's the right decision" and I agreed. Nothing else to do here, apart of waiting for the unlikely case the kernel makes monitors across the filesystem work :)
(In reply to Carlos Soriano from comment #23) > See your comment 20, quoted "I think it's the right decision" and I agreed. > > Nothing else to do here, apart of waiting for the unlikely case the kernel > makes monitors across the filesystem work :) Well there was an unfinished talk about fanotify and I was wondering if there was more infos. A workaround was possible it seem as we talk in comment 11 ? > Refreshing count of visible folders when changing the directory is possible, and it might be an acceptable workaround.
Ah yeah, indeed, what's needs to be done is to invalidate the count metadata of the files. Actually this should work fine, for any folder that is cached/uncached, but the common folder like Donwloads etc are always cached. Same for bookmarked folders. So basically that leads us to invalidate the count metadata of all the files when changing location. Still not sure where that could be done..
Carlos, this means the patch in "review" status should get a "rejected" status because of the 1024 limit, is that right?
Review of attachment 331421 [details] [review]: Rejecting since we can reach the limit of 8800 monitors.
*** Bug 754676 has been marked as a duplicate of this bug. ***
*** Bug 501094 has been marked as a duplicate of this bug. ***
GNOME is going to shut down bugzilla.gnome.org in favor of gitlab.gnome.org. As part of that, we are mass-closing older open tickets in bugzilla.gnome.org which have not seen updates for a longer time (resources are unfortunately quite limited so not every ticket can get handled). If you can still reproduce the situation described in this ticket in a recent and supported software version of Files (nautilus), then please follow https://wiki.gnome.org/GettingInTouch/BugReportingGuidelines and create a new ticket at https://gitlab.gnome.org/GNOME/nautilus/-/issues/ Thank you for your understanding and your help.