GNOME Bugzilla – Bug 522314
gio inotify missing file handling polling suboptimal
Last modified: 2018-05-24 11:21:56 UTC
Currently if you add a watch for a path which doesn't exist, code in the inotify-missing.c file adds a timeout to poll for it: http://svn.gnome.org/viewvc/glib/trunk/gio/inotify/inotify-missing.c?view=markup I think a smarter algorithm would be: 1) Check if given path exists, if so goto success 2) Check whether the parent exists. If not, recurse -> 1 3) Check if the given path exists, if so remove previous watch, goto success In the watch handler on the parent, we check to see if the target exists. If so, we remove the parent watch and go back down the chain.
I guess this is needed to really fix the problem explained here: http://bugzilla.gnome.org/show_bug.cgi?id=487375#c19 My goal in that bug report was to remove the constant polling from gnome-panel, the screen saver, gedit and many other gtk+ apps. You can see this problem if you use powertop on an idle gnome session and then use strace to see what is going on. Cheers.
Hi William, I agree with Colin that polling for nonexistent files is suboptimal. However, the real question is why apps like gnome-panel, gnome-screensaver, gedit, etc would constantly monitor files that don't exist. In the past I had debugging code in gnome-vfs that would print out the missing list - it was empty (as it should be.) with a fully running GNOME session. Specifically speaking about #487375 - ~/.recently-used.xbel should always exist and thus is not impacted by this bug. HTH, John
I just tested this on a Fedora 10 Preview Live USB and gnome-panel is doing inotify_add_watch() every 4 seconds to the following files: /home/liveuser/.config/menus/server-settings-merged /etc/xdg/menus/server-settings-merged /home/liveuser/.config/menus/system-settings-merged /etc/xdg/menus/system-settings-merged /home/liveuser/.config/menus/preferences-merged /home/liveuser/.config/menus/settings-merged /home/liveuser/.config/menus /home/liveuser/.config/menus/applications-merged None of them exists. Is this a Fedora specific bug? can you see this using strace on gnome-panel? what distribution are you using?
No, this is not a fedora specific bug. This is the menu spec in action.
Confirming. This wakes up the panel quite often (as can be seen with powertop).
See bug 476938 for this same issue fixed in gamin before we switched to gvfs for file monitoring.
Hello there, any news about this? In Fedora 12 Rawhide clock-applet is constantly polling for ~/.config/ibus/bus which doesn't exists. And gnome-panel still looks for files that are not present. Is there any plan to improve the watching of non existent resources? it would be nice to avoid those wake ups.
This can still be seen if you strace gnome-shell and you don't have all of the possible menu directories: 1739 1351791285.714723 inotify_add_watch(11, "/home/smcv/.config/menus/gnomecc-merged", IN_MODIFY|IN_ATTRIB|IN_CLOSE_WRITE|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF|IN_UNMOUNT|IN_ONLYDIR) = -1 ENOENT (No such file or directory) 1739 1351791285.714892 inotify_add_watch(11, "/etc/xdg/menus/gnomecc-merged", IN_MODIFY|IN_ATTRIB|IN_CLOSE_WRITE|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF|IN_UNMOUNT|IN_ONLYDIR) = -1 ENOENT (No such file or directory) 1739 1351791285.715063 inotify_add_watch(11, "/home/smcv/.config/menus", IN_MODIFY|IN_ATTRIB|IN_CLOSE_WRITE|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF|IN_UNMOUNT|IN_ONLYDIR) = -1 ENOENT (No such file or directory) 1739 1351791285.715317 inotify_add_watch(11, "/home/smcv/.config/menus/applications-merged", IN_MODIFY|IN_ATTRIB|IN_CLOSE_WRITE|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF|IN_UNMOUNT|IN_ONLYDIR) = -1 ENOENT (No such file or directory) 1739 1351791285.715591 inotify_add_watch(11, "/etc/xdg/menus/applications-merged", IN_MODIFY|IN_ATTRIB|IN_CLOSE_WRITE|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF|IN_UNMOUNT|IN_ONLYDIR) = -1 ENOENT (No such file or directory) 1739 1351791285.715868 inotify_add_watch(11, "/home/smcv/.config/menus", IN_MODIFY|IN_ATTRIB|IN_CLOSE_WRITE|IN_MOVED_FROM|IN_MOVED_TO|IN_CREATE|IN_DELETE|IN_DELETE_SELF|IN_MOVE_SELF|IN_UNMOUNT|IN_ONLYDIR) = -1 ENOENT (No such file or directory) (This is on Debian experimental with menu-xdg installed; exact paths may vary.)
Created attachment 227905 [details] inotify-test.c This test program can reproduce the bug on gio/inotify.
Created attachment 228165 [details] [review] 0001-gio-inotify-optimize-missing-files.patch [PATCH] gio inotify: optimize missing files When monitoring a file in a non-existent directory, gio used to set up a timer every 4 seconds to check if the file is created. This wakes up the process unnecessarily. This patch watches one of the parent directories instead. The missing list and the timer are still used when the watched directory is deleted while it is being watched. But it is only one wake-up, so it does not wakes-up every 4 seconds as before.
I seem to recall there being an argument about this at the time the code was written... I always thought that it should be done this way, but the original author of inotify told me that this way was better. I'm not sure I ever believed him...
There are some general issues with this method. First of all, any particular parent "directory" may be a symlink. In the example /a/b/c used in the code, say /a/b is really a symbolic link to /d/e, we should then look for events in the /d/e directory, looking for the creation of a file /d/e/c which would let us get inotifies on the target directory. Of course, the symlink target might also not exist yet. Say /d exists, but not /d/e. Then we need to watch /d for the appearance of an e file, etc, recursively. Then there is the issue with parents changing. In the case where we're e.g. monitoring say /a/b/c because /a/b/c/d is not available yet, then /a is renamed to /foo. We are now watching /foo/b/c, but are not told about this (as the inode for "c" didn't change. This is also interlinked with the symlink case. In the above example where /a/b was a symlink we started monitoring /d/e, but we also need to monitor the symlink to make sure it doesn't change or is deleted. This problem kinda already exists with the current inotify monitor, since once you're monitoring a directory we don't know if any higher level paths are renamed while we're monitoring. But the issue seems more drastic in this case as a) higher level paths are inherently changing when this operation is used b) you might in the end be completely missing the appearance of a directory with the target pathname. In the existing case you're at least monitoring a directory that once had the expected pathname. It seems to me that a proper implementation of this must watch *all* existing parent directories, as well as any symlinks targets and their parents (the ones that exists at least) in parallel to be able to catch the appearance of the target directory. Of course, in the problematic cases above we're not really *expecting* the parent directories to change, so all these complexities (except part of the symlink problems) are perhaps overkill to manage. Glib is a general purpose library though, so i'm not sure we can just skimp on behaviour because the panel doesn't need it. Maybe we can keep the current stat:ing behaviour by default but add a flag to opt-in to a less "correct" implementation in case you expect the parent dirs to be static.
Created attachment 232904 [details] [review] [PATCH] gio inotify: optimize watching non-existent directories Update the patch with: - a flag (GFileMonitorFlags) to opt-in to this implementation - rebase on git master.
Note to self: revisit bug 736350 when fixing this.
There isn't problem only with missing files, but also with paths without read access. See Bug 737371 for more details.
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/131.