GNOME Bugzilla – Bug 775600
Gio.VolumeMonitor.get() is slow when many automounted shares are mounted
Last modified: 2017-06-06 09:08:13 UTC
When using Gio to load many files over https, gvfs never unmount files: python example: >>> from gi.repository import Gio >>> f=Gio.File.new_for_uri('http://www.google.fr') >>> (s, d, e) = f.load_contents() >>> f.unmount_mountable_with_operation(Gio.MountUnmountFlags.FORCE, None, None, None, None) >>> f.eject_mountable_with_operation(Gio.MountUnmountFlags.FORCE, None, None, None, None) org.gtk.vfs.Daemon /org/gtk/vfs/mounttracker org.gtk.vfs.MountTracker.ListMounts: [Argument: a(sossssssbay(aya{sv})ay) {[Argument: (sossssssbay(aya{sv})ay) ":1.146", [ObjectPath: /org/gtk/vfs/mount/1], "", "http:uri=http%3A%2F%2Fwww.google.fr", "", "", "", "", false, {0}, [Argument: (aya{sv}) {47, 0}, [Argument: a{sv} {"type" = [Variant(QByteArray): {104, 116, 116, 112, 0}], "uri" = [Variant(QByteArray): {104, 116, 116, 112, 58, 47, 47, 119, 119, 119, 46, 103, 111, 111, 103, 108, 101, 46, 102, 114, 0}]}]], {0}]}] Over time, gvfs gets full of files and slow down every gtk apps using Gio.
When calling Gio.VolumeMonitor.get(), this create following dbus message: method call time=1480887682.384720 sender=:1.105 -> destination=:1.93 serial=5 path=/org/gtk/vfs/mounttracker; interface=org.gtk.vfs.MountTracker; member=ListMountableInfo And this list all entries in gvfs cache and it's really slow when having many entries.
Will workaround the issue in Lollypop by calling this: org.gtk.vfs.mountpoint_http /org/gtk/vfs/mount/1 org.gtk.vfs.Mount.Unmount org.gtk.vfs.mountpoint_http /org/gtk/vfs/mount/1 0
https://github.com/gnumdk/lollypop/blob/master/src/utils.py#L23 Here the workaround, every time I call Gio.File.load_contents() on an http uri, I call kill_gfvsd_cache(). This way, no more slow down in Gtk apps using Gio.VolumeMonitor.
Thanks for taking the time to report this. This particular bug has already been reported into our bug tracking system, but please feel free to report any further bugs you find. *** This bug has been marked as a duplicate of bug 509609 ***
Ok, let's reopen this bug and change the bug summary a little bit. The bug is rather about that Gio.VolumeMonitor.get() is slow when there is too many non-user-visible mounts (i.e. automounted shares like http). The volume monitor should not need info about automounted shares probably, so...
It seems that ListMountableInfo is called only from g_daemon_vfs_init (fill_mountable_info respectivelly), which should be called only once... So this should be only problem when starting some GTK+ app, am I right? Hmm, the volume monitor doesn't need info about those non-user-visible mounts, but the client part needs it... What you mean with slow?
I mean calling Gio.VolumeMonitor.get() hangs for 5 to 10 seconds (depends how many http shares are mounted).
Created attachment 341938 [details] dbus-monitor output Here what is happening on DBus while calling Gio.VolumeMonitor.get()
Hmm, not sure how the python bindings work, but I think that this should be called only once, e.g. when g_volume_monitor_get() is called first time from app (I hope python has something like static variables in C)... and I am afraid we can't do much with it (except the autounmounting feature)...
Calling Gio.VolumeMonitor.get() doesn't hang on second call... BTW, if you add an automount feature, it should auto-unmount as soon as possible.
I wonder how many mounts do you have that it causes slowdown about 5-10 seconds. I made a simple script which just calls Gio.VolumeMonitor.get(). It takes about 1 second for 5000 mounts. So I am not sure whether this is a real cause for the hangs...
Created attachment 342936 [details] [review] Do not sent user invisible mounts if not needed I've looked at it again and realized that we really do not need the list of the user invisible mounts in the client, so I propose this patch, but it needs to be carefully tested...
Lollypop is creating a db from Itunes charts: - Reading Itunes Charts (1 request) - Gettings tracks from itunes API (many requests) - Searching track on the web (many requests) So more than 5000 requests I guess... Going to test your patch ;)
Ok, your patch is working here, will do more testing.
Tested all the week end, no regression here... And I'm using Gvfs a lot (smb, sftp, davs, ...)
> So more than 5000 requests I guess... I am afraid of that http backend was not designed for such use, but with this patch and some kind of automounting it should work well... > Tested all the week end, no regression here... And I'm using Gvfs a lot > (smb, sftp, davs, ...) Thanks for the testing...
https://github.com/gnumdk/lollypop/blob/master/src/lio.py I'm currently checking for your patch for using python urllib or Gio. Should I remove Gio code? Is there another API in GNOME allowing me to download items over http?
(In reply to Cédric Bellegarde from comment #17) > https://github.com/gnumdk/lollypop/blob/master/src/lio.py > I'm currently checking for your patch for using python urllib or Gio. > > Should I remove Gio code? Is there another API in GNOME allowing me to > download items over http? Use libsoup. It will integrate with the GNOME, KDE, etc. proxy settings as well.
Cool, thanks!
Ondrej: You should not unconditionally add a new argument to the dbus call. That will break with older versions of the client library, for instance if you're running a flatpak with an older gnome runtime. You need to add a new method and handle a no-such-method error and fall back to the older one.
Thanks for the comment. I worried about it, will add fallback as you suggested.
Created attachment 350198 [details] [review] Do not sent user invisible mounts if not needed g_volume_monitor_get() might be really slow if there is too many mounts, because the list of the mounts is send over D-Bus. It can simply happen due to user invisible mounts, e.g. http. User invisible mounts are ignored by the volume monitor, so it is useless to send them over D-Bus. Improve the D-Bus API and don't send the user invisible mounts if it is not needed.
Attachment 350198 [details] pushed as ceb0daa - Do not sent user invisible mounts if not needed