GNOME Bugzilla – Bug 741915
Builder is slow and deadlocks hard when working over SSH mounted partition
Last modified: 2015-01-05 19:26:03 UTC
At work we have to keep code on our workstations so when working off my laptop I usually ssh mount my workstation and use gedit to edit my code. I tried to use gnome-builder but it does not like working over an ssh mounted file system. First and more critical is the deadlocks: * When searching for a file with the global search bar the application SOMETIMES becomes completely unresponsive and has to be killed on the command line * When selecting a file to open on an ssh mounted partition the application ALWAYS becomes completely unresponsive and has to be killed on the command line I'm guessing either the latency is either causing a race condition or the app is trying to perform file operation that is not supported by the ssh file system. I did not see any output in the terminal. Now for the slowness: * When searching for a file the app becomes laggy and unresponsive for short periods of time This is a common thread with IDE's which do intelligent searching. gEdit doesn't have issues here because it only hits disk to load a directory or load/save files. Some suggestions are to put file lookups in a thread, cache the git index in memory or on the local file system (i.e. the tempdir which is almost guaranteed to be local) and don't perform any unnecessary reads or writes inside the project directory proper. If you need to read from files in the project directory cache the hell out of those reads. Chances are files are not going to change while I'm searching them. To reproduce: 1) mount a filesystem with a git repo on it 2) Go to that directory (is mounted under nautilus it will be put in /run/user/250565/gvfs/sftp:host=<host>/<path_to_git_dir_on_host>) 3) run gnome-builder 4) search for file and try to open it
+ Trace 234444
Stacktrace from GDB. Looks like it is still running but running a bunch of git commands that may be extremely slow over a connection with high latency.
So I let this run and went to lunch. When I came back the file was loaded. This seems to me an issue with ggit_tree_get_by_path being really slow on latent file systems.
Makes sense. I did this synchronously because I wasn't sure of the thread safety of libgit2-glib (although libgit2 underneath should be thread safe). We do the blog comparision in a thread at least (to determine the diff), just the repository discover and blog loading that is done synchronously. I'll see if I can come up with something quick, otherwise I'll look next week.
s/blog/blob
I even had a comment in that code that mentioned that this would occur :) I've pushed a fix for some of this to master. In particular, we load git data in 2 places. The diff renderer in the source editor, and the search index. The global search index builds a fulltext index of all the filenames. I think that was done all asynchonrously. I need to double check. The diff renderer was not, it was loading repository info and gitblob information synchronously. That is now performed in a thread using GTask. I'm still not happy with the slowness loading a new file (it seems like its blocking when using ctrl+shift+o and i need to track down why). Anyway, just pushed a patch that should make some of this better.
Created attachment 293312 [details] [review] make diff renderer use async git operations
Created attachment 293314 [details] [review] make snippet context creation lazy (was calling git config user.email) This patch should speed up startup when running from an sshfs mounted directory. We were creating the snippet context on snippet creation which would hit the filesystem when running 'git config user.email'. Pretty braindead. Startup is much faster for me when running from a sshfs directory now.
Created attachment 293319 [details] [review] avoid hitting disk when generating search results We were hitting the disk once to read refs and master file of the git repository. Not necessary since we can cache that when loading the GgitRepository. This makes searching when using sshfs as fast as local for me.
John, I'm still not happy with the state of the open dialog (it looks like it hangs a bit for me). But I'm curious if things are better for you now otherwise.
Much better. It still lags while loading the index but doesn't hard freeze and once I select a file the latency is what I would expect for loading over a network. A note - keeping latency issues in check with good caching and only writing to local disks for tmp files and internal indexes would be a killer feature as most modern IDE's don't handle this well causing many people to revert to vim/emacs when doing any remote editing. Sorry for my own latency. I've been on vacation. Fixed for me for now. I will open up a new bug if it becomes too slow again. Thanks.
Sounds good. I wonder what we could do in terms of replicating with a traffic shaping script using `tc'. For now, I've just been testing with a hosted VM.
Created attachment 293869 [details] [review] workbench: load git repository asynchronously This slipped through the cracks when fixing loading in sshfs directory.
Comment on attachment 293869 [details] [review] workbench: load git repository asynchronously Attachment 293869 [details] pushed as e66c6fa - workbench: load git repository asynchronously Found another one that was slowing down application startup on sshfs. Looks like loading the git index was falling through the cracks when loading the search bar.