GNOME Bugzilla – Bug 573293
Locking APIs for GIO
Last modified: 2018-05-24 11:47:13 UTC
As discussed at the last GTK+ developer IRC meeting, an API for locking would be useful. As far as I can see, two different use cases needs to be catered for: 1) Locking upon opening, for the entire duration of the file being open. This maps to using a "sharing mode" when opening a file for SMB (i.e. on POSIX when using the Samba client library) and Win32, and O_EXLOCK on BSD and MacOSX. There is no support for this in Linux or POSIX in general as far as I know. 2) Advisory byte range or "record" locks. Can be set and removed on an open file freely while it is open. Such functionality is present for local files on all platforms as far as I know, and in SMB (and NFS, but that looks like local files anyway). Possibly the API could be restricted to just locking/unlocking all of a file in case there is some relevant platform after all that lacks byte range locks. Unfortunately it would not be enough to just do the 2) part, because proper interoperability with typical native Windows apps when accessing documents on shared SMB servers requires "sharing mode" style locking. A typical use case would be OOo on Linux using GIO to open a document on an SMB server and wanting to lock out MS Office from opening the same document. Once GIO has the API, obviously gvfs would then need an implementation for SMB at least. Maybe even for HTTP (think WebDAV).
I don't think its byte range locking is all that interesting. For the kind of apps that the GIO API targets (user level apps reading user "document" files) the major use of locking would be to protect against any other access to the file. There is also the question if we want to have a mandatory or cooperative locking. I don't know for sure, but my guess is that if we want a broader set of backend support (i.e. works on more than just local files) we need to specify cooperative locking.
(In reply to comment #1) > There is also the question if we want to have a mandatory or cooperative > locking. I don't know for sure, but my guess is that if we want a broader set > of backend support (i.e. works on more than just local files) we need to > specify cooperative locking. I think we have to explicitly say that it's undefined whether you get a mandatory or cooperative lock: if you get a gio lock, then no one else can *lock* the file, but it's explicitly undefined whether or not they can do anything else to it if they don't try to lock it first. (Saying it's a cooperative lock would imply that they definitely could.)
I think that for the 1) use case, what is needed is a way to open a file for reading *and* potential rewriting with locking (i.e. "share modes" in the case of SMB and probably WebDAV, and on Win32 and BSD also local files), without any need to close and re-open. First idea: GFileRewritableInputStream *g_file_read_for_update (GFile*, GCancellable*, GError**). and GFileRewritableInputStram would be a subclass of GFileInputStream, with a new rewrite method: GFileOutputStream *g_file_rewritable_input_stream_rewrite (GFileRewritableInputStream*, GError**). The semantics would be that until the rewrite method is called, a GFileRewritableInputStream would act like a GFileInputStream. When rewrite is called, the underlying file is truncated to zero size, and the returned GFileOutput can then be used normally. The GFileInputStream would be unusable after the rewrite method has been called. Something like this would presumably be enough for the kind of apps Alex mentions in comment #1, but should the truncating to zero size when rewrite is called be optional, to allow just partial updating of the file? And what about true random read and write, i.e. having both an input and output stream at the same time for one file?
Why exactly would apps want to do something like that (read and rewrite) by the way? Whats the typical use case? read/write stream support is bug 533547.
The use case is "I'm going to open this document, and then the user might save it one or more times, and then eventually close it, and during that time, no one else can write to the file". (More concretely, the use case is "make OOo's access to SMB files interoperate properly with MS Office [and OOo on Windows].") Given the way the smb backend currently works, Tor's suggestion would probably involve the least rewriting, but if we move around where the actual smbc_open and smbc_close calls occur, we could make it so you just call g_file_lock() at the beginning, then g_file_read(), any number of g_file_append_to()s or g_file_replace()s, and finally g_file_unlock(). In the SMB case, when the backend got a lock() call, it would open the file in the correct mode and deny_mode, and stash the file handle in the backend somewhere. Then the open_for_read() and replace() calls would just return that existing filehandle rather than creating a new one. close_read() would be pretty much a no-op, and close_write() would just do a flush. Then finally in the unlock() call it would actually close the file. (I'm not sure how you implement truncation-proof replace() while continuing to hold the lock, but presumably there's some way to do it?) Other backends would work differently. IIRC, in WebDAV, the lock() call would translate to a LOCK request, which would then store the returned lock token, unlock would do an UNLOCK and delete the previously-stored lock token, and nothing else would need to change at all, other than making sure to add the lock token to the HTTP headers of relevant requests.
Thats what i expected. But the rewriteable stream thing seemed to imply a shorter span of the lock and rewriting only once. The API you propose seems more like what I expected.
Its unfortunate that g_file_read() doesn't have a flags argument, because the best api would be to open it with a LOCK flag and then have a separate unlock call (when you close the document or whatnot).
Is the GIOStream work recently added related to the use cases in this bug? Before setting the GIOStream API in stone, wouldn't it be good to make provisions to later allow for the features discussed here, like adding flags arguments to relevant methods?
Tor: The stream interface is a very generic abstraction, just like GInputStream and GOutputStream. Its used for sockets and files alike. It should not have anything file specific in it. The usecase for it is if you want to open a file for both writing and reading at the same time (which is not previously supported in gio, and is still only supported for local files in glib master).
Also, the locking APIs should not depend on GIOStream. read+write is not implementable on things like WebDAV (it has only GET and PUT), which *does* have locking primitives.
I think Dan's suggestion in comment #5 is potentially confusing, as it would mean that after a g_file_lock() call, a GFile object would have this hidden property of being "locked", and what that then actually means would depend on backend. It would break the basic idea that a GFile object represents just an identifier for a file, as the documents say. As the lockedness would be a property of a GFile object, it is different from theGFileAttribute concept which relate to properties of the underlying real file / WebDAV / whatever object, not the GFile object itself. But I am prepared to be convinced that this is not a problem...
Do people have any fresh thoughts about this? I have planned to do some work on this now during Novell's "Hack Week".
Created attachment 163687 [details] [review] Initial idea OK, so I did not get as far as I would have wished. This patch is a start. I introduce a new enum, GFileReadFlags, and some new values to the GFileCreateFlags. The new GFileReadFlags and GFileCreateFlags values contain a hint to use "share modes" when opening/creating the file, and then what share mode to use. I then add one new function: g_file_read_with_flags() which is otherwise like g_file_read() except that it has a GFileReadFlags argument, too. (Obviously also a g_file_read_async_with_flags() would be needed. Didn't do that yet, but it would follow the same pattern.) How does this approach look?
Created attachment 163747 [details] [review] Updated patch More work. Adds g_file_read_async_with_flags(), and code for the handling of share modes in the various open-for-writing APIs. The readwrite APIs still need to be handled. (Add g_file_open_readwrite_with_flags() and g_file_open_readwrite_async_with_flags(), handle share mode flags in the create_readwrite and replace_readwrite code.) Or something like that... Is there any less intrusive but still clean way to do this?
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/glib/issues/201.