GNOME Bugzilla – Bug 555436
use stable URIs for DNS-SD services
Last modified: 2018-09-21 16:29:02 UTC
When browsing stuff in network:/// and clicking an icon stemming from DNS-SD, the IP address is used. This a) kinda defeats the purpose of DNS-SD; and b) it will become a bigger problem with the connected servers featuring scheduled for GNOME 2.26 (e.g. we want to provide an easy way to turn a transient mount into a persistent connected-server volume).
Created attachment 120140 [details] [review] proposed patch This patch fixes it for me.
Didn't Lennart say to use avahi_nss_support() to figure out whether hostnames work or not ?
Yes, make sure to use avahi_nss_support(). Only when that call returns success (i.e. 1) gethostbyname() and friends know how to deal with mDNS domain names. Hence the code should be like this: if (avahi_nss_support() > 0) { /* Use the code from davidz */ } else { /* Use the old code */ }
Created attachment 120150 [details] [review] updated patch Nice catch, thanks Lennart and Matthias. Here's an updated patch.
I am happy!
So even with the patch in comment 4 there's a couple of problems 1. The name of the (transient) mount is still something like "WebDAV at quad.local" which is kinda an eyesore given that the user most probably got there by double clicking an icon with e.g. the name "davidz's public files at quad.fubar.dk". 2. The URI is of the form dav://quad.local:12345. This is problematic because it's not bookmarkable. The next time the DAV server (e.g. gnome-user-share) restarts (e.g. next login), it's listening at a different port number. Here's an idea. Since we've already declared that the gvfs URIs are private to gvfs (bug 530654 and bug 528670) and not supposed to be passed to non-gio apps [1] it's technically fine to extend the URI format for our own purposes. Now, suppose the dns-sd:// backend instead of handing out URIs of the form dav://guest@quad.local:12345/ instead gives you dav://guest@[resolve:dns-sd,_webdav._tcp,local,davidz%27s%20public%20files%20on%20x61.fubar.dk] and we teach the dav:// backend about resolving such URIs. Then we have all the information at hand we need - the name of the share: davidz's public files at quad.fubar.dk - the domain and the type I whipped up a proof of concept patch for doing this that I will attach; here's a screenshot of the end result (ignore the cluebar, that's just a local patch in my nautilus tree for now) http://people.freedesktop.org/~david/gvfs-stable-uri-for-dnssd.png In order to avoid adding a lot of async avahi code (there's apparently no synchronous API in Avahi (on purpose I believe)) to the dav:// backend, we introduce a new file attribute gvfs::resolved-uri on the dnssd file so it's easy to determine the resolved URI. Issues for discussion - This relies on the Service Name of the DNS-SD DAV server being unique I'm not sure we can actually rely on this (though for small networks it's probably fine). I can't, however, find anything better to use as an identifier. We can, however, add a TXT record to gnome-user-share representing a GUID of some form. Then we'd only provide such URIs if that property is available. - The URI is potentially ugly. I'm not sure this is a problem, other URIs we use (gphoto2, obex, etc.) are all really ugly too. And we don't really show them in the UI. So I'm not sure this matters. - It might complicate a bookmark editor if we have such URIs. People might want to edit them. - Some people might accuse us of embrace+extend for doing this. But I don't think that matters much because we've seen on several instances that URIs from different VFS systems aren't compatible. And even if they were, you'd need a lot of other plumbing to make the system as a whole work well (e.g. connection, cookie, secret sharing etc.). Anyway, just an idea and a few hours worth of hacking. Discuss. [1] : We enforce this by passing FUSE paths that, in gio apps get mapped back to the proper URIs. See http://blog.fubar.dk/?p=104 for more details.
Created attachment 120620 [details] [review] proof of concept patch This is just a rough patch demonstrating the idea. If we want to pursue this, we probably want to add utility API to the GVfs backend classes.
(oh, and of course this patch is wrong insofar it still hardcodes some things (apparently I still need to tear the resolved URI apart). But I think the patch demonstrates the idea, at least enough for discussion.)
I totally agree with using avahi_nss_support() to trigger using the domain name or not. This solves the problem for everything not using dynamic ports. However, I'm not sure about extending the URI space like that... Yes, gvfs uris are not 100% the same as other kinds of URIs, but the reason we do use them is because people understand them, and they are interoperable to like 90% or so. We should strive to minimize divergent extensions. If any extension it should be one that does not break other apps using dav:// (i.e. make them not load the uris we generate). We should instead do something like dav-sd://davidz%27s%20public%20files%20on%20x61%2efubar%2edk.local/
(In reply to comment #9) > I totally agree with using avahi_nss_support() to trigger using the domain name > or not. This solves the problem for everything not using dynamic ports. > > However, I'm not sure about extending the URI space like that... Yes, gvfs uris > are not 100% the same as other kinds of URIs, but the reason we do use them is > because people understand them, and they are interoperable to like 90% or so. > We should strive to minimize divergent extensions. > > If any extension it should be one that does not break other apps using dav:// > (i.e. make them not load the uris we generate). We should instead do something > like dav-sd://davidz%27s%20public%20files%20on%20x61%2efubar%2edk.local/ First, any kind of protocol can be announced using mDNS, gvfs actually already supports discovering FTP servers that way. So I don't think duplicating the URI scheme that way will do any good. Second, UPnP is another mechanism (similar to mDNS) for discovering resources, it's far from far fetched we want to support that in network:/// (look at all the awesome GUPnP stuff going on). The proposed mechanism ([resolve:...]) is extensible to supporting that (e.g. [resolve:upnp,name=whatever]). I think it would be more useful if we could propose a fd.o standard describing how to interpret the host part being of the form "[resolve:...]", maybe in a bigger fd.o documenting concerning interoperability of URIs in the desktop space. That's a daunting task in itself, possibly with very little value on investment. But it's probably the right thing to do. In the interim, unless I've convinced you to take the proposed patch, one resolution is that we use gvfs-dav://, gvfs-davs://, gvfs-ftp:// whenever we use the "[resolve:...]" form for the host name.
In DNS-SD a service instance is identified by a combination of service name + service type + domain, seperated by dots. i.e. something like "Lennart's Files._ftp._tcp.local". This triple can be parsed/created via Avahi's avahi_service_name_join() and avahi_service_name_split() functions. The triple is not a perfect identification for a service, since mDNS defines no global naming system -- only a local one. i.e. there is no way to distuingish a service "foo._ftp._tcp.local" on my home network and one by the same name on the redhat office network. But that is expected and OK behaviour. An Avahi browsing object (AvahiServiceBrowser) is generally used to find all those triples on the network. The callback for such a browser will be called for each triple and will also include the ifindex/proto of the network interface the service was found on. Usually this triple (along with the ifindex/proto) is then resolved with avahi_service_resolver_new() and will then result in hostname/ip address/port and other meta data. If the resolving happens as result of an opened bookmark (i.e. is not a immediately preceeded by browsing) an unspecified ifindex/proto value should be passed to the resolver. (i.e. AVAHI_IF_UNSPEC, AVAHI_PROTO_UNSPEC). The DNS-SD spec even suggests bookmarking this service instance triple for future usage: http://files.dns-sd.org/draft-cheshire-dnsext-dns-sd.txt (part 4.2 "User Interface Presentation") uPNP is very different from mDNS/DNS-SD. Requests to abstract them in some API layer are often heard but make not much sense. The only layer where it makes sense to abstract them is the VFS -- but that only for a subset of services, i.e. ftp+dnssd, webdav+dnssd on one side and the media server stuff for upnp on the other. So, in summary, here's what I suggest: When browsing, show a list of all services found and store the service triple/ifindex/proto for each entry in memory. When the user selects an entry, pass the service triple and ifindex/proto to the resolver and connect to it. If a bookmark is created, only store the service triple, ignore the ifindex/proto. If a bookmark is opened, resolve the service triple, set ifindex/proto to AVAHI_IF_UNSPEC/AVAHI_PROTO_UNSPEC and then connect to it. In all cases the name shown to the user in the UI should stay the service name, and only the service name. The service type and domain should not be showed at all, except when it is needed to distuingish equally named services of different types or from different domains. I'd suggest introducing a simple dns-sd uri scheme for this that simply reflects the dns-formatted service triple as returned by avahi_service_name_join(). i.e. something like this: dns-sd://Lennart's Files._ftp._tcp.local/ If you want to include the service type in the uri scheme it might make sense to use something like this instead: dns-sd+ftp://Lennart's Files._ftp._tcp.local/ (But that would of course be redundant, since we include the service type twice in the URI.) If we want to bookmark a subdir of an ftp/dav share we should simply append it to the URI. i.e. if "dns-sd://Lennart's Files._ftp._tcp.local/" refers to some service on foobar.local:21 with the meta data already including a reference to /home/lennart, then using "dns-sd://Lennart's Files._ftp._tcp.local/another/sub/dir" would refer to an ftp service foobar.local:21 with the dir /home/lennart/another/sub/dir. Apple's version of CUPS uses a similar syntax btw, they name it "mdns://" however. Not sure of the official version does that now too. It might make sense to use the same prefix although i'd argue that dnssd:// is technically more correct than mdns://. KDE's dnssd supports dnssd:// style URL, but I have never played around with that, no idea how complete the support for that is. They seem to follow a different URI-building scheme however: i.e. they use dnssd://local./_http._tcp/Name for the triple Name, _http._tcp and local. I think this is crazy however. (i.e. it suggests listing all services with dnssd://local./ which is however misleading because not really supported by the proto except for debug purposes) In fact Apple's Marc Krochmal (one of the two guys responsible for mDNS/DNS-SD) at one point even talked about submitting an RFC about dns-sd style URLs: http://lists.apple.com/archives/Rendezvous-dev/2004/Dec/msg00001.html Apple/Marc seems to favour the URI scheme I propose, i.e. with dots in proper DNS-order.
Created attachment 121086 [details] [review] better patch OK, here's a better patch for doing this. Notes: - The existing dns-sd:// backend now exports the key information obtained from Avahi under the dns-sd:: name space (see [1]). This is probably useful for other purposes. With this the dns-sd:// gvfs backend sorta-kinda truly wraps Avahi using the GIO API. Personally I think that's useful but I can understand if that's not desirable. Anyway, the main reason for this addition is that we need this information, broken-down, when constructing the GMountSpec daemon side. The patch in comment 7 generated the URI in the dns-sd:// backend but that seems a bit wrong (in general we want to avoid exposing URIs). - We now use URIs of the form proposed by Lennart in comment 11, for example "dav-sd://davidz's public files on quad\.fubar\.dk._webdav._tcp.local" - There's a new gvfs utility function g_mount_spec_resolve_for_dns_sd() that does all the magic (there's some TODO's in it, feedback welcome) - A bug fix in daemon/gvfsbackenddav.c:path_equal() that avoids a crashed - you probably want that hunk as soon as possible and also for stable GVfs - Also show _ssh._tcp services. Without this my sftp mounts won't show up in network:///. You probably also want this for stable GVfs. Discussion - I assume all strings handed out by Avahi are valid UTF-8. I don't know if this assumption is valid.. Lennart? If not, what subset are valid UTF-8 strings? - Dunno if URI's like "dav-sd://davidz's public files on quad\.fubar\.dk._webdav._tcp.local" are valid in gio/gvfs. Seems to work fine for Nautilus and Eye of Gnome; however gedit doesn't like them. We may need to escape this stuff. I don't know. Anyway, discuss. Thanks. [1] : $ gvfs-info "dns-sd://local/davidz's public files on quad\.fubar\.dk._webdav._tcp" display name: davidz's public files on quad.fubar.dk name: davidz's public files on quad\x0c.fubar\x0c.dk._webdav._tcp type: shortcut size: 0 attributes: standard::name: davidz's public files on quad\x5c.fubar\x5c.dk._webdav._tcp standard::display-name: davidz's public files on quad.fubar.dk standard::icon: (fallbacks: FALSE) 1. name:folder-remote standard::type: 5 standard::size: 0 standard::is-virtual: TRUE standard::target-uri: dav-sd://davidz's public files on quad\.fubar\.dk._webdav._tcp.local access::can-write: FALSE access::can-delete: FALSE access::can-trash: FALSE dns-sd::resolved-service_name: davidz's public files on quad.fubar.dk dns-sd::resolved-type: _webdav._tcp dns-sd::resolved-host-name: quad.local dns-sd::resolved-address: 192.168.1.42 dns-sd::resolved-port: 37039 dns-sd::resolved-protocol: IPv4 dns-sd::resolver-supports-mdns: TRUE dns-sd::resolved-txt-u: guest dns-sd::resolved-txt-org.freedesktop.od.session: 2c778f3704fd02abae3d983948e5390f id::filesystem: host=local,type=dns-sd,__mount_prefix=%2F
The service name is guaranteed to be UTF-8, and any UTF-8 string may be used as service name. The service type is guaranteed to be 7bit ASCII. The domain is UTF-8, however Avahi will hand it to the client code in DNS-style escaped syntax, i.e. 7bit, with \ as escape character. With avahi_escape_label() and avahi_unescape_label() you may convert between the escaped and the UTF8 form. If you concatenate the three parts via avahi_service_name_join() then it will take the name as UTF-8, type as 7Bit, and domain as a 7bit-escaped-UTF-8 and return you the resulting string as 7bit-escaped-UTF-8 again. (Please note that mDNS/DNS-SD explcitily uses UTF-8 on the wire (and not punycode!) to encode the full unicode range). The TXT data for services can be anything. It's basically 8bit raw data. However, the spec recommends to use 7bit ASCII for the field names and UTF8 for the field data. Also TXT data fields don't need to be key/value pairs, i.e. it is not required that they include a seperating "=" character. I think it would make sense to only add the TXT fields to the properties if they match these recommendations, i.e. include a "=", with the key being 7bit ascii and the data being valid utf-8. Hmm, you now used "dav-sd://" as URI scheme? So you are suggesting adding "ftp-sd://", "sftp-sd://" as well? Wouldn't it be more appropriate to call that "dav+sd" or similar?
Hmm, those gfile properties don't have an order, do they? An AvahiStringList generally stores the TXT data in reverse order (since it is just a linked list and we wan't to make appending cheap which due to the reverse order becomes prepending which is O(1) for a singly linked list). So, if the gfile props had an order it would make sense to call avahi_string_list_reverse() before we iterate through the list to follow the same order as in the TXT data itself. + avahi_string_list_get_pair (l, &key, &value, NULL); + + s = g_strdup_printf ("dns-sd::resolved-txt-%s", key); + g_file_info_set_attribute_string (info, s, value); + g_free (s); This is problematic. "value" might be set to NULL in case the TXT blob contains a field without '='. I'd suggest adding a line like this after the get_pair() call. if (!g_utf8_validate(key) || !value || !g_utf8_validate(value)) continue;
Created attachment 121165 [details] [review] even better patch OK, here's an updated patch that - Is careful about getting the UTF-8 stuff right. - Encodes the DNS-SD triple into a format suitable for gvfs, e.g. using g_uri_escape_string(). Ideally we'd just use avahi_service_name_split() and avahi_service_name_join() but this is problematic because - I think gvfs URIs needs to be properly escaped. E.g. dav-sd://David's files on foo\.bar._webdav._tcp.local is not permitted. It needs to be dav-sd://David%27s%20files%20on%20foo\.bar._webdav._tcp.local I think. But I must admit I'm not an expert on what URIs are valid in the gvfs world. Alex? - We want to avoid linking anything but the dns-sd:// gvfs backend with Avahi. But we can always copy-paste, the licenses for Avahi and gvfs seems pretty compatible (LGPLv2.1+ resp LGPLv2+), in the worst case we can always ask Lennart for permission to relicense. - Provides some polished utility API for backends to consume. See common/gvfsdnssdutils.[ch] and how it's used in the DAV backend. Notes - We only export key/value pair TXT records with the key being valid UTF-8. Also, we pretend these are strings, e.g. we ignore NUL bytes inbetween. This is probably OK for now, if we want to fix this we need some gio changes to introduce new attribute types. See comments in the part of the patch that touches gvfsbackenddnssd.c. Future development (but am holding off until this patch or similar is in) - port ftp:// and sftp:// backends to understand ftp-sd:// and sftp-sd:// type URIs. Should be trivial. - We want to avoid hard coding what service types the dns-sd:// backend shows. For example, in the future it's not unlikely that the afpfs-ng project [1] might want to supply a gvfs backend for accessing files share from OS X boxen. One way to do this is to make the GVfs dns-sd backend read files with a .dns-sd extension from, say, /usr/share/gvfs/dns-sd/. The content of these files, combined, would replace the dns_sd_types[] array in gvfsbackenddnssd.c today. [1] : http://alexthepuffin.googlepages.com/home
(In reply to comment #13) > Hmm, you now used "dav-sd://" as URI scheme? So you are suggesting adding > "ftp-sd://", "sftp-sd://" as well? Wouldn't it be more appropriate to call that > "dav+sd" or similar? Maybe I misunderstand but why is dav+sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local better than dav-sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local FWIW, I'm not opposed to changing the minus to the plus.. but I'm not sure what difference it would make? Btw, see [1] for what the patch currently exports. [1] : $ gvfs-info "dns-sd://local/davidz's public files on x61\.fubar\.dk._webdav._tcp" display name: davidz's public files on x61.fubar.dk name: davidz's public files on x61\x0c.fubar\x0c.dk._webdav._tcp type: shortcut size: 0 attributes: standard::name: davidz's public files on x61\x5c.fubar\x5c.dk._webdav._tcp standard::display-name: davidz's public files on x61.fubar.dk standard::icon: (fallbacks: FALSE) 1. name:folder-remote standard::type: 5 standard::size: 0 standard::is-virtual: TRUE standard::target-uri: dav-sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local access::can-write: FALSE access::can-delete: FALSE access::can-trash: FALSE dns-sd::name: davidz's public files on x61.fubar.dk dns-sd::type: _webdav._tcp dns-sd::domain: local dns-sd::resolved-host-name: x61.local dns-sd::resolved-address: 192.168.122.1 dns-sd::resolved-port: 60961 dns-sd::resolved-protocol: IPv4 dns-sd::resolver-supports-mdns: TRUE dns-sd::resolved-txt::u: guest dns-sd::resolved-txt::org.freedesktop.od.session: d9022464a004e1ecbd8c3ba648fd47d9 id::filesystem: host=local,type=dns-sd,__mount_prefix=%2F
(In reply to comment #15) > - I think gvfs URIs needs to be properly escaped. E.g. > > dav-sd://David's files on foo\.bar._webdav._tcp.local Actually that's not what avahi_service_name_join() spits out. Instead it will spit a fully escaped string out, following DNS style escaping. It looks like this: David\039s\032files\032on\032foo\.bar._webdav._tcp.local DNS style escaping (i.e. \0...) is different from HTTP/URI escaping (i.e. %...). I guess HTTP/URI style escaping is more appropriate for gvfs URIs, hence it's probably a good idea not to use avahi_service_name_join() in this case. However, if you do this and want to do it completely correctly you'd also have to convert the domain into this format because AVahi will hand it to you in the DNS-escaped way. You can use avahi_unescape_label() to convert back to the unescaped form and then reencode that for your own form -- a_u_l() however will unescape only a single label, not an entire domain. It's probably pointless to spend much time on this however because virtually nobody uses mDNS outside of the .local domain, and "local" certainly needs no escaping. In short: your new patch seems to do it correctly enough. > - We want to avoid hard coding what service types the dns-sd:// backend > shows. For example, in the future it's not unlikely that the afpfs-ng > project [1] might want to supply a gvfs backend for accessing files share > from OS X boxen. One way to do this is to make the GVfs dns-sd backend > read files with a .dns-sd extension from, say, /usr/share/gvfs/dns-sd/. > > The content of these files, combined, would replace the dns_sd_types[] > array in gvfsbackenddnssd.c today. > > [1] : http://alexthepuffin.googlepages.com/home Hmm, eventually I'd also like to see a gvfs backend for daap/dpap/dmap that allows direct copying from music/picture/video files to a local share. That would be another file service using DNS-SD. Also, Apple makes SMB services discoverable with mDNS/DNS-SD, too. Might be yet another protocol.
> Maybe I misunderstand but why is > > dav+sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local > > better than > > dav-sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local > > FWIW, I'm not opposed to changing the minus to the plus.. but I'm not sure what > difference it would make? Btw, see [1] for what the patch currently exports. As I understood it the dash in the URI scheme is often used inside of single protocol names. However when the + is used it describes a combination of protocols as in "this on top of that" or "this with that". Examples for the first one are "view-source://" and "dns-sd://" (yes, both are pretty bad examples, I admit, too lazy to look for better ones). Example for the latter one is "svn+ssh://". Since we combine FTP and DNS-SD here (or WebDAV and DNS-SD) the "+" thus appears more applicable to me. But of course, I don't think this has ever been formalized. And it doesn't really matter either. > > [1] : > > $ gvfs-info "dns-sd://local/davidz's public files on > x61\.fubar\.dk._webdav._tcp" > display name: davidz's public files on x61.fubar.dk > name: davidz's public files on x61\x0c.fubar\x0c.dk._webdav._tcp > type: shortcut > size: 0 > attributes: > standard::name: davidz's public files on x61\x5c.fubar\x5c.dk._webdav._tcp > standard::display-name: davidz's public files on x61.fubar.dk > standard::icon: (fallbacks: FALSE) > 1. name:folder-remote > standard::type: 5 > standard::size: 0 > standard::is-virtual: TRUE > standard::target-uri: > dav-sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local > access::can-write: FALSE > access::can-delete: FALSE > access::can-trash: FALSE > dns-sd::name: davidz's public files on x61.fubar.dk > dns-sd::type: _webdav._tcp > dns-sd::domain: local > dns-sd::resolved-host-name: x61.local > dns-sd::resolved-address: 192.168.122.1 > dns-sd::resolved-port: 60961 > dns-sd::resolved-protocol: IPv4 > dns-sd::resolver-supports-mdns: TRUE > dns-sd::resolved-txt::u: guest > dns-sd::resolved-txt::org.freedesktop.od.session: > d9022464a004e1ecbd8c3ba648fd47d9 > id::filesystem: host=local,type=dns-sd,__mount_prefix=%2F >
(In reply to comment #17) > Hmm, eventually I'd also like to see a gvfs backend for daap/dpap/dmap that > allows direct copying from music/picture/video files to a local share. That > would be another file service using DNS-SD. Also, Apple makes SMB services > discoverable with mDNS/DNS-SD, too. Might be yet another protocol. I'd like to see that too. But it seems there's a lot of people that think it's wrong to do have mounts for things like daap... they want such objects to only show up in e.g. Totem and Rhythmbox.. but I digress. Anyway, my view is that if we make gvfs extensible then we need to make what shows up in dns-sd:// extensible too. (In reply to comment #18) > As I understood it the dash in the URI scheme is often used inside of single > protocol names. However when the + is used it describes a combination of > protocols as in "this on top of that" or "this with that". > > Examples for the first one are "view-source://" and "dns-sd://" (yes, both are > pretty bad examples, I admit, too lazy to look for better ones). Example for > the latter one is "svn+ssh://". > > Since we combine FTP and DNS-SD here (or WebDAV and DNS-SD) the "+" thus > appears more applicable to me. Yeah, that makes sense. I'll change it.
Created attachment 121171 [details] [review] bestest patch ever (ok, bad joke) Here's a small update to the previous patch. Compared to the previous one - change method names from dav-sd,davs-sd to dav+sd,davs+sd - tested with non-local domains, worked fine without changes [1] - sort out some icon and display name bugs until I was satisfied with how browsing worked in nautilus - should use the work-group icon for dns-sd domains in network:/// otherwise domains looks just like servers - use themed_icon_new_with_fallbacks so things like folder-remote-ftp and folder-remote-ssh works - use domain name instead of "dns-sd" for display name when browsing dns-sd://somedomain.net/ $ diffstat gvfs-dnssd-r2073-20081022.patch common/Makefile.am | 1 common/gvfsdnssdutils.c | 510 ++++++++++++++++++++++++++++++++++++++++++ common/gvfsdnssdutils.h | 55 ++++ daemon/Makefile.am | 3 daemon/dav.mount.in | 2 daemon/gvfsbackenddav.c | 87 +++++++ daemon/gvfsbackenddnssd.c | 320 +++++++++++++++++++++------ daemon/gvfsbackendnetwork.c | 2 8 files changed, 895 insertions(+), 85 deletions(-) [1] : Got Lennart to set up something on 0pointer.org, thanks Lennart! $ gvfs-info dns-sd://0pointer.de/Lennart\'s%20Stuff._webdavs._tcp display name: Lennart's Stuff name: Lennart's Stuff._webdavs._tcp type: shortcut size: 0 attributes: standard::name: Lennart's Stuff._webdavs._tcp standard::type: 5 standard::display-name: Lennart's Stuff standard::icon: (fallbacks: FALSE) 1. name:folder-remote 2. name:folder standard::size: 0 standard::is-virtual: TRUE standard::target-uri: davs+sd://Lennart%27s%20Stuff._webdavs._tcp.0pointer.de access::can-write: FALSE access::can-delete: FALSE access::can-trash: FALSE id::filesystem: host=0pointer.de,type=dns-sd,__mount_prefix=%2F dns-sd::name: Lennart's Stuff dns-sd::type: _webdavs._tcp dns-sd::domain: 0pointer.de dns-sd::resolved-host-name: tango.0pointer.de dns-sd::resolved-address: 85.214.72.216 dns-sd::resolved-port: 443 dns-sd::resolved-protocol: UNSPEC dns-sd::resolver-supports-mdns: TRUE dns-sd::resolved-txt::path: /lennart/webdav-lennart
A URI like: dav+sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local seems to contain some duplicated information to me. I.E. the dav part of the scheme implies that the service is of type _webdav._tcp. And since every backend that wants to support dns-sd uris need to get some special code to handle this that could would know what service type to use. Also, the slash-escaped dots are bad, we shouldn't allow dots like that in the non-domain part of the hostname. They should be uri-endcoded. So, I'd prefer a URI like: dav+sd://davidz%27s%20public%20files%20on%20x61%2efubar%2edk.local Also, I don't see why the daemon can't just use avahi to do the resolve. All this multi-process ping-ponging with the dns-sd backend just seems to cause complexity and slowness. dns-sd:// is for graphically browsing the dns-sd network for filesystem shares, not for doing service resolves. In fact, there is a race when using it for resolving like that if dns-sd: isn't mounted yet, as it will return as newly mounted, but all the files not ready yet (they are added async as they are discovered). Also, g_mount_spec_new_for_encoded_dns_sd_triple() looks quite weird. Mount specs are tricky things, and must be handled with care so the system works. A mount spec is a key-value based description of the mount. It must be constructable from the uri in a unique way, and doing so must not do any I/O (or you get problematic i/o on client side when creating GFiles). dav+sd uris look sufficiently "normal" that there is no need for a custom GVfsUriMapper for them, which would mean a URI like the above will get a mount spec like: type="dav+sd" host="davidz%27s%20public%20files%20on%20x61%2efubar%2edk.local" Anything else will just break shit. There is also a problem with services that specify HostnameIsInetAddress=true in the .mount file (which arguably http/dav should do but don't atm), in that this causes a the hostname to be lowercased so uris with different cases of the hostname doesn't point to different mounts. Maybe you'd need a separate .mounts file to handle that. We need to put the backend side dns-sd helpers needed in its own helper lib that links to avahi, so that only avahi using backends pull that in. And use of it should probably be ifdef:ed on HAVE_AVAHI.
(In reply to comment #21) > Also, I don't see why the daemon can't just use avahi to do the resolve. All > this multi-process ping-ponging with the dns-sd backend just seems to cause > complexity and slowness. dns-sd:// is for graphically browsing the dns-sd > network for filesystem shares, not for doing service resolves. I did it this way for very specific reasons. The avahi daemon doesn't cache stuff so using it to map a triple to host/port/txt records would actually be slow. Also, I'm not sure there's a synchronous interface for this cf. http://lists.freedesktop.org/archives/avahi/2006-May/000691.html though I may be misunderstanding how Avahi works. So you'd need to do things asynchronous, just like in the dns-sd:// daemon. I don't think that helps with removing complexity and slowness. On the other hand, the dns-sd:// mount daemon caches (a small subset) of information and, actually, makes the implementation quite a bit simpler than if you had to use Avahi proper (you also need to consider monitoring etc.). > In fact, there > is a race when using it for resolving like that if dns-sd: isn't mounted yet, > as it will return as newly mounted, but all the files not ready yet (they are > added async as they are discovered). I'm aware of that - it comes up a lot during development when you kill the horde of gvfs daemons. The dns-sd:// mount daemon can be fixed to handle this by using a timeout of some sort (results should trickle in seconds just after it's mounted). You'd have the same problem if the backend was using Avahi itself, only it wouldn't be a race; you would have this problem all the time. I'm not sure that's wanted. > Also, g_mount_spec_new_for_encoded_dns_sd_triple() looks quite weird. Mount > specs are tricky things, and must be handled with care so the system works. A > mount spec is a key-value based description of the mount. It must be > constructable from the uri in a unique way, and doing so must not do any I/O > (or you get problematic i/o on client side when creating GFiles). dav+sd uris > look sufficiently "normal" that there is no need for a custom GVfsUriMapper for > them, which would mean a URI like the above will get a mount spec like: > > type="dav+sd" > host="davidz%27s%20public%20files%20on%20x61%2efubar%2edk.local" > > Anything else will just break shit. I you think about it, no mapper will ever needed for DNS-SD triples. The way that DNS-SD works is that the triple _uniquely_ identifies the service - e.g. you won't need to map e.g. http://foo:80 to http://foo etc. etc. (Or maybe my understanding of GVfsUriMapper is weak, there's not a lot of docs and this is tricky stuff.) > There is also a problem with services that specify HostnameIsInetAddress=true > in the .mount file (which arguably http/dav should do but don't atm), in that > this causes a the hostname to be lowercased so uris with different cases of the > hostname doesn't point to different mounts. Maybe you'd need a separate .mounts > file to handle that. Honestly I didn't know about HostnameIsInetAddress but, sure, different .mount files will do I suppose. > We need to put the backend side dns-sd helpers needed in its own helper lib > that links to avahi, so that only avahi using backends pull that in. And use of > it should probably be ifdef:ed on HAVE_AVAHI. For the reasons stated above I don't think we want that.... David
(In reply to comment #21) > A URI like: > > dav+sd://davidz%27s%20public%20files%20on%20x61\.fubar\.dk._webdav._tcp.local > > seems to contain some duplicated information to me. I.E. the dav part of the > scheme implies that the service is of type _webdav._tcp. > > And since every backend that wants to support dns-sd uris need to get some > special code to handle this that could would know what service type to use. > > Also, the slash-escaped dots are bad, we shouldn't allow dots like that in the > non-domain part of the hostname. They should be uri-endcoded. > > So, I'd prefer a URI like: > > dav+sd://davidz%27s%20public%20files%20on%20x61%2efubar%2edk.local I think it wuld be a good thing to use the proper DNS-SD triple here, not leave the service type out. In contrast to what I said earlier the service type wouldn't even be redundant. Why? Because dns-sd service types and gvfs URI schemes don't match 1:1. i.e. Both of the following URI do make sense: sftp+sd://davidz%27s%20public%20files%20on%20x61%2efubar%2edk._sftp-ssh._tcp.local sftp+sd://davidz%27s%20public%20files%20on%20x61%2efubar%2edk._ssh._tcp.local i.e. it makes sense to open an SSH service via sftp+sd, not just sftp services. And there are probably more examples for this, i.e. WebDAV is also HTTP. And so on. So please, keep the full DNS-SD triple in the URIs!
(In reply to comment #22) > (In reply to comment #21) > > Also, I don't see why the daemon can't just use avahi to do the resolve. All > > this multi-process ping-ponging with the dns-sd backend just seems to cause > > complexity and slowness. dns-sd:// is for graphically browsing the dns-sd > > network for filesystem shares, not for doing service resolves. > > I did it this way for very specific reasons. The avahi daemon doesn't cache > stuff so using it to map a triple to host/port/txt records would actually be > slow. Also, I'm not sure there's a synchronous interface for this cf. Actually Avahi's primary job is to do cacheing. 90% of the mDNS spec describes somewhat elaborate caching mechanisms. Avahi is actually capable of merging multiple live queries issued from multiple different local clients into one, even if they are not issued simultaneously but one after the other. That said it is still better to have a single very-long-running query then two long-runnning queries one after the other. > http://lists.freedesktop.org/archives/avahi/2006-May/000691.html > > though I may be misunderstanding how Avahi works. So you'd need to do things > asynchronous, just like in the dns-sd:// daemon. I don't think that helps with > removing complexity and slowness. Yes, network access should always happen asynchronously. And the Avahi API only exposes asynchronous interfaces.
Created attachment 121411 [details] [review] updated patch Here's a patch that uses a new separate libgvfscommon-dnssd library and uses URL style encoding. (I initially wanted to use avahi-gobject but a) didn't want to add a new dependency / .so file; b) the dns-sd backend doesn't use it; c) gvfs on OS X / Windows might want to use the native DNS-SD library; and d) I wanted an even simpler interface optimized for gvfs uses only; e) I wanted a sync interface in addition to the async one)
$ diffstat gvfs-dnssd-r2077-20081026.patch common/Makefile.am | 17 common/gvfsdnssdresolver.c | 1253 ++++++++++++++++++++++++++++++++++++++++++ common/gvfsdnssdresolver.h | 80 ++ common/gvfsdnssdutils.c | 320 +++++++++++ common/gvfsdnssdutils.h | 50 + daemon/Makefile.am | 18 daemon/dav+sd.mount.in | 4 daemon/dav.mount.in | 2 daemon/gvfsbackenddav.c | 143 ++++- daemon/gvfsbackenddnssd.c | 218 ++++--- daemon/gvfsbackendnetwork.c | 2 11 files changed, 2018 insertions(+), 89 deletions(-)
Created attachment 121412 [details] [review] updated patch And here's another updated patch that returns the right mount spec for davs+sd:// URIs (I need to remember to grep my patches for TODO items).
Fixing up the summary to reflect what we really want.
*** Bug 326705 has been marked as a duplicate of this bug. ***
I've commited this with the following fixes: g_vfs_get_dns_sd_uri_for_triple() must uri-encode the service name as its part of the pathname in a uri. g_vfs_encode_dns_sd_triple() -- don't use TRUE for allow_utf8, that results in non-URLs (i.e. they are IRIs) However, there are some outstanding issues: The sync resolver recurses the a mainloop for the default main context. This is really bad, since it means two threads will handle events for the main context. The target links for files listed in the dns-sd backend which use the new triplet dav+sd for don't add the "path" info from the resolve to the path of the uri. I can sort of understand why, as the backend resolves the servicename again, but its still kind of weird to leave out part of the path like that. What do you think about this?
(In reply to comment #30) > I've commited this with the following fixes: > > g_vfs_get_dns_sd_uri_for_triple() must uri-encode the service name as its part > of the pathname in a uri. > > g_vfs_encode_dns_sd_triple() -- don't use TRUE for allow_utf8, that results in > non-URLs (i.e. they are IRIs) Great, thanks for committing this. > However, there are some outstanding issues: > > The sync resolver recurses the a mainloop for the default main context. This is > really bad, since it means two threads will handle events for the main context. Right now this is not a problem as the DAV backend is not threaded. But I hear your concern; it's something we should fix. On top of my head, if we could find the default GMainContext (if any) for the current thread, wouldn't it make the problems go away? Then we'd just use that context for a) timeouts (in the async case) [1]; and b) waiting in the mainloop (for the sync case). We'd also need to be careful to have an AvahiClient per main context (and use the right one) but that's doable too. [1] : g_timeout_add() isn't enough... need to use lower-level API but that's fine. We'd also need test cases for this but I can do that too. Unfortunately it seems there's no way to get the default GMainContext for the current thread (g_main_context_default() uses a global variable) but it *seems* like a patch to GLib could give us this (GLib would store this in TSS); it's IMO not an unreasonable request to ask for some like g_main_context_get_default_for_thread(). Thoughts? > The target links for files listed in the dns-sd backend which use the new > triplet dav+sd for don't add the "path" info from the resolve to the path of > the uri. I can sort of understand why, as the backend resolves the servicename > again, but its still kind of weird to leave out part of the path like that. > What do you think about this? I would have to actually try this out to get an idea about it; I'll follow up.
(In reply to comment #31) > Right now this is not a problem as the DAV backend is not threaded. But I hear > your concern; it's something we should fix. Turns out the DAV backend is actually using threads as alexl pointed out on IRC. Also, didn't get around to fixing this tonight, but a couple of points from a discussion on IRC - For the sync case, don't use a mainloop. Will have to create a separate AvahiClient object with a simple poll function. - For the async case, it's fine to use the default maincontext and deliver the callback on the main thread (it's how GIO works anyway)
(In reply to comment #15) > - Encodes the DNS-SD triple into a format suitable for gvfs, e.g. using > g_uri_escape_string(). Ideally we'd just use avahi_service_name_split() > and avahi_service_name_join() but this is problematic because > > - I think gvfs URIs needs to be properly escaped. E.g. > > dav-sd://David's files on foo\.bar._webdav._tcp.local > > is not permitted. It needs to be > > dav-sd://David%27s%20files%20on%20foo\.bar._webdav._tcp.local > > I think. But I must admit I'm not an expert on what URIs are valid > in the gvfs world. Alex? The escaping sounds reasonable but there's a downside of this. Right now the encoded triple looks like dav+sd://tbzatek%2527s%2520public%2520files%2520on%2520rakosnicek%252eenglab%252ebrq%252eredhat%252ecom._webdav._tcp.local/ which causes matching issues leading to "Mountpoint Already registered" errors. Notice the double escaping there; though we set single-escaped "tbzatek%27s%20public%20files%20on%20rakosnicek%2eenglab%2ebrq%2eredhat%2ecom._webdav._tcp.local" as a hostname to the mountspec. GIO/GVFS does URI escaping itself in some way, escaping all the % characters with %25 once again. The original hostname passed to dav's do_mount() looks like "tbzatek's public files on rakosnicek.englab.brq.redhat.com._webdav._tcp.local", i.e. completely unescaped. This leads to dav+sd://tbzatek's%20public%20files%20on%20rakosnicek.englab.brq.redhat.com._webdav._tcp.local/ URI. We should have this set on the output mount_spec to prevent URI matching issues (confirmed as working).
-- GitLab Migration Automatic Message -- This bug has been migrated to GNOME's GitLab instance and has been closed from further activity. You can subscribe and participate further through the new bug through this link to our GitLab instance: https://gitlab.gnome.org/GNOME/gvfs/issues/65.