After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 783569 - Fix for DNS leaks over NM-VPN on Ubuntu using default dns manager systemd-resolved
Fix for DNS leaks over NM-VPN on Ubuntu using default dns manager systemd-res...
Status: RESOLVED FIXED
Product: NetworkManager
Classification: Platform
Component: IP and DNS config
git master
Other Linux
: Normal critical
: ---
Assigned To: NetworkManager maintainer(s)
NetworkManager maintainer(s)
Depends on:
Blocks:
 
 
Reported: 2017-06-08 19:17 UTC by Nick Stommel
Modified: 2017-06-22 01:43 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
fixes DNS leaks over some NM-VPN connections using systemd-resolved (1.94 KB, patch)
2017-06-08 19:17 UTC, Nick Stommel
none Details | Review
current version of nm-systemd-resolved.c in Ubuntu network manager (12.00 KB, text/x-csrc)
2017-06-09 21:07 UTC, Nick Stommel
  Details

Description Nick Stommel 2017-06-08 19:17:14 UTC
Created attachment 353426 [details] [review]
fixes DNS leaks over some NM-VPN connections using systemd-resolved

I have patched the Network Manager to fix DNS leaks over network-manger VPN links (like those created with network-manager-openvpn) when using systemd-resolved as the default dns-manger/resolver on Ubuntu. This addresses some critical security concerns with DNS leaks over NM-VPN links. 
Please see the following high-priority bug at launchpad: https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1624317
There, I attached a patch for the current version of the network manager on Ubuntu 17.04 (1.4.4-1ubuntu3 zesty). Per request of kai.heng.feng@canonical.com on launchpad, I have patched the latest upstream source, made sure that it compiles correctly without warnings, and attached it here. So far, this is known to solve DNS leaks with network-manager-openvpn but could also solve DNS leaks for other VPNs that use TUN, TAP, or Cisco GRE network interfaces through the network-manager. It would be great to backport this fix to the current Ubuntu distribution!
Comment 1 Thomas Haller 2017-06-09 07:51:45 UTC
there is ipv4.dns-priority setting

Quoting `man nm-settings`:

"Intra-connection DNS priority. The relative priority to be used when determining the order of DNS servers in resolv.conf.  A lower value means that servers will be on top of the file.  Zero selects the default value, which is 50 for VPNs and 100 for other connections.  Note that the priority is to order DNS settings for multiple active connections. It does not disambiguate multiple DNS servers within the same connection profile. For that, just specify the DNS servers in the desired order. When multiple devices have configurations with the same priority, the one with an active default route will be preferred. Note that when using dns=dnsmasq the order is meaningless since dnsmasq forwards queries to all known servers at the same time. Negative values have the special effect of excluding other configurations with a greater priority value; so in presence of at least a negative priority, only DNS servers from connections with the lowest priority value will be used."


Why is that not the right solution?
Comment 2 Beniamino Galvani 2017-06-09 08:07:38 UTC
Hi,

in addition to what Thomas said, I think it is wrong to assume the
connection is a VPN based on the link type, since you can have non-VPN
tun/tap/gre/gretap connections as well, and they are affected by this
patch.

Also, it seems to me that commit [1] should already fix the leak for
VPNs that don't get the default route... did you try if git master of
NM is still affected by this bug?

If it is, can you please attach the output of 'nmcli connection show
<con-id>' for the VPN, and the output of 'systemd-resolve --status'? I
mean, without your patch, to verify why the existing commit is not
working. Thanks!

[1] https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=c4864ba63f4a13e9938a978787490005f5ba48fb
Comment 3 Nick Stommel 2017-06-09 16:30:24 UTC
Oh, sorry I didn't think of that :/
I suppose that commit is a better fix. I'll see if that works instead.
Comment 4 Nick Stommel 2017-06-09 21:07:27 UTC
Created attachment 353487 [details]
current version of nm-systemd-resolved.c in Ubuntu network manager

Wait...that commit from [1] is already found in the zesty version of network-manager 1.4.4-1ubuntu3. See attachment. This suggests that if this is still an issue on Ubuntu 17.04, the problem is still present and unsolved. 

[1] https://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=c4864ba63f4a13e9938a978787490005f5ba48fb
Comment 5 Nick Stommel 2017-06-09 21:58:24 UTC
So the issue persists I believe. Unfortunately I just broke totally broke my installation of Ubuntu 17.04 using 'sudo make install' on the git master of NM and was unable to connect to the internet at all, so someone else is going to have to test or figure that one out. :( 
I'm almost positive, however that the issue would remain. Using openvpn through network-manager-openvpn, the line "DNS Domain: ~." under the VPN link number and info with "systemd-resolved --status" is missing or has nothing after the colon. So the routing-only domain is not used for the openvpn link like tun0, and we have dns leaks. Please contact the devs at canonical assigned to https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1624317
This is a fairly critical bug that I think needs to be worked out with other people more knowledgeable there. I'm afraid I can't help much more, sorry.
Comment 6 Beniamino Galvani 2017-06-12 08:03:43 UTC
(In reply to Nick Stommel from comment #5)

Hi Nick,

thanks for submitting the patch and investigating this issue.

> So the issue persists I believe. Unfortunately I just broke totally broke my
> installation of Ubuntu 17.04 using 'sudo make install' on the git master of
> NM and was unable to connect to the internet at all, so someone else is
> going to have to test or figure that one out. :( 

I guess you can easily undo the changes with 'make uninstall' and
reinstalling the deb package.

> I'm almost positive, however that the issue would remain. Using openvpn
> through network-manager-openvpn, the line "DNS Domain: ~." under the VPN
> link number and info with "systemd-resolved --status" is missing or has
> nothing after the colon. So the routing-only domain is not used for the
> openvpn link like tun0, and we have dns leaks. Please contact the devs at
> canonical assigned to
> https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1624317
> This is a fairly critical bug that I think needs to be worked out with other
> people more knowledgeable there. I'm afraid I can't help much more, sorry.

It's not easy to understand which the actual problem is from the
launchpad bug because of some contrasting comments.

Can you please show the connection configuration and the status of
systemd-resolve when using the original NM package?

 nmcli connection show <con-id>
 systemd-resolve --status
Comment 7 Nick Stommel 2017-06-13 00:21:41 UTC
**********
noctua@corinth:~$ systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 5 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218

Link 2 (wlo1)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 192.168.1.1
          DNS Domain: home

**********
noctua@corinth:~$ nmcli connection show tun0
connection.id:                          tun0
connection.uuid:                        a61ca484-3ca9-4e88-b6e1-574b4e17ca54
connection.stable-id:                   --
connection.interface-name:              tun0
connection.type:                        tun
connection.autoconnect:                 no
connection.autoconnect-priority:        0
connection.timestamp:                   1497284475
connection.read-only:                   no
connection.permissions:                 
connection.zone:                        --
connection.master:                      --
connection.slave-type:                  --
connection.autoconnect-slaves:          -1 (default)
connection.secondaries:                 
connection.gateway-ping-timeout:        0
connection.metered:                     unknown
connection.lldp:                        -1 (default)
ipv4.method:                            manual
ipv4.dns:                               
ipv4.dns-search:                        
ipv4.dns-options:                       (default)
ipv4.dns-priority:                      100
ipv4.addresses:                         10.38.1.6/32
ipv4.gateway:                           10.38.1.5
ipv4.routes:                            { ip = 10.38.1.1/32, nh = 10.38.1.5, mt = 50 }
ipv4.route-metric:                      50
ipv4.ignore-auto-routes:                no
ipv4.ignore-auto-dns:                   no
ipv4.dhcp-client-id:                    --
ipv4.dhcp-timeout:                      0
ipv4.dhcp-send-hostname:                yes
ipv4.dhcp-hostname:                     --
ipv4.dhcp-fqdn:                         --
ipv4.never-default:                     no
ipv4.may-fail:                          yes
ipv4.dad-timeout:                       -1 (default)
ipv6.method:                            link-local
ipv6.dns:                               
ipv6.dns-search:                        
ipv6.dns-options:                       (default)
ipv6.dns-priority:                      100
ipv6.addresses:                         
ipv6.gateway:                           --
ipv6.routes:                            
ipv6.route-metric:                      -1
ipv6.ignore-auto-routes:                no
ipv6.ignore-auto-dns:                   no
ipv6.never-default:                     no
ipv6.may-fail:                          yes
ipv6.ip6-privacy:                       -1 (unknown)
ipv6.addr-gen-mode:                     stable-privacy
ipv6.dhcp-send-hostname:                yes
ipv6.dhcp-hostname:                     --
ipv6.token:                             --
tun.mode:                               1 (tun)
tun.owner:                              --
tun.group:                              --
tun.pi:                                 no
tun.vnet-hdr:                           no
tun.multi-queue:                        no
GENERAL.NAME:                           tun0
GENERAL.UUID:                           a61ca484-3ca9-4e88-b6e1-574b4e17ca54
GENERAL.DEVICES:                        tun0
GENERAL.STATE:                          activated
GENERAL.DEFAULT:                        yes
GENERAL.DEFAULT6:                       no
GENERAL.VPN:                            no
GENERAL.ZONE:                           --
GENERAL.DBUS-PATH:                      /org/freedesktop/NetworkManager/ActiveConnection/4
GENERAL.CON-PATH:                       /org/freedesktop/NetworkManager/Settings/4
GENERAL.SPEC-OBJECT:                    /
GENERAL.MASTER-PATH:                    --
IP4.ADDRESS[1]:                         10.38.1.6/32
IP4.GATEWAY:                            10.38.1.5
IP4.ROUTE[1]:                           dst = 10.38.1.1/32, nh = 10.38.1.5, mt = 50
IP6.ADDRESS[1]:                         fe80::376b:6f85:5cb7:142/64
IP6.GATEWAY:
                            
**********
extended test from https://dnsleaktest.com

Test complete


Query round Progress... Servers found
  1          ......              2
  2          ......              1
  3          ......              2
  4          ......              1
  5          ......              1
  6          ......              2
IP	Hostname	ISP	Country
173.239.219.2	ip-2-219-239-173.east.us.northamericancoax.com	LogicWeb Inc	United States
71.242.0.136	none	Verizon Internet Services	United States
71.242.0.214	none	Verizon Internet Services	United States

**********
As you can see, the 'routing-only domain' line "DNS Domain: ~." is missing and DNS leaks are clearly happening while connected to the VPN as queries are being routed to the ISP (Verizon in this case).
Comment 8 Nick Stommel 2017-06-13 00:21:58 UTC
After issuing the systemd-resolved API method/bus call SetLinkDomains(5 (the if_index of the vpn con-id 'tun0' in this case), ".", TRUE), the output of systemd-resolve --status for the vpn con-id 'tun0' looks like this:

Link 5 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218
          DNS Domain: ~.

and there are no DNS leaks, as the routing-only domain is used.

**********
From SYSTEMD.NETWORK(5):

"The "routing-only" domain "~." (the tilde indicating definition of a routing domain, the dot referring to the DNS root domain which is the implied suffix of all valid DNS names) has special effect. It causes all DNS traffic which does not match another configured domain routing entry to be routed to DNS servers specified for this interface. This setting is useful to prefer a certain set of DNS servers if a link on which they are connected is available.

This setting is read by systemd-resolved.service(8). "Search domains" correspond to the domain and search entries in resolv.conf(5). Domain name routing has no equivalent in the traditional glibc API, which has no concept of domain name servers limited to a specific link."

**********
So the problem here is that when using network-manager-openvpn/network-manager-openvpn-gnome through the network-manager, there is no option to specify the routing-only domain, and even manually specifying the DNS servers and entering "." or "~." under "Edit Connections"->"<VPN Connection Name>"->"IPv4 Settings"->"Method=Automatic (VPN) addresses only"->"Search Domains" has no effect (i.e. the routing-only domain is not set).

Then we have a problem where DNS queries leak by design of systemd-resolved. But fortunately, also by design, we can choose to use the'routing-only domain', which solves the problem of DNS leaks using systemd-resolved. But the network-manager does not seem to be able to issue this call for some reason, perhaps because it views "." or "~." as invalid 'Search Domains' and doesn't parse or send that information correctly to nm-systemd-resolved.c. 

So...my patch is basically a stopgap measure to force the network-manager to make the bus call for the routing-only domain on a VPN connection. However, this may not be desired for all VPN connections, so obviously it's not optimal. There needs to be a change here where we can manually specify the routing-only domain for a VPN connection to prevent DNS leaks. 

I think a lot of confusion on the launchpad thread is the fact that many bugs were marked duplicates of something that is actually two separate issues. Split-horizon DNS setups with openconnect are fundamentally different from DNS leaks over a 'total internet' VPN provider through which all domain-names are resolved like 'Private Internet Access' (or any number of privacy-focused VPN service providers), which I'm using through network-manager-openvpn.
Comment 9 Nick Stommel 2017-06-13 00:49:53 UTC
Rather odd behavior happens when trying to specify "." or "~." in the line "Edit Connections"->"<VPN Connection Name>"->"IPv4 Settings"->"Method=Automatic (VPN) addresses only"->"Search Domains".

**********
Here is the network config file where "." is specified under the "Search Domains" from /etc/NetworkManager/system-connections/US-East :

[connection]
id=US-East
uuid=cf291340-3c52-4347-8ce9-e609bdecec32
type=vpn
permissions=user:noctua:;
secondaries=
timestamp=1497311475

[vpn]
auth=SHA1
ca=/home/noctua/Documents/openvpn/openvpn-legacy-tcp/ca.crt
cipher=BF-CBC
comp-lzo=yes
connection-type=password
dev=tun
dev-type=tun
password-flags=1
proto-tcp=yes
remote=us-east.privateinternetaccess.com:443
remote-cert-tls=server
reneg-seconds=0
username=<my username here>
service-type=org.freedesktop.NetworkManager.openvpn

[ipv4]
dns=209.222.18.222;209.222.18.218;
dns-search=.;
ignore-auto-dns=true
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=ignore

**********
And THIS is the output of systemd-resolved for the cond-id 'tun0':

Link 5 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218


As you can see, the dns-search=.; is ignored entirely and "." is not passed to SetLinkDomains, the line "DNS Domain: ~." is missing.

**********
Here is the network config file where "~." is specified under the "Search Domains" from /etc/NetworkManager/system-connections/US-East :

[connection]
id=US-East
uuid=cf291340-3c52-4347-8ce9-e609bdecec32
type=vpn
permissions=user:noctua:;
secondaries=
timestamp=1497314475

[vpn]
auth=SHA1
ca=/home/noctua/Documents/openvpn/openvpn-legacy-tcp/ca.crt
cipher=BF-CBC
comp-lzo=yes
connection-type=password
dev=tun
dev-type=tun
password-flags=1
proto-tcp=yes
remote=us-east.privateinternetaccess.com:443
remote-cert-tls=server
reneg-seconds=0
username=<my username here>
service-type=org.freedesktop.NetworkManager.openvpn

[ipv4]
dns=209.222.18.222;209.222.18.218;
dns-search=~.;
ignore-auto-dns=true
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=ignore

**********
And THIS is the output of systemd-resolved for the cond-id 'tun0':

Link 9 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218
          DNS Domain: \126


**********
Something....wrong is happening when the network-manager parses the config file and sends the domains to call SetLinkDomains in nm-systemd-resolved.c. Because... '\126' is clearly not "." or "~.". In fact, it appears to be the octal value for the ASCII character "V" which...really makes no sense. The domains "." or "~." specified and correctly listed in the config file as dns-search=.; or dns-search=~.; are not being passed to SetLinkDomains in as is, which suggests a parsing error....or something in nm-systemd-resolved.c. 

I think the easiest solution would be to allow "." to be parsed as a valid domain name under the dns-search label. That would effectively allow us to choose to use the routing-only domain.
Comment 10 Nick Stommel 2017-06-13 00:53:55 UTC
TL;DR

I think the easiest solution would be to allow "." to be parsed as a valid domain name under the dns-search label when it is sent to nm-systemd-resolved.c . That would effectively allow us to choose to use the routing-only domain, thereby solving any problems with DNS leaks over network-manager-openvpn.
Comment 11 Nick Stommel 2017-06-13 02:45:54 UTC
That is, setting "Edit Connections"->"<VPN Connection Name>"->"IPv4 Settings"->"Method=Automatic (VPN) addresses only"->"Search Domains" to contain the string "." should correctly set the routing-only domain for that connection (the network config file for the VPN connection I showed above DOES correctly contain the line "dns-search-.;", but as shown above, nothing happens. Instead, I think it gets invalidated and thrown away during parsing as a non-domain ....somewhere (I really can't find where) and is not passed to SetLinkDomains in nm-systemd-resolved.c. I have attempted to trace the parsing of this dns-search label through the network manager but honestly I have no clue where it happens. This should be an easy fix though for someone familiar with the workings of the network manager dns-manager. Thanks for your help, I believe that allowing the valid use of "." to optionally specify the routing-only domain is the optimal solution.
Comment 12 Thomas Haller 2017-06-13 08:04:27 UTC
NM's suggested solution to avoid DNS leaks is setting ipv4.dns-priority to a negative value. What is the shortcoming of that and why is that not sufficient to achieve what you want?


I think the shortcoming is, that a negative ipv4.dns-priority basically disables DNS configuration for all other interfaces. Contrary to "~.", which still uses other interfaces to resolve names on other interfaces if they are FQDN and the domain is configured on the other interface.

The shortcoming of ~. is that it is a missing feature that only works with systemd-resolved. It's only better then ipv4.dns-priority in case of a rather unusual usecase (like, if you want to avoid DNS leaks, but still resolve some particular FQDN).
Comment 13 Nick Stommel 2017-06-13 17:54:21 UTC
Because when using systemd-resolved as a DNS manager (from Ubuntu 16.10 onward, this is NOT optional) a negative ipv4.dns-priority DOES NOT disable DNS configuration for all other interfaces. systemd-resolved could care less what priority the network-manager assigns a connection; it sends queries to all configured DNS servers on all interfaces (thereby leaking DNS queries by design). 

[credit to Vincent from the launchpad thread]
"I quote the systemd.resolved.service doc) "Multi-label names are routed to all local interfaces that have a DNS sever configured (...) If lookups are routed to multiple interfaces, the first successful response is returned".

So basically all the dns servers defined in all of your links are fair game. DNS requests are sent to all of them at the same time and whichever replies first win the day!"

So literally the only solution here to prevent DNS leaks is to allow for the setting of the routing-only domain ".", which [from SYSTEMD.NETWORK(5)] "causes all DNS traffic which does not match another configured domain routing entry to be routed to DNS servers specified for this interface." 

This problem is NOT solved in the current version of NM, please acknowledge the problem presented by systemd-resolved and provide a fix, such as optionally allowing for the setting of the routing-only domain as suggested above.
Comment 14 Thomas Haller 2017-06-13 18:34:19 UTC
(In reply to Nick Stommel from comment #13)
>  systemd-resolved could care less
> what priority the network-manager assigns a connection; it sends queries to
> all configured DNS servers on all interfaces (thereby leaking DNS queries by
> design). 

if the ipv4.dns-priority is negative, NM wouldn't even configure any other DNS servers in sd-resolved.

documented:

... Negative values have the special effect of excluding other configurations with a greater priority value; so in presence of at least a negative priority, only DNS servers from connections with the lowest priority value will be used."
Comment 15 Nick Stommel 2017-06-13 19:05:08 UTC
Okay...so how do I manually set the ipv4.dns-priority to be negative? Sorry, but it's not clear how to do this, a stepwise solution would be appreciated. 
If this cannot be accomplished manually and the network manager already sets the ipv4.dns-priority as negative for the vpn connection-id, it's clearly not working, as there are apparent DNS leaks (as shown above and by many other users in the launchpad thread) using network-manager-openvpn with systemd-resolved as the DNS manager.
Comment 16 Thomas Haller 2017-06-13 19:16:38 UTC
`nmcli connection modify "$NAME" ipv4.dns-priority -42`

the default value (if left at zero) is +50 (for VPN) and +100 for other.


Beniamino had a branch to support those special domains like "~.". I don't claim it shouldn't be done. But ipv4.dns-priority should give you something similar (not identical).
Comment 17 Nick Stommel 2017-06-13 20:54:37 UTC
(same results running command without backticks and with sudo)
noctua@corinth:~$ `nmcli connection modify "tun0" ipv4.dns-priority -42`

noctua@corinth:~$ systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 3 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218

Link 2 (wlo1)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 192.168.1.1
          DNS Domain: home

Extended Test results from https://dnsleaktest.com/
**********
Test complete


Query round Progress... Servers found
  1          ......              2
  2          ......              3
  3          ......              1
  4          ......              2
  5          ......              2
  6          ......              2
IP	Hostname	ISP	Country
71.242.0.150	none	Verizon Internet Services	United States
173.239.219.2	ip-2-219-239-173.east.us.northamericancoax.com	LogicWeb Inc	United States
71.242.0.174	none	Verizon Internet Services	United States
71.242.0.230	none	Verizon Internet Services	United States
**********

I still get DNS leaks where queries are clearly being sent to my ISP even if this command is run with/without sudo. So...that doesn't work and I believe my original claim about systemd-resolved ignoring connection priority stands.
Comment 18 Beniamino Galvani 2017-06-13 21:43:37 UTC
(In reply to Nick Stommel from comment #17)
> So...that doesn't work and I believe
> my original claim about systemd-resolved ignoring connection priority stands.

This seems really strange. The code handling priorities is independent from which DNS backend (resolv.conf,dnsmasq,sd-resolved) is configured.

Please check:

1) that you have applied the change to the right connection (you can see the list of connections with 'nmcli c'). Probably 'tun0' is the name of the in-memory connection created by NM for the tun0 device created by openvpn; instead you should apply it to VPN itself.

2) that you have re-activated the VPN connection after the change
Comment 19 Thomas Haller 2017-06-13 22:54:07 UTC
Seems there is a bug with dns-priority together with systemd-resolved.

Please review https://cgit.freedesktop.org/NetworkManager/NetworkManager/log/?h=th/fix-dns-priority-for-resolved-bgo783569
Comment 20 Nick Stommel 2017-06-13 23:08:50 UTC
Nope, same results unfortunately. I have tried basically everything:

noctua@corinth:~$ nmcli c
NAME      UUID                                  TYPE             DEVICE 
Bn80rrF8  11d6931f-8d5b-4f47-8a1e-38f33117811b  802-11-wireless  wlo1   
US-East   cf291340-3c52-4347-8ce9-e609bdecec32  vpn              wlo1   
tun0      2476d73d-a1b6-4b0f-adf3-552bd095d38a  tun              tun0   
tun0      0058ff81-1d38-4d8b-a486-a29619bb7b99  tun              --     
tun0      56ca187e-67bb-438e-b20a-9ed44c223619  tun              --     

Here's where I try every possible way to apply 'ipv4.dns-priority -42'

noctua@corinth:~$ sudo nmcli connection modify uuid  ipv4.dns-priority -42
0058ff81-1d38-4d8b-a486-a29619bb7b99  56ca187e-67bb-438e-b20a-9ed44c223619
11d6931f-8d5b-4f47-8a1e-38f33117811b  cf291340-3c52-4347-8ce9-e609bdecec32
2476d73d-a1b6-4b0f-adf3-552bd095d38a  
noctua@corinth:~$ sudo nmcli connection modify uuid 0058ff81-1d38-4d8b-a486-a29619bb7b99 ipv4.dns-priority -42
noctua@corinth:~$ sudo nmcli connection modify uuid 56ca187e-67bb-438e-b20a-9ed44c223619 ipv4.dns-priority -42
noctua@corinth:~$ sudo nmcli connection modify uuid 2476d73d-a1b6-4b0f-adf3-552bd095d38a ipv4.dns-priority -42
noctua@corinth:~$ sudo nmcli connection modify uuid cf291340-3c52-4347-8ce9-e609bdecec32 ipv4.dns-priority -42
noctua@corinth:~$ sudo nmcli connection modify tun0 ipv4.dns-priority -42noctua@corinth:~$ sudo nmcli connection modify US-East ipv4.dns-priority -42

And then I tried reactivating the connection each time successfully through the GUI and then through nmcli, for example:

noctua@corinth:~$ sudo nmcli connection modify US-East ipv4.dns-priority -42
noctua@corinth:~$ sudo nmcli connection down US-East 
Connection 'US-East' successfully deactivated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/10)
noctua@corinth:~$ sudo nmcli --ask connection up US-East 
A password is required to connect to 'US-East'.
Password (vpn.secrets.password): 
VPN connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/12)

I even tried:
noctua@corinth:~$ sudo nmcli connection reload tun0
noctua@corinth:~$ sudo nmcli connection reload US-East

But every single time, I still I get the same results testing for DNS leaks, that DNS queries are clearly being sent to my ISP (Verizon in this case). So...this doesn't appear to work :( 

I think the only real solution here to stop DNS leaks over NM-VPN connections is to provide an option to specify the routing-only domain or make "." a valid search domain name on systems that force the use of systemd-resolved (so...basically every major Linux distro including those based off Ubuntu, Arch Linux, AND the upcoming release of Debian Stretch). This is a rather serious problem, hence the kinda hackish patch I made forcing the systemd-resolved method SetLinkDomains to accept the routing-only domain ".". Ultimately, I think there needs to be the OPTION to specify the routing-only domain to prevent DNS leaks. 

All those posts on the original launchpad thread, the Arch forums, and various duplicate bug threads of the original launchpad bug https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1624317 where users put the line 'dhcp-option DOMAIN-ROUTE .' in client openvpn config files in conjunction with the (now available in several package repositories) update-systemd-resolved script at https://github.com/jonathanio/update-systemd-resolved all amount to basically the same thing I did with my less-than-optimal patch: calling the systemd-resolved API method SetLinkDomains using the routing-only domain ".". So I think supporting this is very important indeed. 

Even more simply, "." could be allowed/parsed as a valid search domain when reading the dns-search line of a config file in /etc/NetworkManager/system-connections/<con-id> containing something like
[ipv4]
dns=209.222.18.222;209.222.18.218;
dns-search=.;
ignore-auto-dns=true
method=auto

I am positive that domains otherwise considered 'valid' ("." or "~." don't appear to be allowed, see above posts) are passed to SetLinkDomains. For example, if I set dns-search=bogusdomain; in the config file (or through the GUI) I end up with the following:

noctua@corinth:~$ sudo cat /etc/NetworkManager/system-connections/US-East 
[sudo] password for noctua: 
[connection]
id=US-East
uuid=cf291340-3c52-4347-8ce9-e609bdecec32
type=vpn
permissions=user:noctua:;
secondaries=
timestamp=1497394667

[vpn]
auth=SHA1
ca=/home/noctua/Documents/openvpn/openvpn-legacy-tcp/ca.crt
cipher=BF-CBC
comp-lzo=yes
connection-type=password
dev=tun
dev-type=tun
password-flags=1
proto-tcp=yes
remote=us-east.privateinternetaccess.com:443
remote-cert-tls=server
reneg-seconds=0
username=<my username here>
service-type=org.freedesktop.NetworkManager.openvpn

[ipv4]
dns=209.222.18.222;209.222.18.218;
dns-priority=-42
dns-search=bogusdomain;
method=auto

[ipv6]
addr-gen-mode=stable-privacy
dns-search=
ip6-privacy=0
method=ignore


Now notice the line under tun0 that says 'DNS Domain: bogusdomain'

noctua@corinth:~$ systemd-resolve --status 
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 16 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218
          DNS Domain: bogusdomain

Link 2 (wlo1)
      Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 192.168.1.1
          DNS Domain: home

So theoretically, all we need to do is allow "." to be parsed as a valid domain here.
Comment 21 Nick Stommel 2017-06-13 23:38:51 UTC
Oh sorry, I didn't see your patch. I will see if I can backport the patch, thanks Thomas for looking into it! Also, um why dns-priority -42? Can it be any (reasonably sized) negative number?
Comment 22 Nick Stommel 2017-06-14 03:36:28 UTC
Hmmmmm...I can't seem to get a backport working, but I somehow managed to get the git-master patched with your work from https://cgit.freedesktop.org/NetworkManager/NetworkManager/log/?h=th/fix-dns-priority-for-resolved-bgo783569
up and running. And...it appears like we don't have DNS leaks! Here is the output of systemd-resolved --status:

noctua@corinth:~$ systemd-resolve --status
Global
         DNS Servers: 209.222.18.222
                      209.222.18.218
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 3 (tun0)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 2 (wlo1)
      Current Scopes: LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

and here is the output from the extended test of dnsleaktest.com:

Test complete

Query round Progress... Servers found
  1          ......              1
  2          ......              1
  3          ......              1
  4          ......              1
  5          ......              1
  6          ......              1
IP	Hostname	ISP	Country
173.239.216.29	ip-29-216-239-173.east.us.northamericancoax.com	LogicWeb Inc	United States

...so it looks like this *might* be fixed upstream! Would it be possible for you to backport this fix for the Ubuntu package maintainers? I'm afraid I tried but nothing different happened, so this might be dependent on a number of previous commits or stuff not found in the current zesty package. It would be amazing to have a solution for this 'downstream', thanks for your work!
Comment 23 Nick Stommel 2017-06-14 04:06:35 UTC
Edit: I should add, that this works ONLY if ipv4.dns-priority is set to be negative in the config file in .../etc/NetworkManager/system-connections/<con-name> such that it contains something like:
[ipv4]
dns-priority=-42
dns-search=
method=auto 

When I properly placed "[main] dns=systemd-resolved" in the ../etc/NetworkManager/NetworkManager.conf file (I actually didn't have systemd-resolved properly configured as the dns-manager when I manually installed the NM git master with your patches applied), this is the (correct) output of systemd-resolved --status:

noctua@corinth:~$ systemd-resolve --status
Global
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 5 (tun0)
      Current Scopes: DNS
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no
         DNS Servers: 209.222.18.222
                      209.222.18.218

Link 2 (wlo1)
      Current Scopes: LLMNR/IPv4 LLMNR/IPv6
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

***********

So it works! It would be amazing if you could get this fix backported successfully for the Ubuntu folks, I haven't had much luck. Thank you so much!
Comment 24 Beniamino Galvani 2017-06-19 12:56:31 UTC
Branch th/fix-dns-priority-for-resolved-bgo783569 looks good to me
Comment 26 Thomas Haller 2017-06-19 13:27:36 UTC
ipv4.dns-priority should now work as intended.

Then there is bug 746422 which is about adding support for routing-only domains for systemd-resolved (still work in progress).

I think this fixes the present bug.



I think the original patch is not correct,  because it hard-codes an exclusive DNS configuration for VPN (which is not always wanted, and a change in behavior).
Also, it determines whether there is a VPN based on the link-type and doesn't work if you have multiple VPNs active at the same time.

Anyway, I think this issue is fixed (or the remaining parts are a duplicate of bug 746422).


Closing as fixed. Please reopen if something is missing. Thanks.
Comment 27 Nick Stommel 2017-06-22 01:43:59 UTC
I successfully backported the merged branch to the current version of the network-manager package in the Ubuntu 17.04 repositories! See https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1624317/comments/81
Thanks so much for your work :)