After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 349151 - [review] dcbw/vpn-reconnect: automatically reconnect VPN if dropped
[review] dcbw/vpn-reconnect: automatically reconnect VPN if dropped
Status: RESOLVED FIXED
Product: NetworkManager
Classification: Platform
Component: VPN (general)
git master
Other Linux
: Normal major
: ---
Assigned To: Dan Williams
NetworkManager maintainer(s)
: 439134 562670 581562 582295 607106 680955 (view as bug list)
Depends on:
Blocks: nm-1.0
 
 
Reported: 2006-07-29 09:47 UTC by Ed Catmur
Modified: 2017-10-26 14:32 UTC
See Also:
GNOME target: ---
GNOME version: 2.15/2.16


Attachments
VPN auto-connect script (4.74 KB, application/octet-stream)
2010-04-14 04:12 UTC, Jacob
  Details
bgo349151-vpn-reconnect.patch (2.08 KB, patch)
2014-09-26 20:59 UTC, Dan Williams
none Details | Review

Description Ed Catmur 2006-07-29 09:47:18 UTC
When the network connection goes down and up again, any VPN connections are disconnected. NetworkManager should remember VPN state and attempt to restore VPN connections when the underlying network comes back up.

Related: it would be useful if one could set NetworkManager to automatically connect to a VPN when starting up.
Comment 1 Antony Mee 2006-08-05 14:12:48 UTC
Remembering what the state of secondary connections was before a disconnect would be quite difficult to do nicely. I don't think for example that one would always want VPN connections to come back to life after a disconnect-reconnect.

Ed's second comment however would be very useful, and would perhaps solve the first issue in most cases.  I for example use several open wireless networks that use VPN for security (ie. wireless only provides access to the VPN server).  In souch cases the wireless is useless without the paired VPN connection and it would be ideal if the VPN auto-started when that wireless connection came alive.  Such auto starting would also bring the VPN up if the wireless 'reconnected'.
Comment 2 Mark Stosberg 2007-03-13 22:40:24 UTC
I can confirm that having the VPN come up automatically would be a nice feature and fits in with the NetworkManager "Just Works" philosophy. 

This this bug should be changed to the "CONFIRMED" state. 

  Mark
Comment 3 Tom Verdaat 2007-12-20 22:12:00 UTC
I would like to confirm and support the request for autoconnect of VPN. It would be nice if something like an autoconnect checkbox could be added to the vpn settings dialogs. Lets keep it simple and have it autoconnect using *any* available network connection.
Comment 4 Christopher Aillon 2007-12-28 21:11:31 UTC
Should be made easier by the work ongoing on HEAD...
Comment 5 Christopher Aillon 2007-12-29 15:50:20 UTC
*** Bug 439134 has been marked as a duplicate of this bug. ***
Comment 6 tonnzor 2008-06-01 18:15:25 UTC
+1

My VPN (PPTP) connection is often lost. Please, add an option to reconnect.
Comment 7 Hugo 2008-11-29 12:30:04 UTC
The latest Network Manager in Ubuntu 8.10 has this feature, so I guess it can be closed.
Comment 8 Dan Williams 2008-11-29 21:32:53 UTC
*** Bug 562670 has been marked as a duplicate of this bug. ***
Comment 9 Hugo 2008-12-06 23:35:59 UTC
In #562670 I tried reporting that the functionality seems to be implemented, but does not work yet. It got marked as a duplicate of this one, so I will make my plea here to have it fixed.

I will happily supply debug information. 
Comment 10 Dan Williams 2009-05-07 23:44:29 UTC
VPNs should automatically reconnect if they were disconnected.

VPNs should also be able to "tied" to another connection so that if that connection is activated, the VPN is too.
Comment 11 Dan Williams 2009-05-07 23:44:35 UTC
*** Bug 581562 has been marked as a duplicate of this bug. ***
Comment 12 Mark Stosberg 2009-05-12 13:33:31 UTC
I vote for a better bug name than:

"[enh] automatic VPN stuff"

I recommend:

"NetworkManager should attempt to restore VPN connections"
Comment 13 Dan Williams 2009-05-12 22:49:56 UTC
*** Bug 582295 has been marked as a duplicate of this bug. ***
Comment 14 Max 2009-05-18 17:09:01 UTC
I can confirm that this bug is still present on Gnome 2.26.1 (Ubuntu 9.10).
Comment 15 Miroslav 2009-06-08 20:29:37 UTC
Still does not reconnect while connected to Wi-Fi. Ubuntu 9.10
Comment 16 Henrik Johansson 2009-07-24 11:51:40 UTC
+1
Comment 17 Shwan 2009-10-03 09:54:15 UTC
+1, Is there any plans for resolving this issue as it is critical actually , a real security issue for me
Comment 18 c3679055 2009-10-17 11:48:16 UTC
any timeline?
Comment 19 markus 2009-10-27 09:06:00 UTC
Heyy. This really bothers me and many others. I suppose it's not a very hard thing to fix this (for someone who is already familiar with the code)?
Comment 20 sniper_ft 2009-11-24 20:47:01 UTC
vote for this.
Comment 21 jaa3 2009-11-24 21:43:21 UTC
+1
Comment 22 Undertaker 2009-11-25 11:20:42 UTC
This bug really bothers.
Comment 23 mail4kit 2009-11-29 16:15:42 UTC
+1
Comment 24 Jason Dagit 2010-01-14 19:30:59 UTC
+1
Comment 25 Dan Williams 2010-01-16 01:25:52 UTC
*** Bug 607106 has been marked as a duplicate of this bug. ***
Comment 26 Edgar Cherkasov 2010-01-28 14:43:51 UTC
+1
Comment 27 ironwit 2010-02-01 18:17:23 UTC
+1
Comment 28 sunbird 2010-02-12 14:43:00 UTC
+1
Comment 29 Steve 2010-02-25 23:54:49 UTC
I don't know if it is related, but my OpenVPN connection is set to connect automatically, but does not do so when my 3G/Mobile connection comes up (automatically), presumably because it takes several seconds for this to come up, by which time the 'automatic' openvpn connect has already tried and failed.
Comment 30 Jacob 2010-02-28 14:52:30 UTC
+1
Comment 31 Alexey Kosilin 2010-04-10 11:59:12 UTC
+1
Comment 32 Shwan 2010-04-10 12:19:40 UTC
why is this bug market as prio. Normal , it higher than that, a security issue actually... and Why is the status NEW it is 4 years old! I will make a new report
Comment 33 Mark Stosberg 2010-04-12 14:33:27 UTC
I don't see how creating a new bug report were help if there is difficulty getting to ones that already exist.
Comment 34 Shwan 2010-04-12 19:43:58 UTC
I didn't made one as there was many! All of you haveing this problem go an install kvpnc, it does reconnect if VPN i droped and it actually doesent dropp it like NM, if the internet connection goes down it crashesh for me.. maybe just a lucid bug, anyways The connection doesnt drop normally but the vpn does and KVPNC just works!
Comment 35 Jacob 2010-04-12 20:51:30 UTC
I have a script which auto-connects the VPN if a wireless connection with a specific name goes online. Whenever my wireless drops and comes back up, the VPN goes down with it, but once the wireless reconnects the VPN goes up automatically too. It's a script I got from another NetworkManager bug report and I modified it quite a bit to take actual names rather than connection ID's. You just drop it in /etc/NetworkManager/dispatcher.d/ with the right permissions and it just does its thing.

Because NetworkManager reports certain preconfigured connections as "active" even when they're down, sometimes the script tries to raise the VPN on the wrong wireless network, because NetworkManager reports multiple networks as being online even when they're not. It's a little odd, but it just means you have to disconnect the VPN by hand.

If anyone wants this script despite some of the annoyances that come with it, and some of the difficulty in setting it up (editing a script by hand; proper permissions; verifying it's working through syslog) I'll upload it here.
Comment 36 julian 2010-04-12 22:03:55 UTC
Ubuntu 9.10, and there box to tick is there, but it doesn't seem to work...+1!
Comment 37 Steve 2010-04-13 18:45:59 UTC
Jacob, I'd quite like a look at your script, please!
Comment 38 Jacob 2010-04-14 04:12:25 UTC
Created attachment 158670 [details]
VPN auto-connect script

Here's the script I use to work around this bug. Unfortunately, under certain circumstances this script can be too aggressive and autostart the VPN when you don't want it. Please review bug #418745 for more details on this quirk.

I will not answer support questions here. Please hover over my name directly above this comment to get my e-mail address.

I finally found the author  of the original script: <http://mail.gnome.org/archives/networkmanager-list/2009-February/msg00098.html>. I've since heavily edited to be easier to use. The two biggest improvements are that it uses names, not UUID's, to look up a connection (but that'll confuse it if you have two connections with the same name!) and it logs to syslog so one can see what it's thinking.

I believe this script needs NetworkManager 0.7, since 0.7 has major changes to its dbus interface that this script takes advantage of.

You'll want to change the following lines...
  VPN="<school>"
  ACTIVE_CONNECTION="Auto <school>"
  UID=1000
...to fit your own system. Drop the script into /etc/NetworkManager/dispatcher.d/, make root the owner and the group, and make it executable by everyone. (That's how I have it; you may not need such broad execution permissions.)

Now reconnect your network and watch your syslog. Messages like "No such VPN in NetworkManager configuration" or "No such connection in NetworkManager configuration" mean you haven't set up VPN or ACTIVE_CONNECTION with the exact names.

Hopefully a developer fixes this sooner rather than later so we don't have to rely on this script. I'd hate to see this become the reason to ignore the bug.
Comment 39 Jacob 2010-04-14 04:14:03 UTC
(In reply to comment #38)
> Please review bug #418745 for more details on this quirk.

I meant bug #615702.
Comment 40 Jacob 2010-04-14 04:15:51 UTC
Authors deserve their credit, so since I found the original author, I'll put a link to his post here: http://mail.gnome.org/archives/networkmanager-list/2009-February/msg00098.html
Comment 41 Jacob 2010-04-14 04:16:33 UTC
...and that last comment belonged in bug #615702. I'm really sorry for the spam.
Comment 42 Sven M 2010-08-09 11:59:29 UTC
I stumpled about this feature request while searching for auto connection for vpn.

Would be nice to have this feature in a future release.
Comment 43 Johannes Storm 2011-01-23 11:43:56 UTC
perhabs someone likes a new much smaller script based on nmcli of mn-8.1
and is writen as shell-script:

#! /bin/bash

REQUIRED_CONNECTION_NAME="<Connection1>"
VPN_CONNECTION_NAME="<VpnConnection1>"


activ_con=$(nmcli con status | grep "${REQUIRED_CONNECTION_NAME}")
activ_vpn=$(nmcli con status | grep "${VPN_CONNECTION_NAME}")
if [ "${activ_con}" -a ! "${activ_vpn}" ];
then
    nmcli con up id "${VPN_CONNECTION_NAME}"
fi
Comment 44 Envel 2011-06-23 17:23:40 UTC
Is there any progress on this bug? The issue is annoying. I don't think that it's a big deal to fix it.
Comment 45 Mark Stosberg 2011-06-23 17:42:47 UTC
Envel, since it's not a big deal, would you like to prepare the patch?
Comment 46 Envel 2011-06-23 20:13:00 UTC
Are there any problems with this that I don't know?
Comment 47 Yarik 2011-10-22 09:52:58 UTC
FC14, +1
Comment 48 Domen Kožar 2012-01-01 21:36:09 UTC
Works with NM 0.9:

https://gist.github.com/1547663
Comment 49 Pavel Simerda 2012-08-16 20:49:41 UTC
This seems to be tightly related to bug 680955 which also have security implications. I'm afraid the whole use case of using VPN for securing some/all outgoing communication should be reconsidered.
Comment 50 Adam 2012-09-05 16:11:04 UTC
I'd like to help clarify things here as there are numerous different bug reports which all differ slightly from each other, but around the same problem. This is how I see it.

Problems:

- The current 'Connect automatically' option when editing a VPN does nothing.
- The VPN does not connect on startup of a (wireless) connection.
- The VPN does not re-connect if the (wireless) connection is lost and then re-connects.

Solution:

- Bind the VPN to a specific (wireless) network connection. 

Modify the 'Connect automatically' option to 'Connect automatically to: [wireless network connection name]'. So when that network connection is started, the VPN automatically connects. And if the network connection is lost and it re-connects, the VPN also reconnects.


Some fixes have been suggested in this report and at launchpad (bit.ly/fgUqtm). An application called 'vpnautoconnect' (http://sourceforge.net/projects/vpnautoconnect/) was suggested and seems to do the same job. Domen Kožar has also posted that https://gist.github.com/1547663 works with NM 0.9.

I know I have posted multiple problems in one, but this is due to different bug reports being marked as duplicates of this one. This single solution will resolve all the bugs.

I would also recommend this be marked as a security issue. Private information is at risk of being leaked across an insecure network if the connect is lost. VPN usage has increased dramatically in recent years as privacy and anonymity become more essential. Network Manager needs to keep up with this demand.

Many thanks to anyone who works on this. Sorry I can't code myself, just trying to help out a little with my input.
Comment 51 Pavel Simerda 2012-09-05 16:26:38 UTC
> - The current 'Connect automatically' option when editing a VPN does nothing.
> - The VPN does not connect on startup of a (wireless) connection.
> - The VPN does not re-connect if the (wireless) connection is lost and then
> re-connects.
> 
> Solution:
> 
> - Bind the VPN to a specific (wireless) network connection. 

That's what bug 560471 is about and it's being worked on. And it doesn't
solve the third point above which applies even to manually connected VPN.

> Modify the 'Connect automatically' option to 'Connect automatically to:
> [wireless network connection name]'.

This would in the exact opposite direction, see 560471.

> I know I have posted multiple problems in one, but this is due to different bug
> reports being marked as duplicates of this one. This single solution will
> resolve all the bugs.

Unfortunately not.
 
> I would also recommend this be marked as a security issue.

How?

> VPN usage has increased dramatically in recent years as privacy and anonymity
> become more essential. Network Manager needs to keep up with this demand.
> 
> Many thanks to anyone who works on this. Sorry I can't code myself,
> just trying to help out a little with my input.

Sure, no problem.
Comment 52 Steve 2012-09-05 16:52:12 UTC
(In reply to comment #50)

> Solution:
> 
> - Bind the VPN to a specific (wireless) network connection. 
> 
> Modify the 'Connect automatically' option to 'Connect automatically to:
> [wireless network connection name]'. So when that network connection is
> started, the VPN automatically connects. And if the network connection is lost
> and it re-connects, the VPN also reconnects.

  I'd actually like my VPN to connect automatically, when *any* Internet connection comes up, beit Ethernet, my home or office wifi, or mobile phone tethering.

  Some of these are flakey, and go up and down (esp, for example if I enable tethering on the phone, after booting my lappy).  What would be really really nice is for my VPN to come up, with *any* network connection, and further for it to re-connect the VPN, if the VPN link goes down, but the network stays up.

  What I would be less than keen on, is to have to create 4 identical copies of the VPN connection, for each network connection ... and that's without mentioning wifi hotspot access, so at the very least, it would be necessary to bind the VPN to one or *more* specific network connections.
Comment 53 Domen Kožar 2012-09-05 16:57:34 UTC
Currently this is a security issue since not all users will check/know if VPN is actually connected all the time due to existing checkbox not working.

What would be the next step when we have basic implementation for VPN autoconnect (something basic as my script I posted) is for NetworkManager to make sure no packet goes ever out of any interface without a secure line.
Comment 54 Pavel Simerda 2012-09-05 18:52:17 UTC
>   I'd actually like my VPN to connect automatically, when *any* Internet
> connection comes up, beit Ethernet, my home or office wifi, or mobile phone
> tethering.

Agreed.

>   Some of these are flakey, and go up and down (esp, for example if I enable
> tethering on the phone, after booting my lappy).  What would be really really
> nice is for my VPN to come up, with *any* network connection, and further for
> it to re-connect the VPN, if the VPN link goes down, but the network stays up.

Yep. A working connection should be enough to start the VPN for users with computers that are *always* used with VPN.

>   What I would be less than keen on, is to have to create 4 identical copies of
> the VPN connection, for each network connection ...

That would be painful. But even if these all bugs are fixed, you will have times
when the physical connections run but VPN does not. There are ways to avoid it
but these are not so easy. This is not fixable with just plain reconnection.

One way would be to cooperate with firewalld, another way would be to (by configuration) avoid setting up the default gateway entirely but this is still susceptible to rogue network attacks with the knowledge of what you want to access.

Firewall would have to block all outgoing connections except the VPN setup
until the default route goes through the VPN. An even then it should not let
your applications connect to services on the local network.

> and that's without
> mentioning wifi hotspot access, so at the very least, it would be necessary to
> bind the VPN to one or *more* specific network connections.

It will be exactly the other way round. (Physical) connections will trigger VPN
connections and delay the broadcasted status until VPN is set up. See bug 560471.
Comment 55 Pavel Simerda 2012-09-05 18:55:55 UTC
> What would be the next step when we have basic implementation for VPN
> autoconnect (something basic as my script I posted) is for NetworkManager to
> make sure no packet goes ever out of any interface without a secure line.

Your requrement is badly stated. Actually without letting packets out, you
won't start the VPN. This can only be handled by a carefully set up dynamic
firewall.

Actually, this is what IPsec was build for and not generic VPNs. IPsec sets
up security policy, OpenVPN just opens a tunnel, nothing more. This probably
can't be handled in a generic way.
Comment 56 Adam 2012-10-05 17:43:55 UTC
So what about the problem that the 'Connect Automatically' checkbox does nothing?

Many users on Launchpad are confused about what it does and are reporting it as a bug. As this won't fix, can we get the checkbox removed?
Comment 57 Pavel Simerda 2012-10-05 17:50:48 UTC
It would be nice to have autoconnect working.
Comment 58 Adam 2012-10-05 18:45:14 UTC
That would be nice, but I thought it wouldn't fix? Michael at http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=589533 said it doesn't work and should probably be greyed out.

What is it's intended function anyway? Connect on start up? Connect when wireless connection is found? I don't quite understand.

Either way, the bug has been open on Launchpad for over 4 years so I'm guessing it's not going to be fixed anytime soon.

Surely, removing the option for now and then putting it back once it's working is the best option?
Comment 59 Pavel Simerda 2012-10-05 20:52:52 UTC
> What is it's intended function anyway? Connect on start up? Connect when
> wireless connection is found? I don't quite understand.

Automatically connect once it's possible.

> Either way, the bug has been open on Launchpad for over 4 years so I'm guessing
> it's not going to be fixed anytime soon.

Launchpad is not really a good place for NetworkManager bug reports.

> Surely, removing the option for now and then putting it back once it's working
> is the best option?

I would personally prefer it fixed. With the next release, if it's possible. But of course you can make it greyed out by a distribution patch if it bothers you.
Comment 60 Adam 2012-10-05 23:20:24 UTC
It doesn't bother me, I was just trying to help
Comment 61 Pavel Simerda 2012-10-07 15:27:01 UTC
Thanks.
Comment 62 Dan Winship 2013-05-02 16:05:04 UTC
NM bugzilla reorganization... sorry for the bug spam.
Comment 63 Dan Williams 2014-09-26 20:59:37 UTC
Created attachment 287210 [details] [review]
bgo349151-vpn-reconnect.patch

Reconnect the VPN connection if it was previously connected, but failed, and it wasn't the underlying device that failed.  Turns out to be simpler than I though, thanks to a lot of the reogranization we've done in the VPN code in the last year.
Comment 64 Dan Williams 2014-09-26 21:07:36 UTC
Note that this only covers the case of reconnect on failure, it doesn't do anything for "autoconnect", since that has slightly different semantics and would probably be covered by stuff like bug 680955.
Comment 65 Dan Winship 2014-09-29 15:16:42 UTC
(In reply to comment #63)
> Reconnect the VPN connection if it was previously connected, but failed, and it
> wasn't the underlying device that failed.

This doesn't really cover that many cases does it? The initial request from comment 0 was for "when the network connection goes down and up again"

The patch looks fine... I assume it's tested, and there's no problem with calling nm_manager_activate_connection() from within vpn_connection_state_changed()?
Comment 66 David Woodhouse 2014-09-29 15:47:01 UTC
If the VPN fails, I have to click somewhere to re-activate it. Then the authentication dialog pops up and I have to enter my password or the next one-time-key or whatever. Once I go through the authentication, I am finally back on the VPN except I have a *different* IP address, because Cisco's IP allocation appears to suck. So all my existing connections are broken.

Aside from the fact that it doesn't fix many of the interesting cases, this patch doesn't even fix anything but that initial 'click to reconnect' bit. The majority of the pain is still there.

I've been looking at what it would take to fix the rest of the issues. At http://david.woodhou.se/openconnect-reconnect.patch there's support for making openconnect remember its session authentication cookie and re-use it. I have to manually run nm-openconnect-service with --persist, and I could do with a way of knowing *why* the connection gets torn down each time. If it's taken down due to the underlying connection going down, I want a seamless reconnect. If it's taken down manually, I want to re-authenticate next time.

So there are few things to think about in how we handle reconnection...
Comment 67 David Woodhouse 2014-09-29 15:52:35 UTC
Note that for the case this patch *does* fix, I wonder if we'd be better off handling that in the specific VPN service anyway?

Reconnecting after a lost connection is something we want to do carefully, with appropriate delays between attempts to avoid mounting an unintentional denial of service attack. Or not at all depending on the *reason* we lost the connection. And that reason may be very specific to the type of VPN client, so not something it's easy to handle generically. Some VPN clients do it for themselves anyway, rather than just exiting when their first connection is lost.
Comment 68 Dan Williams 2014-09-30 20:42:22 UTC
(In reply to comment #67)
> Note that for the case this patch *does* fix, I wonder if we'd be better off
> handling that in the specific VPN service anyway?
> 
> Reconnecting after a lost connection is something we want to do carefully, with
> appropriate delays between attempts to avoid mounting an unintentional denial
> of service attack. Or not at all depending on the *reason* we lost the
> connection. And that reason may be very specific to the type of VPN client, so
> not something it's easy to handle generically. Some VPN clients do it for
> themselves anyway, rather than just exiting when their first connection is
> lost.

Seamless reconnect is great, and if we can do that we should.

But clearly NM needs to be aware, because otherwise (especially with a reconnect backoff) there will be a period of time where you aren't actually connected.  Second, with VPNs that provide tunnel devices, that device may go away and come back depending on the VPN, so the IP config that NM has may well get deleted by the kernel, and when the VPN comes back, it could be completely different.

IIRC the only way to preserve your connections across reconnects is if the VPN uses a persistent tunnel device, the IP addresses/routes stay assigned to it, and the details are the same on reconnect.  If the device gets re-created, or the IP details get cleared, then all your TCP connections and bound UDP sockets will get closed.

There are also a couple different "VPN reconnect" use-cases that we are talking about; my patch only addresses #1:

1) reconnect on unexpected VPN failure (this bug)
2) reconnect over suspend/resume (bug 680955)
3) reconnect over link change (eg, undock/dock a laptop or WWAN -> WiFi)

Only #1 can really seamlessly reconnect with the same IP address details.  #2 and #3 have no guarantee of that, unless the VPN itself has the ability to re-assign the same IP address to the tunnel, and even then, since the underlying kernel interface has disconnected, we've already broken underlying connectivity.

openvpn has a "feature" where the server will accept connections from completely new IP addresses that have a known cookie & keys, and thus it can theoretically survive #2 and #3 without disconnecting the openvpn tunnel interface, as long as timeouts aren't exceeded.  Does openconnect have anything like that?
Comment 69 David Woodhouse 2014-09-30 21:13:54 UTC
(In reply to comment #68)
> openvpn has a "feature" where the server will accept connections from
> completely new IP addresses that have a known cookie & keys, and thus it can
> theoretically survive #2 and #3 without disconnecting the openvpn tunnel
> interface, as long as timeouts aren't exceeded.  Does openconnect have anything
> like that?

Yes, and the patch I showed makes use of it. If you reconnect with the same authentication cookie, you'll pick up the same session with the same IP address.

In fact, arguably *any* VPN whose IP allocation policy doesn't include "if the IP address this user had last time is still available, then give it back to them" is fairly stupid.


For openconnect we do need to know *why* a disconnect happens though, so we know whether to sign off and terminate the session (and thus invalidate the cookie), or whether we should use the cookie again. There are some configurations where each user is limited to one session at a time, where the current NM setup (without any of the changes we're discussing) doesn't work correctly. It *needs* to re-use the existing cookie, if the connection is lost without a sign-off.
Comment 70 Dan Williams 2014-10-07 15:23:00 UTC
David, does openconnect have a way of indicating that the tunnel is down and that it is trying to connect?

The only complication I see with keeping the tunnel up + addressed is that the tunnel might be "down" for an extended time, and I'd like to show status for that somehow.  Which requires that the VPN plugin be able to communicate back to NM when it was "down" but trying to reconnect, and NM needs to be able to indicate to the plugin to reconnect when a resume happens or interfaces switch around.

openvpn looks like it has some of this through the management interface, but I can't quite get it to work.
Comment 71 David Woodhouse 2014-10-07 15:50:10 UTC
It invokes vpnc-script with reason=reconnect when it comes back up. It is a matter of moments to make it invoke vpnc-script with a suitable reason string when the disconnect happens...
Comment 72 Dan Williams 2014-10-07 22:26:39 UTC
(In reply to comment #71)
> It invokes vpnc-script with reason=reconnect when it comes back up. It is a
> matter of moments to make it invoke vpnc-script with a suitable reason string
> when the disconnect happens...

Ok, here's what I propose:

1) a new "supports-reconnect=true" key in the plugin's .name file for those VPN plugins that can keep IP addressing alive temporarily while they attempt to reconnect after the connection is severed

2) a new Reconnect(asa{sv}) plugin method which NetworkManager calls to indicate link events (eg, switching connections/links), suspend/resume, or other events.  When this method is called, the plugin triggers a reconnect attempt in the VPN service itself.  For example, nm-openvpn-service would send SIGUSR1 to openvpn to request a reconnect.

3) a new TransientDisconnect plugin signal which the VPN plugin emits to indicate that the link has disconnected

----

Proposed flow, two cases:

a) VPN disconnect triggered by external reason (eg, something in the Internet broke)
a1) plugin notices disconnect and emits TransientDisconnect signal
a2) NetworkManager receives the TransientDisconnect signal and changes Connectivity=NM_CONNECTIVITY_LIMITED, but does *not* clean up the IP configuration.
a3) NM starts a timer (how long, 60s or 120s???) which if expired fully disconnects the connection
a4) VPN plugin re-establishes connectivity and emits Ip4Config/Ip6Config signals with any updated IP configuration
a5) NM applies the updated IP configuration to the tunnel interface
a6) profit

b) VPN disconnect triggered by link change or suspend/resume sequence
b1) NM disconnects old link - does nothing with VPN plugin, but this will remove the host-route to the VPN gateway that NM adds, which might cause VPN packets to fail with "no route to host" if the VPN claims the host's local subnet
b2) some amount of time passes
b3) NM connects new link; calls Reconnect() method of VPN plugin
b4) VPN plugin triggers a reconnect in the VPN service
b5) if the VPN plugin needs some new secrets, it can emit SecretsRequired() to request new secrets from the auth dialog (and should include any hint that the auth-dialog might need to process the reconnect request)
b5a) VPN auth dialog returns some secrets
b5b) NM calls the NewSecrets() method of the VPN plugin with these new secrets
b5c) VPN plugin continues connecting with the new secrets/info
a6) VPN plugin re-establishes connectivity and emits Ip4Config/Ip6Config signals with any updated IP configuration
a7) NM applies the updated IP configuration to the tunnel interface
a8) profit

Does that look like it should work for openconnect?  I think this sequence will work OK for openvpn and possibly vpnc.
Comment 73 Dan Williams 2014-10-07 22:28:47 UTC
Oh, a missed step for the link change case:

b2a) NM may set Connectivity=NM_CONNECTIVITY_LIMITED if the link change takes more than a short amount of time
Comment 74 Thomas Haller 2014-10-08 06:53:09 UTC
(In reply to comment #72)
> (In reply to comment #71)
> > It invokes vpnc-script with reason=reconnect when it comes back up. It is a
> > matter of moments to make it invoke vpnc-script with a suitable reason string
> > when the disconnect happens...
> 
> Ok, here's what I propose:
> 
> 1) a new "supports-reconnect=true" key in the plugin's .name file for those VPN
> plugins that can keep IP addressing alive temporarily while they attempt to
> reconnect after the connection is severed
> 
> 2) a new Reconnect(asa{sv}) plugin method which NetworkManager calls to
> indicate link events (eg, switching connections/links), suspend/resume, or
> other events.  When this method is called, the plugin triggers a reconnect
> attempt in the VPN service itself.  For example, nm-openvpn-service would send
> SIGUSR1 to openvpn to request a reconnect.
> 
> 3) a new TransientDisconnect plugin signal which the VPN plugin emits to
> indicate that the link has disconnected
> 
> ----
> 
> Proposed flow, two cases:
> 
> a) VPN disconnect triggered by external reason (eg, something in the Internet
> broke)
> a1) plugin notices disconnect and emits TransientDisconnect signal
> a2) NetworkManager receives the TransientDisconnect signal and changes
> Connectivity=NM_CONNECTIVITY_LIMITED, but does *not* clean up the IP
> configuration.
> a3) NM starts a timer (how long, 60s or 120s???) which if expired fully
> disconnects the connection
> a4) VPN plugin re-establishes connectivity and emits Ip4Config/Ip6Config
> signals with any updated IP configuration
> a5) NM applies the updated IP configuration to the tunnel interface
> a6) profit
> 
> b) VPN disconnect triggered by link change or suspend/resume sequence
> b1) NM disconnects old link - does nothing with VPN plugin, but this will
> remove the host-route to the VPN gateway that NM adds, which might cause VPN
> packets to fail with "no route to host" if the VPN claims the host's local
> subnet
> b2) some amount of time passes
> b3) NM connects new link; calls Reconnect() method of VPN plugin

b3-2) NM must also re-add the route to the VPN gateway (especially, if the VPN gets the default route).

> b4) VPN plugin triggers a reconnect in the VPN service
> b5) if the VPN plugin needs some new secrets, it can emit SecretsRequired() to
> request new secrets from the auth dialog (and should include any hint that the
> auth-dialog might need to process the reconnect request)
> b5a) VPN auth dialog returns some secrets
> b5b) NM calls the NewSecrets() method of the VPN plugin with these new secrets
> b5c) VPN plugin continues connecting with the new secrets/info
> a6) VPN plugin re-establishes connectivity and emits Ip4Config/Ip6Config
> signals with any updated IP configuration
> a7) NM applies the updated IP configuration to the tunnel interface
> a8) profit
> 
> Does that look like it should work for openconnect?  I think this sequence will
> work OK for openvpn and possibly vpnc.
Comment 75 David Woodhouse 2014-10-08 11:51:24 UTC
For OpenConnect your first case is basically just improving the UI visibility for something that OpenConnect already does. It will retry the connection every 10 seconds up to a configurable maximum timeout which defaults to 5 minutes, at which point it gives up.

In the context of NM I'm not sure why we really need to give up at all. This is your point (a3). If you're getting transient failures, and if the 'not working' status is clearly displayed, why *not* just keep trying to reestablish the connection? It's just a temporary routing outage somewhere which prevents us from reaching *some* servers.

Either way, I suppose we'd invoke OpenConnect with the --reconnect-timeout=0 option in order to *stop* doing this for itself, and let NM handle it.

Perhaps it should be NM which invokes the same 'Reconnect' operation, in fact. Why should this be different between the various cases (routing outage, link loss/change, suspend/resume, DHCP renew with different address)? In all of those cases OpenConnect *will* happily reconnect using the new underlying link and will reinvoke vpnc-script with "reason=reconnect". Or you can ask it not to, and invoke a 'Reconnect' method on the nm-openconnect-service which will actually spawn a new instance of /usr/sbin/openconnect.

Can we make the handling of all these cases basically the same?

And why a new 'Reconnect'? What was wrong with the existing Connect method? If you have enough secrets to reconnect, fine. If not, you need to ask for them. Which you'd already accounted for.
Comment 76 Dan Williams 2014-10-08 20:02:15 UTC
(In reply to comment #75)
> For OpenConnect your first case is basically just improving the UI visibility
> for something that OpenConnect already does. It will retry the connection every
> 10 seconds up to a configurable maximum timeout which defaults to 5 minutes, at
> which point it gives up.
> 
> In the context of NM I'm not sure why we really need to give up at all. This is
> your point (a3). If you're getting transient failures, and if the 'not working'
> status is clearly displayed, why *not* just keep trying to reestablish the
> connection? It's just a temporary routing outage somewhere which prevents us
> from reaching *some* servers.

Yeah I thought about that too, which makes VPNs "on until you turn them off, even if they aren't connected".

Which is different than other connections.  It would be like when you walk out of WiFi range but NM kept wlan0's IP address and tried to reassociate to the non-existent SSID forever, until you explicitly told NM to stop or switch to a different SSID.  Yes this is a straw-man because VPN isn't like an L2 connection, but it's still inconsistent.

If IP addressing is still configured while the VPN reconnects, there are some questions:

What about non-VPN stuff while the VPN is reconnecting, and the VPN doesn't support split DNS or you aren't running a caching nameserver?  It's easy to say "you should run a caching nameserver" but not every VPN admin sets the domain and not everyone can/will run a caching nameserver.

What is the experience if it takes 5 or 10 minutes to re-establish VPN connectivity, but you still have perfectly fine regular Internet connectivity?

What is the experience like for private connections (testing network, private LAN, etc) that don't have regular connectivity?  eg if you're connected to the VPN and then you suspend and plug into a private LAN and then resume?  Or if you resume on the work network that doesn't need VPN?

A the highest level, here are the major cases and some proposed solutions:

1) intermittent problem somewhere on the Internet: VPN handles reconnect itself, signals NM when connectivity drops and when it is restored; if VPN does not support this itself, NM could reconnect using a modified form of the patch I posted above

2) suspend/resume: if NM reconnects whatever connection was previously connected, or the new connection specifies the VPN in its secondaries property, NM allows the VPN to reconnect; otherwise NM terminates the VPN

3) same link change (due to carrier change, DHCP lease change, etc): see #2

4) cross link change: this is the complex one because we cannot detect when to retry and when it will fail.  For example, if your WiFi network requires the VPN but your wired network does not (like Red Hat's), and you go from undocked -> docked.  The VPN reconnect will never, ever succeed but tun0 will still have an IP address assigned, routes will still be directed through tun0, and VPN DNS server will still be configured.  Any thoughts on what we do here?

> Either way, I suppose we'd invoke OpenConnect with the --reconnect-timeout=0
> option in order to *stop* doing this for itself, and let NM handle it.

I think I'd actually rather let the plugins handle all the reconnect logic themselves, since they know best how their underlying daemon does it.  NM can give hints/signals about stuff that's happened, and the VPN can signal to NM about certain states (like transient disconnects) so NM can update it's connectivity state.  Does that sound valid?

> Perhaps it should be NM which invokes the same 'Reconnect' operation, in fact.
> Why should this be different between the various cases (routing outage, link
> loss/change, suspend/resume, DHCP renew with different address)? In all of
> those cases OpenConnect *will* happily reconnect using the new underlying link
> and will reinvoke vpnc-script with "reason=reconnect". Or you can ask it not
> to, and invoke a 'Reconnect' method on the nm-openconnect-service which will
> actually spawn a new instance of /usr/sbin/openconnect.
> 
> Can we make the handling of all these cases basically the same?

That's a great question.  Is there *any* reason that we might want to treat an intermittent VPN failure (concentrator reboot with a server message maybe?) differently than a link change?  I keep thinking "maybe" but I can't come up with any concrete cases for it.  Perhaps some will appear during implementation...

> And why a new 'Reconnect'? What was wrong with the existing Connect method? If
> you have enough secrets to reconnect, fine. If not, you need to ask for them.
> Which you'd already accounted for.

I thought there was value in the VPN plugin knowing about a reconnect versus a completely new connection attempt.  I guess if the VPN plugin keeps enough state internally, like which connection is already active, then calling Connect() again would be sufficient.

-----

So the outstanding issue in my mind is what to do about cross-link changes.  It's hard to automatically detect whether to keep the VPN up across those changes, and doing so unconditionally fails for known cases.

There's always the rabbit-hole of adding a "conflicting connections" property which you could set to say "always disconnect this connection when activating this other connection", kinda like the opposite of the existing 'secondaries' property.  I guess that could be useful in some other instances, but I don't really like this solution because it creates too many ambiguous inter-dependencies between connections, and too much potential for disastrous mis-configuration.

Another alternative is adding a VPN property like "link-change-reconnect" which defaults to FALSE to preserve existing behavior.  But when set to TRUE the VPN would be preserved across link changes and the user would have to manually disconnect.  However, I'd want to use this behavior at Red Hat, which means I run into the VPN-doesn't-work-on-ethernet problem, which means I'd want the conflicting connections thing, etc etc.

Other thoughts?
Comment 77 David Woodhouse 2014-10-08 22:29:51 UTC
(In reply to comment #76)
> (In reply to comment #75)
> > And why a new 'Reconnect'?
> 
> I thought there was value in the VPN plugin knowing about a reconnect versus a
> completely new connection attempt.  I guess if the VPN plugin keeps enough
> state internally, like which connection is already active, then calling
> Connect() again would be sufficient.

I suspect that at least for OpenConnect we need that state *anyway*. See what I said in comment 69 about issues with the current setup in cases where the server limits the number of active sessions you can have. As things stand, we are *abandoning* an active session without ever terminating it (if we lose carrier, etc.), and then we try to make a *new* session... and some servers say 'no'.



> 4) cross link change: this is the complex one because we cannot detect when to
> retry and when it will fail.  For example, if your WiFi network requires the
> VPN but your wired network does not (like Red Hat's), and you go from undocked
> -> docked. 

Well... there's more fun to be had here. Imagine my email client is configured, for a certain account, to know that "I must have access to the corporate network". When I fire up my mail client I actually want it to *automatically* trigger a VPN connection... if it needs to. But not if I'm already on the corporate network directly with a physical connection. So perhaps we want a more general mechanism for determining *which* physical networks a given VPN should operate over, which would solve both at once?
Comment 78 Dan Williams 2014-10-23 22:27:43 UTC
David, could you take a look at the dcbw/vpn-reconnect branch?  The VPN plugin should do these things if it supports reconnect:

1) if it notices that connectivity has dropped for any reason, it should set the service state to STARTING.  NM may change the default route away from the VPN and show enter the CONNECTING connectivity state.  We can potentially fine-tune this behavior.

2) you're basically in a normal "connect" code path; the plugin can request secrets interactively with the SecretsRequired signal (including hints) if it needs them

3) when it has successfully reconnected and has updated IP config (or even if it hasn't changed), it should re-emit that configuration to NM just like during the initial connection

4) set the "can-persist=true" option in the VPN's /etc/NetworkManager/VPN/*.name file in the [VPN Connection] section

With these changes (see the nm-openvpn 'dcbw/openvpn-restart' branch for an example) NM won't disconnect the VPN connection across link changes, and won't even disconnect it across suspend/resume.  There's also a new NMSettingVpn 'persist' boolean (that isn't hooked up yet) which will control this behavior from the connection side.  Since people with existing connections won't be expecting this behavior, we can't enable it by default for existing configurations.

Let me know how it goes!
Comment 79 Dan Williams 2014-10-28 23:19:41 UTC
Reworked a couple patches, dcbw/vpn-reconnect posted for review now.
Comment 80 Dan Williams 2014-10-28 23:21:53 UTC
*** Bug 680955 has been marked as a duplicate of this bug. ***
Comment 81 Dan Winship 2014-10-31 16:02:33 UTC
> libnm/libnm-util: add VPN 'persistent' property

I'm not convinced that we can't just enable persistence for existing connections. Sure, people won't be expecting it, but when it happens I'd expect them to be pleasantly surprised, not confused/angry...

(Ah, well, except see below on the "move VPN gateway route" commit...)

>-		if (strcmp (*iter, NM_SETTING_VPN_SERVICE_TYPE) && strcmp (*iter, NM_SETTING_VPN_USER_NAME))
>+		if (!_nm_utils_string_in_list (*iter, vpn_known_keys))

For future ease of maintenance it seems like it would be better to do "if (g_object_class_find_property (...)"

Although, adding new NMSettingVpn properties is dangerous since they could conflict with existing VPN data items... We might need to mangle the name somehow or something?

(None of the supported VPN plugins uses "persistent", so we're OK on that one at least...)


> vpn: allow plugins to re-enter the STARTING state to indicate reconnect

>-	if (priv->vpn_state == STATE_CONNECT)
>+	if (priv->vpn_state == STATE_NEED_AUTH || priv->vpn_state == STATE_CONNECT)

Why are those changes needed?


> vpn: add plugin 'can-persist' key

I would expect this information to be provided by the plugin via SetConfig...


> vpn/core: move VPN gateway route between devices when routing changes

Hm... so we're trying to deal with the "switch from wifi to ethernet" case too? I think the VPN plugin would need to be able to separately signal that it can handle persistence across IP address changes... or do we just let it try and fail if it can't?

I still don't like NMSettingVpn:persistent, but I think if we're going to have persistence-across-devices, then we'd need opt-in for at least that case...


> vpn: reconnect on service failures (bgo #349151)

"RETRY_FAILED" is easily misinterpreted ("the retry attempt failed"). RETRY_AFTER_FAIL?
Comment 82 Thomas Haller 2014-11-03 15:32:49 UTC
+     * NMSettingVpn:persistent:
+     *
+     * If the VPN service supports persistence, and this property is %TRUE,
+     * the VPN will attempt to stay connected across link changes and outages,
+     * until explicitly disconnected.
+     **/

This makes it sound as if the property  is only useful if the plugin supports it. I think in the other case it is useful too (because then nm-policy will retry instead).




Pushed two minor fixups




I agree with danw, "can-persist" should be provided at runtime by the plugin not via configuration. Is there a reason it cannot?
Comment 83 Dan Williams 2014-11-04 18:06:55 UTC
(In reply to comment #81)
> > libnm/libnm-util: add VPN 'persistent' property
> 
> I'm not convinced that we can't just enable persistence for existing
> connections. Sure, people won't be expecting it, but when it happens I'd expect
> them to be pleasantly surprised, not confused/angry...

I thought about that, and here's the downside... not all networks support the VPN connection, like the Red Hat internal network doesn't support connections to the VPN because hey that would be pointless :)

So if you are associated to your home wifi network (or Red Hat Guest) and VPN to get your mail, then when you show up at work and dock your laptop the VPN would try to activate on the internal network and ask for your password (the token code long expired) but it can't fail until it gets those credentials.  So you're stuck with a modal dialog you have to cancel whenever you dock your laptop, if you were using the VPN at home and forgot to disconnect it.  I'm not sure that's very delightful...

Which is the reason why I wanted to make it opt-in; if there are significant downsides I thought we should keep things as they are, and let people who say "oh cool, I want that feature but I understand there are some downsides" to enable it and become delighted because they understand that they opted in.

> (Ah, well, except see below on the "move VPN gateway route" commit...)
> 
> >-		if (strcmp (*iter, NM_SETTING_VPN_SERVICE_TYPE) && strcmp (*iter, NM_SETTING_VPN_USER_NAME))
> >+		if (!_nm_utils_string_in_list (*iter, vpn_known_keys))
> 
> For future ease of maintenance it seems like it would be better to do "if
> (g_object_class_find_property (...)"

Fixed up in the same commit, that's a much better solution.

> Although, adding new NMSettingVpn properties is dangerous since they could
> conflict with existing VPN data items... We might need to mangle the name
> somehow or something?
> 
> (None of the supported VPN plugins uses "persistent", so we're OK on that one
> at least...)

Yeah, I suppose we could namespace any new keys we add to the setting.  Do you want me to namespace 'persistent' or leave as-is?

> > vpn: allow plugins to re-enter the STARTING state to indicate reconnect
> 
> >-	if (priv->vpn_state == STATE_CONNECT)
> >+	if (priv->vpn_state == STATE_NEED_AUTH || priv->vpn_state == STATE_CONNECT)
> 
> Why are those changes needed?

It turns out that the "interactive" connect path didn't set STATE_CONNECT the same way the non-interactive path does, and since any plugin that supports reconnect would also support interactive mode, I ran into this.  So you're right, the real fix is to make sure plugin_new_secrets_cb() (interactive) sets STATE_CONNECT just like plugin_need_secrets_cb() does (non-interactive, through really_activate())

See "fixup! vpn: allow plugins to re-enter the STARTING state to indicate reconnect".

> > vpn: add plugin 'can-persist' key
> 
> I would expect this information to be provided by the plugin via SetConfig...

Done.  Reworked and renamed the old "add 'can-persist' option" commit to "vpn: allow plugins to indicate 'can-persist' capability".

> > vpn/core: move VPN gateway route between devices when routing changes
> 
> Hm... so we're trying to deal with the "switch from wifi to ethernet" case too?

Yes, they aren't that different really.  For example, OpenVPN servers have a capability (--float) to accept connections from new IP addresses if the connection verifies against an old connection:

"Allow  remote peer to change its IP address and/or port number, such as due to DHCP (this is the default if --remote is not used).  --float when specified with --remote allows an OpenVPN session to initially connect to a peer at a known address, however if packets arrive from  a new address and pass all authentication tests, the new address will take control of the session.  This is useful when you are connecting to a peer which holds a dynamic address such as a dial-in user or DHCP client."

But really, any VPN that can notice and handle reconnections would work here, even if it has to request new credentials.  David's suggestion above was to do the reconnect logic in the VPN plugin itself, since there are some VPNs that can do this automatically, while others (vpnc) would need a full kill-and-relaunch.

> I think the VPN plugin would need to be able to separately signal that it can
> handle persistence across IP address changes... or do we just let it try and
> fail if it can't?

Exactly.  If the plugin supports persistence it's expected to be able to notice that the connection has timed out or been severed and that it should reconnect.  I don't think there's enough reason to keep this functionality separate.

> I still don't like NMSettingVpn:persistent, but I think if we're going to have
> persistence-across-devices, then we'd need opt-in for at least that case...

I don't think this is much different than suspend/resume though, since the world may have changed on resume and you may now have Ethernet connected or you may be connected to a different WiFi AP or even a network that can't handle the VPN.  Given all that I went for opt-in...

> > vpn: reconnect on service failures (bgo #349151)
> 
> "RETRY_FAILED" is easily misinterpreted ("the retry attempt failed").
> RETRY_AFTER_FAIL?

RETRY_AFTER_FAILURE?  Done in a fixup.

Pushed and reposted for review.
Comment 84 Dan Winship 2014-11-05 17:35:31 UTC
(In reply to comment #83)
> So if you are associated to your home wifi network (or Red Hat Guest) and VPN
> to get your mail, then when you show up at work and dock your laptop

but that's the switching devices/connections case; do we need opt-in for the case where you reactivate the same connection? (Eg, close laptop, walk to conference room, open laptop. Or, flaky wifi connection occasionally drops and then reconnects.)

> Yeah, I suppose we could namespace any new keys we add to the setting.  Do you
> want me to namespace 'persistent' or leave as-is?

I didn't check the ssh-based VPN plugin because I don't have it checked out and I don't remember its name... Assuming it also doesn't use 'persistent', I'm fine with not namespacing it.


everything else looks good
Comment 85 Dan Williams 2014-11-06 15:32:47 UTC
(In reply to comment #84)
> (In reply to comment #83)
> > So if you are associated to your home wifi network (or Red Hat Guest) and VPN
> > to get your mail, then when you show up at work and dock your laptop
> 
> but that's the switching devices/connections case; do we need opt-in for the
> case where you reactivate the same connection? (Eg, close laptop, walk to
> conference room, open laptop. Or, flaky wifi connection occasionally drops and
> then reconnects.)

Yeah, we could theoretically do this without opt-in for those cases where the same connection is re-connected.  And I think we could do that for all VPNs without persistent=true and it would still make sense.  Could do that as a follow-on patch.

> > Yeah, I suppose we could namespace any new keys we add to the setting.  Do you
> > want me to namespace 'persistent' or leave as-is?
> 
> I didn't check the ssh-based VPN plugin because I don't have it checked out and
> I don't remember its name... Assuming it also doesn't use 'persistent', I'm
> fine with not namespacing it.

It doesn't use persistent:

https://github.com/danfruehauf/NetworkManager-ssh/blob/master/src/nm-ssh-service.h