GNOME Bugzilla – Bug 701242
Dispatcher not called when shutting down NetworkManager
Last modified: 2020-11-12 14:29:06 UTC
I have dispatchers registered for when a network interface is brought up and down, and while they are being called when bringing the network up, they are NOT called when bringing the network down. I was able to verify this by putting echo statements in my dispatchers and checking /var/log/messages.log: a print statement appeared when starting NetworkManager, but not when stopping it.
Stopping NetworkManager may not be a relevant test as it's not equivalent to stopping a network connection. More tests using nmcli commands would be needed to verify.
I did a few more tests and it did call the dispatcher when disabling the network connection without shutting down NetworkManager. I'm using Arch which uses systemd to manage the daemons, and the way it shuts down NetworkManager may not allow it to call the dispatchers. This is the contents of the service file: [Unit] Description=Network Manager After=syslog.target Wants=network.target Before=network.target [Service] Type=dbus BusName=org.freedesktop.NetworkManager ExecStart=/usr/sbin/NetworkManager --no-daemon # Suppress stderr to eliminate duplicated messages in syslog. NM calls openlog() # with LOG_PERROR when run in foreground. But systemd redirects stderr to # syslog by default, which results in logging each message twice. StandardError=null # NM doesn't want systemd to kill its children for it KillMode=process [Install] WantedBy=multi-user.target Alias=dbus-org.freedesktop.NetworkManager.service Also=NetworkManager-wait-online.service In my specific case, I noticed a problem because I mount and unmount NFS volumes with a dispatcher, so during shutdown the volumes never unmount and my computers that connect wirelessly end up freezing on shutdown because of this. This started happening after NetworkManager was updated, but I don't know if it's due to the NetworkManager update or if it simply was never unmounting the NFS volumes and a different update causes it to freeze in this case.
AFAIK the 'down' action is too late for any cleanup and I don't know how well 'pre-down' currently works. Also, generally, shutting down NetworkManager doesn't imply the network gets down. Otherwise you couldn't keep your network running during NetworkManager restart or from initramfs instance to the system instance. But in many cases the connection assumption doesn't work. While this will be improved with new development, some of the cases will be still uncovered. This looks like a problem in NFS as you cannot expect the network to be available from boot to shutdown and/or to warn you reliably before outage.
When I did my tests, I printed out the $1 and $2 arguments to the dispatcher script, and I didn't get any pre-down notification when I disabled my wireless connection, just down. In my specific case, you are right about it being an issue with NFS: another update to rpcbind was released, and that appears to have fixed the problem I was having. I don't know if you still consider this a valid issue for cases when the NetworkManager will bring down the network when shutting down.
(In reply to comment #4) > I don't know if you still consider this a valid issue for cases > when the NetworkManager will bring down the network when shutting down. If NetworkManager brings down an interface as part of its cleanup, it should say so in the logs. If the information doesn't get to the dispatcher because NetworkManager exits, then that's indeed a bug and should be solved. Are you still interested? Is anyone else interested in checking this?
This was working for me somewhere around Jan 25 2015, but has stopped somewhere after that. Same behavior as described in the bug report - I can see 'up' actions being executed on startup/connection, but the 'down' and 'pre-down' only appear if I either manually disconnect via the applet, or log out. Shutting down or restarting does not execute scripts in either 'down' or 'pre-down' (no such log messages in the journal. I'm using the 'pre-down' action to unmount any cifs shares I have mounted, and this not working makes shutdown hang at "unmounting <share>" for 90 seconds. It looks to me that the service is either not called on shutdown, or is killed before it can run. Currently on systemd219-1 and networkmanager 1.0.0-2, archlinux x64 with [testing] enabled. I have tried rolling back to the systemd and nm versions that were out around the date it still worked for me, but no dice.
(In reply to chrno-sphered from comment #6) > This was working for me somewhere around Jan 25 2015, but has stopped > somewhere after that. Same behavior as described in the bug report - I can > see 'up' actions being executed on startup/connection, but the 'down' and > 'pre-down' only appear if I either manually disconnect via the applet, or > log out. Shutting down or restarting does not execute scripts in either > 'down' or 'pre-down' (no such log messages in the journal. > > I'm using the 'pre-down' action to unmount any cifs shares I have mounted, > and this not working makes shutdown hang at "unmounting <share>" for 90 > seconds. > > It looks to me that the service is either not called on shutdown, or is > killed before it can run. > > Currently on systemd219-1 and networkmanager 1.0.0-2, archlinux x64 with > [testing] enabled. I have tried rolling back to the systemd and nm versions > that were out around the date it still worked for me, but no dice. When NetworkManager is stopped, it leaves the connections up. It does that on purpose, for example not do break connectivity during restart or NetworkManager. Since it doesn't down the connection, the dispatcher scripts don't run either. That seams correct to me.
pre-down was added before 1.0 and should work as intended (right?). It does not work during shutdown, for the reasons discussed above. I think this bug is fixed.
What good does it do to keep the connection up when the machine is going down? All it does is prevent the correct cleanup before turning the connection off. Also, if I stop the network manager, I actually expect it to stop the connection as well. It was working fine before. If that's a change, what's the last version that was working as I described?
(In reply to chrno-sphered from comment #9) > What good does it do to keep the connection up when the machine is going > down? NetworkManager gets killed by SIGTERM. It doesn't know that the system is going down or not. systemd doesn't kill NetworkManager any differently during shutdown. Leaving the connection up, allows non-destructive restart of NetworkManager. > All it does is prevent the correct cleanup before turning the connection off. > Also, if I stop the network manager, I actually expect it to stop the > connection as well. > It was working fine before. If that's a change, what's the last version that > was working as I described? That was the behavior <= 0.9.8.
For me, a non-destructive restart is way less useful than a clean termination. I just don't see the benefit. The disadvantage of current behavior is pretty clear to me on the other hand. I'm sure laptop users agree. Any chance of having some sort of configuration option that controls the behavior?
In my case, I was using the dispatcher hooks to connect to an NFS filesystem when the network is brought up and then disconnect from the NFS filesystem when it is brought down. When NFS isn't disconnected cleanly, it will hang indefinitely on system shutdown. IIRC on a wired connection the hang would sometimes happen, but on wifi the hang will always happen. (this is in a case where the wifi is active from startup until shutdown)
How about this: You already define a systemd-inhibit lock for sleep to safely disconnect all connections. Wouldn't also defining a lock for shutdown allow you to detect the system is going down and allow networkmanager to run the 'pre-down'/'down' actions? I also see that going to sleep does not run 'pre-down' and 'down' actions at all. This is wrong, since the connection is pretty much guaranteed to go down when the system goes to sleep.
Same issue with laptop/NFS here. I agree with chrno-sphered - a configuration option in NM to execute 'pre-down' and 'down' on going to sleep and shutdown would be very helpful.
(In reply to Aleksei from comment #14) > Same issue with laptop/NFS here. > > I agree with chrno-sphered - a configuration option in NM to execute > 'pre-down' and 'down' on going to sleep and shutdown would be very helpful. In NM 1.4 dispatcher scripts are run before suspending the system. We don't have any special handling of shutdown, though.
No one commented on my suggestion in comment #13, is it not possible?
(In reply to chrno-sphered from comment #16) > No one commented on my suggestion in comment #13, is it not possible? That sounds like a possibility. If NM exits, it currently always leaves devices up. One could argue, that dispatcher scripts about connection-down are correctly *not* invoked because also the interfaces do not get down (at least not yet and not due to NM). So, if you want to have dispatcher events about interfaces going down, it seems we should actually tear down the interfaces when NetworkManager exits during shutdown. But it's not clear to me which devices should be actually taken down. e.g. what about having the root file system from the network? Maybe instead we should continue to leave interfaces up, but emit a new dispatcher event called "shutdown", which is like "pre-down" with the difference that the interface is not about to be brought down by NM (but possibly shortly after by system power-off). For that, NM should listen to logind events. If login notifies about system-shutdown, we could set a flag, and when NM exits with the flag set, we would emit these "shutdown" events.
bugzilla.gnome.org is being shut down in favor of a GitLab instance. We are closing all old bug reports and feature requests in GNOME Bugzilla which have not seen updates for a long time. If you still use NetworkManager and if you still see this bug / want this feature in a recent and supported version of NetworkManager, then please feel free to report it at https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/ Thank you for creating this report and we are sorry it could not be implemented (workforce and time is unfortunately limited).