After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 751017 - NetworkManager service doesn't stop reliably during shutdown
NetworkManager service doesn't stop reliably during shutdown
Status: RESOLVED FIXED
Product: NetworkManager
Classification: Platform
Component: general
1.0.x
Other Linux
: Normal normal
: ---
Assigned To: NetworkManager maintainer(s)
NetworkManager maintainer(s)
Depends on:
Blocks:
 
 
Reported: 2015-06-15 20:23 UTC by nucrap
Modified: 2015-07-01 15:20 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
openrc shutdown log (996 bytes, text/plain)
2015-06-27 16:31 UTC, nucrap
  Details
shutdown log including nm debug messages (72.21 KB, text/plain)
2015-06-27 16:32 UTC, nucrap
  Details
lspci -v (22.18 KB, text/plain)
2015-06-27 16:34 UTC, nucrap
  Details
Logging message change (497 bytes, patch)
2015-06-29 20:14 UTC, Jacob
none Details | Review
dispatcher: don't exit with failure in case D-Bus service stopps (1.45 KB, patch)
2015-06-30 10:12 UTC, Thomas Haller
none Details | Review

Description nucrap 2015-06-15 20:23:36 UTC
On my system I get the following warning on shutdown:


* Stopping NetworkManager ... [ ok ]
* Stopping D-BUS system message bus ... [ ok ]
nm-dispatcher: Could not get the system bus. Make sure the message bus is running!


As you see the network manager service still seems to be active after having been stopped. Btw I see this warning only when I shutdown while being connected to wlan.
Comment 1 Thomas Haller 2015-06-16 14:17:16 UTC
Is there not more information in the log-files? Can you reproduce this reliable, or often? Which distribution?
Comment 2 nucrap 2015-06-16 15:21:45 UTC
> Is there not more information in the log-files?

Unfortunately this message does not even appear in the shutdown logs, so I had to type it. I have no idea why it's not in rc.log.


> Can you reproduce this reliable, or often?

I see this warning on every shutdown, except if I disconnect from my wifi network previously. So yes, I can reproduce this reliably.


> Which distribution?

It's funtoo (a gentoo flavor).
Comment 3 Thomas Haller 2015-06-16 15:59:53 UTC
(In reply to nucrap from comment #2)
 
> > Which distribution?
> 
> It's funtoo (a gentoo flavor).

is this without systemd?
Comment 4 nucrap 2015-06-16 16:01:30 UTC
it's openrc
Comment 5 Thomas Haller 2015-06-17 10:15:49 UTC
starting/stoping services is the job of the init-system.

Upstream provides .service files for systemd, which are supposed to work.

There is also an upstream directory 'initscript' [1] but it is badly maintained. Very likely your distribution does not make use of these scripts directly.


Basically, it's a downstream bug because your distribution has to get the init-system right for you. Please report it to funtoo/gentoo.


I'm closing this bug as NOTGNOME for now, but feel welcome to reopen or add further comments. 



[1] http://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/initscript
Comment 6 nucrap 2015-06-27 16:31:49 UTC
Created attachment 306201 [details]
openrc shutdown log
Comment 7 nucrap 2015-06-27 16:32:55 UTC
Created attachment 306203 [details]
shutdown log including nm debug messages
Comment 8 nucrap 2015-06-27 16:34:19 UTC
Created attachment 306204 [details]
lspci -v
Comment 9 nucrap 2015-06-27 16:43:16 UTC
I have reported this issue downstream (https://bugs.funtoo.org/browse/FL-2541) and was told to reopen this bugreport :P

My wifi connection is reliable using 2Ghz. I was on kernel 4.0 and now I'm at 4.1.

The downstream funtoo dev was able to reproduce this on his wired connection too, see https://bugs.funtoo.org/browse/FL-2541

I have added some more informative log files for your convenience.


Hope we can figure this out :)
Comment 10 Thomas Haller 2015-06-29 13:21:14 UTC
The log from comment 7 shows that NetworkManager does *not* invoke dispatcher *after* starting to shutdown.
Shutdown starts with the line:
  "caught signal 15, shutting down normally."

I don't see how the commentator on the upstream bug "notice[s] that NM hangs on nm-dispatcher". The attached log doesn't show that -- but it shows the warning of nm-dispatcher.

nm-dispatcher was started before shutting down NetworkManager, but it was not terminated before stopping D-Bus. That is not NetworkManager's fault but the fault of the init-system.
Comment 11 Oleg Vinichenko 2015-06-29 14:53:28 UTC
A clarification of nm-dispatcher hang would be an attempt to reproduce nucrap problem, the one reported in https://bugs.funtoo.org/browse/FL-2541. Indeed, after sending shutdown to system and plugging off ethernet cable before nm and dbus stopped makes a warning described by reporter. Personally, I getting no any problems otherwise. A warning message and hangs last about 15-20 seconds and system halts anyway.
I'd like to get more clarification about a nm-dispatcher and who and when starts it, and stops it. Funtoo and Gentoo does provide an init script only for NetworkManager only and not any sub-child processes NetworkManager starts and stops on it's own.
Comment 12 Thomas Haller 2015-06-29 15:45:13 UTC
(In reply to Oleg Vinichenko from comment #11)
> A clarification of nm-dispatcher hang would be an attempt to reproduce
> nucrap problem, the one reported in https://bugs.funtoo.org/browse/FL-2541.
> Indeed, after sending shutdown to system and plugging off ethernet cable
> before nm and dbus stopped makes a warning described by reporter.
> Personally, I getting no any problems otherwise. A warning message and hangs
> last about 15-20 seconds and system halts anyway.

The logfile from comment 7 doesn't show any hang. But it *does* show the warning from the original bug-report. So I would assume that the hang is either a different issue, or it is not necessary to reproduce the issue.

> I'd like to get more clarification about a nm-dispatcher and who and when
> starts it, and stops it. Funtoo and Gentoo does provide an init script only
> for NetworkManager only and not any sub-child processes NetworkManager
> starts and stops on it's own.

nm-dispatcher is not started by NetworkManager directly but by D-Bus activation.
I'm not familiar with how to configure with non-systemd systems. But as far as NetworkManager (core daemon) is concerned, it is just another D-Bus service.


Yes, the warning is ugly and it hints to a misconfiguration. But otherwise the issue is harmless (a hang on showdown might be a real issue).
Comment 13 Jacob 2015-06-29 19:37:29 UTC
I did some poking around to find out about how D-Bus services get deactivated, and I found that they don't. It sounds like implementations should be smart about when they close based on their own use cases.

It seems like if D-Bus goes down, shouldn't the nm-dispatcher expect that it's perhaps due to a system shutdown and just close with it?

(My impression is systemd cares about dbus services, so I imagine systemd would go through and terminate dbus services before its own implementation of dbus goes down.)
Comment 14 Jacob 2015-06-29 20:14:44 UTC
Created attachment 306327 [details] [review]
Logging message change

This would be a good adjustment to make the error seem less scary when expected (like when the system is shutting down). I'm investigating why the error is appearing in openrc stderr instead of going to the logger, since nm-dispatcher appears to be logging to the right place.
Comment 15 Thomas Haller 2015-06-30 10:12:48 UTC
Created attachment 306382 [details] [review]
dispatcher: don't exit with failure in case D-Bus service stopps

The initsystem should stop nm-dispatcher before stopping D-Bus
service. However, on some systems that is not implemented, so
nm-dispatcher should not exit with a failure message when the
system bus disappears.
Instead it should just assume that D-Bus service was stopped and
it should exit gracefully.

Based-on-patch-by: Jacob <jacobgodserv@gmail.com>
Comment 16 Thomas Haller 2015-06-30 10:29:16 UTC
(In reply to Jacob from comment #14)
> Created attachment 306327 [details] [review] [review]
> Logging message change

@Jacob: are you ok with the "Based-on-patch-by:"? Do you want me to change it?
Comment 17 Jacob 2015-07-01 12:54:08 UTC
Sorry for the delayed response. That looks great to me!
Comment 18 Thomas Haller 2015-07-01 15:20:35 UTC
(In reply to Thomas Haller from comment #15)
> Created attachment 306382 [details] [review] [review]
> dispatcher: don't exit with failure in case D-Bus service stopps
> 

patch merged upstream:

master: http://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=8fdf19863290a82e4000c1549cbc114210f545f1

nm-1-0: http://cgit.freedesktop.org/NetworkManager/NetworkManager/commit/?id=03491caf8351228860fc29d916f9e5c280de27f7




Closing this bug now.


For a feel-good-sensation, the initsystem should stop nm-dispatcher before stopping D-Bus daemon.
However if it doesn't, it's not a real problem. The warning was now downgraded to a info message, and the process will termiante with exit status 0 (indicating normal termination).