After an evaluation, GNOME has moved from Bugzilla to GitLab. Learn more about GitLab.
No new issues can be reported in GNOME Bugzilla anymore.
To report an issue in a GNOME project, go to GNOME GitLab.
Do not go to GNOME Gitlab for: Bluefish, Doxygen, GnuCash, GStreamer, java-gnome, LDTP, NetworkManager, Tomboy.
Bug 620932 - Add simple network online/offline API
Add simple network online/offline API
Status: RESOLVED FIXED
Product: glib
Classification: Platform
Component: network
2.25.x
Other Linux
: Normal normal
: ---
Assigned To: gtkdev
gtkdev
Depends on:
Blocks: 646290
 
 
Reported: 2010-06-08 07:57 UTC by Ross Burton
Modified: 2011-11-14 19:03 UTC
See Also:
GNOME target: ---
GNOME version: ---


Attachments
gsettingsbackend: remove useless check (997 bytes, patch)
2011-09-28 21:14 UTC, Dan Winship
committed Details | Review
Add _g_io_module_get_default(), use to simplify other *_get_default()s (11.66 KB, patch)
2011-09-28 21:14 UTC, Dan Winship
committed Details | Review
add GNetworkMonitor, for... monitoring the network (17.58 KB, patch)
2011-09-28 21:14 UTC, Dan Winship
none Details | Review
nm: implement a simple NetworkManager-based GNetworkMonitor (11.87 KB, patch)
2011-09-28 21:15 UTC, Dan Winship
none Details | Review
GInetAddressMask: new type for internet address range matching (17.94 KB, patch)
2011-10-04 17:49 UTC, Dan Winship
none Details | Review
add GNetworkMonitor, for... monitoring the network (61.98 KB, patch)
2011-10-04 17:55 UTC, Dan Winship
none Details | Review
GInetAddressMask: new type for internet address range matching (19.35 KB, patch)
2011-11-07 20:27 UTC, Dan Winship
committed Details | Review
add GNetworkMonitor, for... monitoring the network (65.98 KB, patch)
2011-11-07 20:29 UTC, Dan Winship
committed Details | Review

Description Ross Burton 2010-06-08 07:57:05 UTC
Networking subsystems are sadly not uniform on all supported systems, there being several systems in use:

* Network Manager
* ConnMan
* Win32
* Conic (Maemo)
* Solaris (can't recall the name of this)
* There are probably more

In MeeGo we've replicated the same code to abstract these to a single online/offline boolean too many times.

The super-simple API would be something like:

GNetworkConnectivity * g_network_connectivity_new ();

With a "connected" gboolean property that apps can listen to.  It probably makes sense for this to be implemented with extension points.

For applications that need more information than "is there a connection" (such as GUPnP, that needs to know when new interfaces are bought up and down) there could be a slightly more advanced API.  I've only had three hours sleep so I'll come back to this shortly. :)
Comment 1 Tommi Komulainen 2010-06-08 08:18:00 UTC
You'll probably want to define "connected" more clearly since it can mean a few different things, some more useful than others in different contexts:
1) have an IP address (so what if it's 169.something and you can't reach anyone beyond your router)
2) can reach local network (but access to, say gnome.org times out or is dropped)
3) can almost reach the internet (except that access to gnome.org gives you a login form instead)
4) can reach the internet (gnome.org gives the actual gnome home page)
Comment 2 Dan Winship 2010-06-08 09:26:22 UTC
(In reply to comment #1)
> 2) can reach local network (but access to, say gnome.org times out or is
> dropped)

We don't really want to use "gnome.org" though, because that machine is unreachable sometimes, when there are problems with the colo, or the hardware is being upgraded, etc. We'd want to use some multi-homed host, call it "ping.gnome.org" or something, with a few different IP addresses around the world. (Perhaps hosted by the sites doing gnome.org's DNS, or else have each advisory board member host an instance, or something.)

> 3) can almost reach the internet (except that access to gnome.org gives you a
> login form instead)
> 4) can reach the internet (gnome.org gives the actual gnome home page)

For simplicity, the test resource should be non-html. Then if you get back a text/html response, you automatically know it's a login form (or a click-to-accept-these-terms-of-service form).
Comment 3 Ross Burton 2010-06-08 10:11:20 UTC
Interestingly ConnMan doens't have this problem.  When you have a network connection of some sort (the interface is up) the state is "connected".  It then proceeds to fetch a page from a controlled server and examines it to determine if you need to login and so on.   When you have a real connection, the state is "online".

The question is should GIO do this or should this be done by subsystems if they wish?  I'd actually say that the subsystems should do it, they bought the interface up so at that point they should verify the connection.
Comment 4 Richard Hughes 2010-06-08 11:42:14 UTC
PackageKit has connman/NetworkManager/unix networking code to try and detect in an easy way if the user has a wired, wireless, mobile or no connection. It's not much good knowing just if I'm online, I want to know how much the update is going to cost me in real money.

Needless to say, if the new API provides more than just "online" or "offline" I would switch to it in a heartbeat and remove a few tens of thousands of lines of code in PK.
Comment 5 Dan Winship 2010-11-10 17:54:04 UTC
(In reply to comment #3)
> The question is should GIO do this or should this be done by subsystems if they
> wish?  I'd actually say that the subsystems should do it, they bought the
> interface up so at that point they should verify the connection.

regardless of who figures it out, we still probably want to export the "stuck behind a for-pay wifi hotspot" vs "actually connected to the internet" distinction in the API.

dcbw: NM doesn't currently try to make that distinction. Do you consider it to be within NM's domain, or that a problem for higher levels?

(In reply to comment #4)
> PackageKit has connman/NetworkManager/unix networking code to try and detect in
> an easy way if the user has a wired, wireless, mobile or no connection. It's
> not much good knowing just if I'm online, I want to know how much the update is
> going to cost me in real money.

Though some users have unlimited mobile data plans, and others have wifi hubs that take 3G dongles, so you can't really reliably know if the network is free or metered.

Related, it is sometimes useful to be able to characterize the network as either "fast" or "slow", but again all you can reliably say (without doing bandwidth tests) is whether the local link is fast or slow.
Comment 6 Colin Walters 2011-05-16 20:40:33 UTC
(In reply to comment #5)
> (In reply to comment #3)
> > The question is should GIO do this or should this be done by subsystems if they
> > wish?  I'd actually say that the subsystems should do it, they bought the
> > interface up so at that point they should verify the connection.
> 
> regardless of who figures it out, we still probably want to export the "stuck
> behind a for-pay wifi hotspot" vs "actually connected to the internet"
> distinction in the API.

I can see that being useful, but there's no reason not to implement the basic interface which we simply define as saying 

* There is a default route backed by an interface that is up

That'd be very useful for both apps and OS components.

Whether the nameservers are reachable, given hostnames can be accessed etc. is obviously a big mess, but it'd be better to have working apps for the ethernet plug/unplug case at home now, and not stall waiting for a mess of heuristics for "am I behind a paywall".
Comment 7 Ross Burton 2011-05-16 20:46:46 UTC
I've been thinking about this again recently so may hack up something involving gio/glib-networking/extension points soon with a simple "am I on a network' boolean method.
Comment 8 David Zeuthen (not reading bugmail) 2011-05-17 13:52:44 UTC
(In reply to comment #7)
> I've been thinking about this again recently so may hack up something involving
> gio/glib-networking/extension points soon with a simple

This would be great. It turns out I need something like this for my gnome-shell work for 3.2 so when I asked on #gnome-shell whether the shell had an abstraction it was suggested to do this in Gio as a singleton and we dug up this bug.

> "am I on a network'
> boolean method.

I don't think we need that much information actually.. and it's a very hard question to answer because people interpret that in very different ways (paywalls, vpns etc.).

Specifically what I need is just a signal that, basically, says "hey, something related to networking changed - try connecting, reconnecting or checking if your connection is still up" which would cover events such as

 - network connection established / torn down
 - paywall cleared
 - vpn connection established / torn down

The docs should emphasize that "checking if your connection is still up" includes using an _application-level_ ping - for example, for IMAP it includes sending the NOOP command and waiting for a response.

We should mention how often we expect this signal to fire - for example, we could say that applications need to cope with the signal firing N times right after each other (allowing implementations to be simpler). Or we could say that it's rate-limited and will not fire more than once a second (allowing users to be simpler). I think we should go for the latter.

Since this is a singleton, we should guarantee that signals are always emitted in the default main context

I think just starting out with a GNetworkManager class with this ::changed signal would be a good start. We can always add other stuff later - but I don't think apps will need anything else that the ::changed signal (unless the app is a network indicator, of course - but in that case it should use the native NM or ConnMan APIs).
Comment 9 David Zeuthen (not reading bugmail) 2011-05-17 13:55:20 UTC
(In reply to comment #8)
> I think just starting out with a GNetworkManager class with this ::changed
                                   ^^^^^^^^^^^^^^^

Eh, I obviosuly meant GNetworkMonitor - in the same spirit as the recently added GTimezoneMonitor class :-)
Comment 10 Dan Winship 2011-05-17 14:39:57 UTC
(In reply to comment #8)
> > "am I on a network'
> > boolean method.
> 
> I don't think we need that much information actually.. and it's a very hard
> question to answer because people interpret that in very different ways
> (paywalls, vpns etc.).

But most of the time when you're offline, you're *obviously* offline (ie, no network interface other than loopback is up). And in that case, apps should not try to connect and then eventually time out, they should just say "sorry, offline".

But yes, going on/off VPN is an important thing to signal as well.

So I'd say have a gboolean network-up property, which is TRUE if you have a default route, and FALSE otherwise, and note explicitly that notify::network-up may be emitted even when the value didn't actually change, if something interesting about the network configuration has changed.

Alternatively/additionally, we could have "network-upgraded" and "network-downgraded" signals, where the former means "you might be able to connect to hosts that were unreachable before" and the latter means "you might have just lost connections to hosts you were connected to".)
Comment 11 David Zeuthen (not reading bugmail) 2011-05-17 14:59:43 UTC
(In reply to comment #10)
> (In reply to comment #8)
> > > "am I on a network'
> > > boolean method.
> > 
> > I don't think we need that much information actually.. and it's a very hard
> > question to answer because people interpret that in very different ways
> > (paywalls, vpns etc.).
> 
> But most of the time when you're offline, you're *obviously* offline (ie, no
> network interface other than loopback is up). And in that case, apps should not
> try to connect and then eventually time out, they should just say "sorry,
> offline".

Yeah, t's a _nice_ optimization to tell the app that "hey, we know for a fact that networking is down so don't even try" (== value of :network-up property).. so.. sure... and it's nice the app can tell the user "sorry, offline", I didn't think of that obvious requirement.

Anyway, FWIW, I don't think the cost of checking/reconnecting is very high.. and since we don't plan on emitting the signal too often it shouldn't matter at all. And well-written apps will serialize the checks _anyway_ so they shouldn't care too much about a lot multiple signal emissions.


> But yes, going on/off VPN is an important thing to signal as well.
> 
> So I'd say have a gboolean network-up property, which is TRUE if you have a
> default route, and FALSE otherwise, and note explicitly that notify::network-up
> may be emitted even when the value didn't actually change, if something
> interesting about the network configuration has changed.

Sure, as long as ::notify::network-up emission is basically what my ::changed signal in comment 8 is about, I'm fine with that - that's what you're suggesting, right?

> Alternatively/additionally, we could have "network-upgraded" and
> "network-downgraded" signals, where the former means "you might be able to
> connect to hosts that were unreachable before" and the latter means "you might
> have just lost connections to hosts you were connected to".)

Going on a VPN without split-DNS might actually mean both - you can now reach corporate servers but since you are using the new DNS (or even proxy), you are now unable to resolve certain host-names (or even access certain IP ranges because of dubious filtering...) that you were able to resolve before... my point is that words like "upgraded" and "downgraded" are kinda dangerous to use because they might not mean the same to everyone.... so I'd rather keep it simple and not include this.
Comment 12 Dan Williams 2011-05-17 15:34:23 UTC
I think simple is the way to go here to start with; extend it later on if people scream loud enough, but in the end you *really* don't want to ever get it too complicated, since that's simply not what glib is supposed to be doing.  The smarts get left to the connection managers, and glib might want to eventually grow "plugins" for those connection managers, but internally most of it's logic should be dead simple.

One thing I'd suggest though; some way of pushing additional information about what network connection was just started/stopped if there's a connection manager running that can handle that sort of thing.  At least NM and connman keep information about the defined  network connections and can pass this along; for NM it's the connection UUID and ID tags.  That way clients could, if they chose, do smart things like checking your work mail only when your work VPN starts and take the account offline when the VPN stops.  Those could be additional signals though, separate from ::network-up.

The generic implementation would talk netlink on Linux, and would call the network "up" only if the following three conditions were met:

1) at least one interface has the flag IFF_UP
2) at least one interface with IFF_UP has an IP address
3) the routing table has a default route (and maybe check that the default route can actually pass traffic from the interface found in (1) and (2), but that's probably not necessary)

False positive here (ie, thinking network is up when it's really not) is much better than false negative (ie, thinking network is down when it's really up) from a user's perspective, *except* in the case where you have an IM client running that tries to log in with username/password when the connection is not secure, like in a coffee shop.  But those people are stupid and should be using SSL anyway.
Comment 13 Dan Williams 2011-05-17 15:41:49 UTC
There's also a few more problems:

1) some people just use their machines on a private network, and would want networking considered "up" when on a local subnet without necessarily having a default route; but without a default route you can't have access to the Internet for IPv4 at least

2) IPv4 vs. IPv6; what if one has connectivity but the other doesn't?  Do you consider the network "up" if IPv6 is connected but IPv4 is not?  Ideally yes; but too many programs don't handle IPv6 yet.  That might be fine though, since these stupid programs will just try to connect and fail.  One thing you might think about is having ::network4-up and ::network6-up as separate signals that apps could listen to.

3) IPv6 has 3 classes of address, link-local, site-local, and global.  So if some interface has only a site-local IPv6 address, do you consider IPv6 networking "up" or not?  Do we consider IPv4 link-local and IPv6 link-local connectivity to be "network-up" as well?
Comment 14 Ross Burton 2011-05-17 16:12:22 UTC
I'd say you're "down" if there is only the loopback interface (or nothing), anything else that *could* be routable is "up".  As you say, false positive is better than false negative and we don't want to be asserting that networking is down just because we haven't anticipated a weird networking setup or site-local services.

(looking at dwmw2 who does all sorts of "interesting" routing and forwarding tricks on his internal network)
Comment 15 Dan Winship 2011-07-27 17:58:11 UTC
(In reply to comment #13)
> 1) some people just use their machines on a private network...
> 2) IPv4 vs. IPv6; what if one has connectivity but the other doesn't?...
> 3) IPv6 has 3 classes of address, link-local, site-local, and global...

Had a thought last night:

    gboolean g_network_monitor_can_reach (GNetworkMonitor     *monitor,
                                          GSocketConnectable  *connectable,
                                          GCancellable        *cancellable,
                                          GError             **error);

(and corresponding async version). This would attempt to determine whether @connectable refers to a host that is currently reachable, without actually trying to connect to it and waiting for a timeout.

The dumb implementation would just return TRUE when the network was "up" and FALSE when it was "down".

A smarter implementation could also check things like mDNS-resolvable hostnames, link/site-local IP addresses, IP addresses corresponding to locally-running virtual machines, etc.

So then, the app can use this API to decide whether to mark a given service online or offline.
Comment 16 Ross Burton 2011-08-01 12:36:14 UTC
Hm, interesting.  I think I like it.  It would complicate programs like libsocialweb which currently has a global online/offline toggle, so the "am I online?" logic would have to fall into the individual service plugins as only they know what hosts they'll be hitting.

However isn't there almost always a default route, so any form of "can I reach..." won't work because it would need to try the default route which will only fail when you try. Also if your router is up but your ADSL line is down again we'll have false positives.

As IPv6 becomes more integral I guess this API is more relevant though.
Comment 17 Colin Walters 2011-08-01 13:06:07 UTC
(In reply to comment #16)> 
> However isn't there almost always a default route,

Not if you haven't gotten a successful DHCP return for example.

You are right that this wouldn't help most of post-DHCP cases like Web clickthrough/paywall.

Hm.  So with danw's suggestion I think we clearly still need some event based API that tells you *when* you should even try calling g_network_monitor_can_reach().

The event state would be pretty simple:

OFFLINE    - there is no possibility g_network_monitor_can_reach() would succeed
ONLINE     - g_network_monitor_can_reach() may succeed
INTERNET_ONLINE   - We checked that an HTTP request on the public internet worked
FILTERED          - We are ONLINE, but our public request failed for some reason

So we'd go ONLINE if we have a default route for IPv4 basically.  And we'd prefer most apps wait for INTERNET_ONLINE or FILTERED.

dcbw would obviously have a lot more knowledge about this.
Comment 18 Ross Burton 2011-08-02 11:30:30 UTC
I agree that this is far more useful.

FWIW, ConnMan uses the "ready" for "we have a networking connection", "login" for "we know we're behind a captive portal" and "online" for "we've verified that we're on the public internet".

Are there plans for NM to support captive portal detection?  Integrating that into the shell would be most excellent with the Networking applet popping up a browser widget to login, so that your browser session isn't trashed with the redirects.
Comment 19 Colin Walters 2011-09-15 13:41:03 UTC
Also, we should have an API where applications can request the operating system show the network settings.  Maybe this one should live in GTK+ though.
Comment 20 Bastien Nocera 2011-09-18 12:00:01 UTC
(In reply to comment #18)
> I agree that this is far more useful.
> 
> FWIW, ConnMan uses the "ready" for "we have a networking connection", "login"
> for "we know we're behind a captive portal" and "online" for "we've verified
> that we're on the public internet".
> 
> Are there plans for NM to support captive portal detection?  Integrating that
> into the shell would be most excellent with the Networking applet popping up a
> browser widget to login, so that your browser session isn't trashed with the
> redirects.

NM has no support for captive portals, but there's holes in the status enumeration to handle those cases. IMO, the enum should be replicated and the GNetworkMonitor simply export the value of that status.

The enum, with holes in it for expansion:
http://cgit.freedesktop.org/NetworkManager/NetworkManager/tree/include/NetworkManager.h#n78

Anything more complicated should talk directly to the provider. In terms of implementation, a gio extension point sounds like a way to implement it, if the various backends go to unwieldly.
Comment 21 Dan Winship 2011-09-28 21:14:52 UTC
Created attachment 197712 [details] [review]
gsettingsbackend: remove useless check

GMemorySettingsBackend is always present, so there's no need to check
for "no backends available"
Comment 22 Dan Winship 2011-09-28 21:14:57 UTC
Created attachment 197713 [details] [review]
Add _g_io_module_get_default(), use to simplify other *_get_default()s

Add _g_io_module_get_default(), which implements the
figure-out-the-best-available-module-that-is-actually-usable logic,
and use that to simplify g_proxy_resolver_get_default(),
g_settings_backend_get_default(), g_tls_backend_get_default(), and
g_vfs_get_default().
Comment 23 Dan Winship 2011-09-28 21:14:59 UTC
Created attachment 197714 [details] [review]
add GNetworkMonitor, for... monitoring the network

Add GNetworkMonitor and its associated extension point, and implement
a dummy default subclass that always claims the network is available.
Comment 24 Dan Winship 2011-09-28 21:15:09 UTC
Created attachment 197715 [details] [review]
nm: implement a simple NetworkManager-based GNetworkMonitor
Comment 25 Dan Winship 2011-09-28 21:18:27 UTC
Comment on attachment 197715 [details] [review]
nm: implement a simple NetworkManager-based GNetworkMonitor

>+/* Doesn't feel worth depending on nm-glib just for this... */
>+#define NM_STATE_CONNECTED 70

oops, forgot to fix that part :)

anyway, this doesn't implement the "can_reach" stuff I talked about before, NOR does it allow distinguishing "really connected" and "stuck behind a portal". But it's a start.
Comment 26 Dan Williams 2011-09-28 23:43:59 UTC
Patch looks fine.  There are plans for NM to do captive portal detection (see TODO in git), and what will happen there is that NM will enter the CONNECTED_LOCAL state, perform any portal/connectivity detection, and only enter the CONNECTED_GLOBAL state when it's verified that there is a connect to the actual internet.
Comment 27 Ross Burton 2011-09-29 10:22:35 UTC
Excellent, I'd been meaning to work on this but other more important stuff always crops up first.

Hopefully I can knock up a connman implementation shortly.  The sooner I can deprecate SwOnline the better.
Comment 28 Colin Walters 2011-09-29 15:34:58 UTC
I think extension points for this are lame because they don't actually support e.g. having both NetworkManager and connman mashed into the same OS at runtime - one has to win.

And if one has to win, we might as well just have a --enable-network-backend=NetworkManager configure flag.  We could in theory have --enable-network-backends=NetworkManager,connman and at runtime detect between them if someone cared.
Comment 29 Colin Walters 2011-09-29 15:40:02 UTC
One scenario people have complained about in the past is developers or testers who are running a local HTTP server on their laptop (bound to 127.0.0.1), but Firefox says they're offline.

We need to decide where the solution for this lives in the stack.  Options:

1) Applications should special case the loopback interface (strcmp() for 127.0.0.1/localhost)
2) A special tool for NetworkManager to say "I'm online".  dcbw has opposed this in the past, but anyways the basic idea here is that state would propagate to GNetworkMonitor and apps would think they're online.

I think the right approach is 2) personally, but open to ideas.
Comment 30 Colin Walters 2011-09-29 15:43:29 UTC
I am most interested in this patch series when we start trying to implement g_network_monitor_can_reach() by the way.
Comment 31 Bastien Nocera 2011-09-29 15:59:16 UTC
(In reply to comment #29)
> One scenario people have complained about in the past is developers or testers
> who are running a local HTTP server on their laptop (bound to 127.0.0.1), but
> Firefox says they're offline.

It's pretty clear that it's the app's job to figure out whether they're using the local loopback.
Comment 32 Colin Walters 2011-09-29 16:07:27 UTC
(In reply to comment #31)
> (In reply to comment #29)
> > One scenario people have complained about in the past is developers or testers
> > who are running a local HTTP server on their laptop (bound to 127.0.0.1), but
> > Firefox says they're offline.
> 
> It's pretty clear that it's the app's job to figure out whether they're using
> the local loopback.

Or here's actually a related (possibly even more likely) scenario, from comment https://bugzilla.gnome.org/show_bug.cgi?id=620932#c15

Locally-running VMs.  So I guess what I'm getting at is the relation of danw's "network-available" property and specifically "no remote hosts at all are reachable".

If you're an app like Firefox that can take in arbitrary URLs or IP addresses, I guess you can strcmp() for 127.0.0.1 (and the IPv6 one), but how are you to detect bridged VMs?

So basically if I have a bridged VM set up, would the "network-available" boolean always be on?  

And I do think asking apps to strcmp() for 127.0.0.1 etc. is pretty horrible - we should try hard to do that for them.

Hm, so danw does say that apps should basically monitor the property and call can_reach() again.  But it seems like we're heading to the property always evaluating to TRUE, in which case we should toss it and just have a signal "changed".
Comment 33 Dan Winship 2011-09-29 16:11:22 UTC
(In reply to comment #28)
> I think extension points for this are lame because they don't actually support
> e.g. having both NetworkManager and connman mashed into the same OS at runtime
> - one has to win.

That's not really any different from many of the other extension points. Eg, you can build both the inotify and fam GFileMonitor backends, but only one will get used at runtime.

And by using an extension point, people can make their own external implementations too, which is useful given the big list in comment 0.
Comment 34 Colin Walters 2011-09-29 16:30:00 UTC
(In reply to comment #33)
> (In reply to comment #28)
> > I think extension points for this are lame because they don't actually support
> > e.g. having both NetworkManager and connman mashed into the same OS at runtime
> > - one has to win.
> 
> That's not really any different from many of the other extension points. Eg,
> you can build both the inotify and fam GFileMonitor backends, but only one will
> get used at runtime.

Yes, I am actually arguing against most extension points in general.  I think they should be used in very very few cases.  dconf and gvfs are good uses because they're just avoiding dependency cycles basically.

And yeah, I think the inotify/fam thing boils down to a really complicated way to call inotify_init1() and fall back to FAM, when we could just as easily hard code it.

> And by using an extension point, people can make their own external
> implementations too, which is useful given the big list in comment 0.

If by "big" you mean 5 =)  Of which let's assume the Maemo one is dead, so that's 4.  And do we really want the win32 and solaris implementations to be external?

Now, one rationale to make them extension points is for *api-unstable* things.  In this case, the extension point gets shipped with the component (e.g. NetworkManager).  Is that your ultimate plan?  If so that makes sense generally to me, and I take back my arguments against an extension point for this.

The file monitoring thing I think is a historic mistake and not an example to emulate.
Comment 35 Dan Winship 2011-09-29 16:58:19 UTC
(In reply to comment #34)
> Yes, I am actually arguing against most extension points in general.  I think
> they should be used in very very few cases.  dconf and gvfs are good uses
> because they're just avoiding dependency cycles basically.

Which we are (or at least will be) also doing here; there's no point in using D-Bus calls to NM to figure out what networks are available when it's MUCH MUCH easier to do using libnm-glib.
Comment 36 Dan Winship 2011-09-29 23:13:48 UTC
actually...

Almost all of the functionality can be done network-managing-framework-agnostically by using libnl to get updates directly from the kernel; we'd watch for changes to the routing table, and update our ideas of what hosts were reachable based on that. We don't need to know if it's NetworkManager or ConnMan or whatever who is making the changes...

(This does not make it any less portable-to-non-linux than before, because NetworkManager already depends heavily on libnl anyway.)

("Almost" all of the functionality, because this wouldn't distinguish actually-connected from stuck-behind-a-captive-portal.)
Comment 37 Colin Walters 2011-09-29 23:35:02 UTC
Hmm, is libnl something that we want all apps to be linking to?  I guess looking at the kernel code it's designed for this sort of thing, it's got multicast.  So that'd be fine.

No strong objections here, but note that libnl just recently underwent an incompatible API/ABI bump (so they can parallel install apparently).  The only version in Fedora is 1.1, I guess that's what NetworkManager uses.

Debian has 1.1 and some git snapshot from 2009 in experimental.

So...maybe figuring out wtf is going on with that library would be good to do.  Possibly dcbw knows?
Comment 38 Matthias Clasen 2011-09-30 01:55:23 UTC
Review of attachment 197712 [details] [review]:

Looks fine to me
Comment 39 Matthias Clasen 2011-09-30 01:58:21 UTC
Review of attachment 197713 [details] [review]:

Makes a lot of sense to me.
Comment 40 Matthias Clasen 2011-09-30 02:14:48 UTC
As far as general philosophy is concerned, I agree with Colin that extension
points are more often than not a bad idea.

I disagree with the idea that its ok to have a dbus api that is 'much much
harder' to use than the C bindings.

Getting info directly from the kernel sounds appealing if a) we can get all the
info we need and b) the libnl stability/availability gets investigated
Comment 41 Michael Meeks 2011-09-30 08:59:46 UTC
> Hmm, is libnl something that we want all apps to be linking to? 

I'd say no (personally), and of course given our relatively simply netlink use-case, it should be reasonably easy to implement / import & hide the small bits of libnl we really want.
Comment 42 Lennart Poettering 2011-09-30 13:55:34 UTC
I don't think you really should be using libnl if all you want to know is whether there's a default route and whether the iface that the default route is on is UP and RUNNING. Avahi does not use libnl and does that, and even much more complex stuff.

a) socket(PF_NETLINK, SOCK_DGRAM, NETLINK_ROUTE)
b) bind(fd, { .nl_family = AF_NETLINK, .nl_groups = RTMGRP_LINK|RTMGRP_IPV4_ROUTE|RTMGRP_IPV6_ROUTE, .nl_pid = 0 });
c) setsockopt(fd, SOL_SOCKET, SO_PASSCRED, &1, 1);
d) send(fd, ... { .nlmsg_flags = NLM_F_ACK|NLM_F_REQUEST|NLM_F_DUMP }, ...);
e) for (;;) { recvmsg(fd, data...); if (SCM_CREDENTIALS.uid != 0) continue; process(data...); }

i.e. e) is to be integrated in the main loop.

and process() then simply has to iterate through the message payload and look for RTM_NEWLINK, RTM_DELLINK, RTM_NEWROUTE, RTM_DELROUTE and count default routes and match up the ifaces. Initially it should synchronously wait until NLMSG_DONE is read which the kernel sends after the app received the full dump of iface configuration after d).

It's really not that hard and actually all documented in rtnetlink(7) and netlink(7). unlike libnl the kernel netlink API has been stable since kernel 2.2.

If you need inspiration see iface-linux.c and netlink.c in avahi-core/ in the Avahi package, which I wrote back in kernel 2.2 times.
Comment 43 Dan Winship 2011-10-04 17:45:55 UTC
Attachment 197712 [details] pushed as 38d21f6 - gsettingsbackend: remove useless check
Attachment 197713 [details] pushed as 1481b7b - Add _g_io_module_get_default(), use to simplify other *_get_default()s
Comment 44 Dan Winship 2011-10-04 17:49:54 UTC
Created attachment 198237 [details] [review]
GInetAddressMask: new type for internet address range matching

Eg, for matching a GInetAddress to a range like "10.0.0.0/8" or
"fe80::/10"

====

based on code stolen from the gnome proxy backend, so it would be used
there too
Comment 45 Dan Winship 2011-10-04 17:55:36 UTC
Created attachment 198238 [details] [review]
add GNetworkMonitor, for... monitoring the network

Add GNetworkMonitor and its associated extension point, provide a base
implementation that always claims the network is available, and a
netlink-based implementation built on top of that that actually tracks
the network state.

====

OK, this works except that for some reason, there doesn't seem to be a
netlink notification when the default IPv4 route gets removed, so once
you get online, it thinks you're always online after that. Not sure if
this is a kernel bug or a known quirk or what, so I didn't do anything
about it yet.

Dan? Lennart? Any thoughts?

(If we have to deal with it, I guess the fix is to keep track of the
gateway for the default route as well, and remove the default route
when you no longer can_reach() the gateway.)

As for the division of code: I made GNetworkMonitorBase public so that
other things could derive from it (eg, an NM backend that adds "real
internet" vs "captive portal" information). Except that then I moved
GNetworkMonitorNetlink out of GNetworkMonitorBase, so that I could use
GNetworkMonitorBase in the test program, and it's really the netlink
version that they'd want to derive from... so, probably the netlink
code needs to go back into Base, and I can just add a property to make
it "dumb" for test program purposes.
Comment 46 Lennart Poettering 2011-10-04 20:59:09 UTC
I am pretty sure you need to set NLM_F_ACK too to get a guaranteed reply even if there is no data available.
Comment 47 Lennart Poettering 2011-10-04 21:00:10 UTC
And I'd recommend to use TIOCINQ instead of MSG_PEEK to allocate your buffer size when reading.
Comment 48 Lennart Poettering 2011-10-04 21:07:25 UTC
But I don't see anything wrong otherwise.
Comment 49 Dan Winship 2011-10-17 14:10:54 UTC
(In reply to comment #47)
> And I'd recommend to use TIOCINQ instead of MSG_PEEK to allocate your buffer
> size when reading.

AFAICT, TIOCINQ is only meaningful for ttys
Comment 50 Lennart Poettering 2011-10-17 16:35:10 UTC
Sorry, I meant FIONREAD (aka SIOCINQ) which works on sockets, too. See udp(7) near the end. (yeah that page is about UDP, but it actually works on netlink, too).
Comment 51 Dan Winship 2011-11-07 20:27:00 UTC
Created attachment 200933 [details] [review]
GInetAddressMask: new type for internet address range matching

Eg, for matching a GInetAddress to a range like "10.0.0.0/8" or
"fe80::/10"
Comment 52 Dan Winship 2011-11-07 20:29:06 UTC
Created attachment 200934 [details] [review]
add GNetworkMonitor, for... monitoring the network

Add GNetworkMonitor and its associated extension point, provide a base
implementation that always claims the network is available, and a
netlink-based implementation built on top of that that actually tracks
the network state.

====

Ended up being forced to just re-dump the routing table every time
anything changes (to deal with the problem of changes to the default
route never being indicated). Not totally happy with the current state
of this: when enabling or disabling an interface via NM, it ends up
emitting network-changed multiple times. Probably I'll just end up
increasing the delay.

Also, still not sure how it will be extended to allow captive wifi
portal indication...

Anyway, will commit this soon if there are no objections
Comment 53 Matthias Clasen 2011-11-08 06:06:58 UTC
Review of attachment 200933 [details] [review]:

Sounds like something very testable, so there should really be at least a few basic tests to go along with this.

::: gio/ginetaddressmask.c
@@ +39,3 @@
+ * of the base address are relevant for matching purposes. These are
+ * often given in string form. Eg, "10.0.0.0/8", or "fe80::/10".
+ */

The examples would be more useful if you added an example of what addresses are matched by these.

@@ +266,3 @@
+ * delimited by a "/". If it is not present, then the length is
+ * assumed to be the full length of the address.
+ *

Should add a few examples of valid strings here.
Comment 54 Dan Winship 2011-11-08 13:18:30 UTC
(In reply to comment #53)
> Review of attachment 200933 [details] [review]:
> 
> Sounds like something very testable, so there should really be at least a few
> basic tests to go along with this.

GInetAddressMask gets thoroughly tested by the network-monitor test added in the second patch.
Comment 55 Matthias Clasen 2011-11-10 00:47:53 UTC
Ah, ok. I hadn't noticed that. Thats great, then.
Comment 56 Dan Winship 2011-11-14 19:03:46 UTC
pushed. captive wifi portal detection can happen in a new bug

Attachment 200933 [details] pushed as eb9755d - GInetAddressMask: new type for internet address range matching
Attachment 200934 [details] pushed as fe5ba0f - add GNetworkMonitor, for... monitoring the network