GNOME Bugzilla – Bug 677688
Improve networking support
Last modified: 2016-03-31 14:02:59 UTC
Currently Boxes uses libvirt 'user' networking which is very limited. It's slow, the VM cannot be connected to (ssh, http, ...), ... It's enough for most use cases, but I can see why people would want something more complete We could offer some basic network configuration on VM creation and in the properties * outbound access (default, current behaviour) * no network (is that needed at all?) * full network access (the box would be exposed on the network as if it was a real machine, this means address assigned by the LAN dhcpd, possible to connect to the VM, ...) But this needs design input, and the 3rd item needs libvirt work as well.
This one is very obvious to me. We must do this automatically. I still can't remember the diff between different network options of libvirt so this will only confuse most users. Anyway, network should just work (TM) and user need not care about it. We of course need to do some work in libvirt/qemu level to make the default network not be limited by any means.
(In reply to comment #1) > This one is very obvious to me. We must do this automatically. Automatically exposing VMs on the LAN has various security implications (from the sysadmins not wanting this to the VM being exposed to Windows worms that may lurk on the LAN) that's why I suspect some kind of simple "networking" setup will be needed. Please not I only suggested 3 easy to describe choices, not the tons of network settings that are available to us through libvirt. Anyway, the main point of this bug is to not forget to get design input about this ;)
We could also draw a line there, and say that advanced network configuration should be done via virt-manager or manual xml editing (for example, I remember editing networking option in VirtualBox XML files)
I think a simple two states option "do you want this VM to appear or your network" or something makes a lot of sense. As an user that's all I care about, really.
(In reply to comment #4) > I think a simple two states option "do you want this VM to appear or your > network" or something makes a lot of sense. As an user that's all I care about, > really. I think you are talking about a different thing. This bug is about choosing the right network emulation in libvirt/qemu for the VM, not whether the VM is visible on the network or not. The VMs boxes currently create are private to the user.
The networking configuration we use is not very interesting by itself. What is interesting is what useful user-visible features we can get by using different networking mechanisms. What comes to mind is mostly better performance, and being able to access the VM from other machines (ie having it visible on the network). So this bug is very much about making the VM visible on the network, from comment #1 "full network access (the box would be exposed on the network as if it was a real machine)"
Yeah exactly. I don't care about the underlying libvirt/qemu network configuration. I'd just like to be able to choose is reachable from the network (I can ssh the guest from the host for example) or if it's living in its own private network.
Myllynen pointed briefly introduced me to something called macvtap. Something to look into? http://lost-and-found-narihiro.blogspot.fi/2012/07/fedora-17-kvm-use-macvtap.html https://bugzilla.redhat.com/show_bug.cgi?id=855740
(In reply to comment #7) > Yeah exactly. I don't care about the underlying libvirt/qemu network > configuration. I'd just like to be able to choose is reachable from the network > (I can ssh the guest from the host for example) or if it's living in its own > private network. Both of these things are the problems I constantly struggle with in vms: - Is it on the network ? - How do I ssh into it ? Answering these two questions in an easy-to-understand way in the properties would be awesome.
I think we basically want to have two possible modes: 1. A NAT:ed out only mode where the VM is safely shielded from incomming stuff on the network. 2. "Full network support", where the VM behaves essentially as a separate computer on the network. Additionally, in 1, even though we don't bridge external traffic into the VM it should be possible to access it from the host machine, and we should make it easy to figure out how to do this, so that you can e.g. ssh into the VM from the host itself.
Note that (as currently implemented) mode 1 incurs a noticeable performance degradation. Mode 2 requires host collaboration to set up a bridge, which can be provided by libvirt but only if the system libvirtd is running. Regarding "how do I ssh into it", libvirt is able to answer this, both via DHCP snooping and by running a program (qemu-ga) in the virtual machine. I don't know the details exactly.
Created attachment 233282 [details] [review] util: API to detect if system libvirtd is available
Created attachment 233283 [details] [review] installer: Use bridge net if system libvirt is available Use bridge networking in new VMs if system libvirt is available. This network is much faster than the user networking but unfortunately requires system libvirtd to be running to take care of the privileged action needed. Since system libvirtd is typically started on boot when libvirt is installed, we can perhaps rely on it being available at subsequent launch of the created VM in future? NB: We don't need to connect to system libvirt for this to work.
Created attachment 233284 [details] [review] libvirt-machine: Fix visibility of a few methods
Created attachment 233285 [details] [review] libvirt-machine: Allow changing network interface Allow changing network interface from properties view.
The attached patches implements something that IMHO would be a good starting point at least. Its based on a patch and suggestion from Paolo Bonzini. Issues: * Not at all sure about the labels in properties view, suggestions more than welcome. * On changing the interface in combo, I get a crash but stack is only inside gtk+ so its probably my gtk+ installation so please try them out and let me know if you also get a crash. The interface *is* changed though. * Requires git master of libvirt-glib (release due monday and our git master already requires it anyways).
Review of attachment 233285 [details] [review]: ::: src/libvirt-machine.vala @@ +624,3 @@ + combo.active = bridge_net? 0 : 1; + var property = add_property (ref list, _("Network"), combo); + if (!bridge_net && !is_system_libvirt_running ()) I don't like calling a blocking function like is_system_libvirt_running() here...
Review of attachment 233285 [details] [review]: ::: src/libvirt-machine.vala @@ +631,3 @@ + try_change_network_interface (bridge_net); + update_domain_config (); + property.refresh_properties (); No need to refresh_properties here. The combo changing will not affect the set of properties for the machine. This is what is causing your crash, as it destroys the combo while in the changed signal handler.
Created attachment 234089 [details] [review] libvirt-machine: Allow changing network interface Allow changing network interface from properties view.
This version fixes the last network code so that it works (although it still does a sync call). However, reading the docs at http://libvirt.org/formatdomain.html#elementsNICSVirtual it seems like we should not generally be using bridge-to-lan, but rather network mode. This also gets rid of my worry of hardcoding "vibr0" as the interface name. The question is, can we use this in a session libvirtd domain: <interface type='network'> <source network='default'/> </interface> Or will that refer to a network on the session libvirtd? If so, we need to create our own networks there. Can we do that?
Also, why does libvirt-glib have gvir_connection_get_networks() etc ifdefed out?
(In reply to comment #21) > This version fixes the last network code so that it works (although it still > does a sync call). However, reading the docs at > http://libvirt.org/formatdomain.html#elementsNICSVirtual it seems like we > should not generally be using bridge-to-lan, but rather network mode. This also > gets rid of my worry of hardcoding "vibr0" as the interface name. > > The question is, can we use this in a session libvirtd domain: > <interface type='network'> > <source network='default'/> > </interface> > > Or will that refer to a network on the session libvirtd? If so, we need to > create our own networks there. Can we do that? Unfortunately at this time, the 'session' libvirtd is pretty much restricted to only using the 'user' mode networking stack because that's the only thing that can be configured without root privileges. The bridge/network modes are only available to VMs running in the 'system' libvirtd which has fully privileges to screw around with host networking. Latest QEMU did introduce a setuid helper program to allow a non-root QEMU to attach itself to a pre-configured host bridge. This is kind of sucky for general desktop usage though, because you need the admin to configure the bridge, and then edit a config file to whitelist individual users+bridges. Two other approaches we'd like to consider in libvirt are 1. Split off the bit of 'system' libvirtd that deals with network configuration into a separate virtnetworkd and allow the session libvirtd to talk to that to deal with networking. Obviously policykit authentication would be involved here still 2. If network manager had the ability to configure bridges, and create TAP devices, the session libvirtd could talk to network manager to request creation of the TAP device + bridge. NM could pass back the pre-opened FD of the TAP device to libvirtd, which could then pass it to QEMU.
The patch above does seem to work, although it relies on a bunch of things: * system libvirtd is running * system libvirtd has a network configured that uses the "vibr0" interface * qemu has been configured to allow the user to connect to vibr0 The last part is not on by default upstream, but fedora ships /etc/qemu/bridge.conf with: allow virbr0 So, on fedora (only) this patch works, unless the user changed anything.
(In reply to comment #24) > The patch above does seem to work, although it relies on a bunch of things: > > * system libvirtd is running > * system libvirtd has a network configured that uses the "vibr0" interface > * qemu has been configured to allow the user to connect to vibr0 > > The last part is not on by default upstream, but fedora ships > /etc/qemu/bridge.conf with: > allow virbr0 > > So, on fedora (only) this patch works, unless the user changed anything. Ah, I think Paolo didn't know this is fedora-specific and hence I didn't know either. :( Why is it disabled upstream by default btw?
Maybe its risky security-wise to let anyone use the bridge. I dunno.
> I think Paolo didn't know this is fedora-specific I did, it was me who added the /etc/qemu/bridge.conf file to the Fedora QEMU. :) It's disabled upstream because QEMU doesn't know that a virbr0 could exist in the system. In Fedora we chose to integrate the bridge helper with libvirt, but that's a distro choice, not a QEMU choice.
(In reply to comment #27) > > I think Paolo didn't know this is fedora-specific > > I did, it was me who added the /etc/qemu/bridge.conf file to the Fedora QEMU. > :) Then you misunderstood what I meant by "Is this *all* upstream?". :) > It's disabled upstream because QEMU doesn't know that a virbr0 could exist in > the system. In Fedora we chose to integrate the bridge helper with libvirt, > but that's a distro choice, not a QEMU choice. Hmm.. perhaps we could also check if virbr0 is up? Don't know if then there is a need to check if system libvirtd is running?
> Hmm.. perhaps we could also check if virbr0 is up? Don't know if then there is > a need to check if system libvirtd is running? It is the same. Perhaps that on virbr0 is a better check though, indeed. What changes in Fedora is not "whether virbr0 exists", it is "whether qemu-bridge-helper is allowed by default to access virbr0".
(In reply to comment #29) > > Hmm.. perhaps we could also check if virbr0 is up? Don't know if then there is > > a need to check if system libvirtd is running? > > It is the same. Perhaps that on virbr0 is a better check though, indeed. > > What changes in Fedora is not "whether virbr0 exists", it is "whether > qemu-bridge-helper is allowed by default to access virbr0". Good point but how do i check that in an not so ugly way?
I'm not sure I follow everything, but note that what libvirt calls "Bridged networking" apparently won't work if your computer's connection is through wifi, which I would expect to be the case relatively often: http://wiki.libvirt.org/page/Networking#Bridged_networking_.28aka_.22shared_physical_device.22.29
sorry bridging with wifi appears possible after all: http://ubuntuforums.org/showthread.php?t=1766674 http://blog.bodhizazen.net/linux/bridge-wireless-cards/
(In reply to comment #0) > * full network access (the box would be exposed on the network as if it was a > real machine, this means address assigned by the LAN dhcpd, possible to connect > to the VM, ...) I think bridged networking isn't a use case for Boxes. Such VMs should go in the system libvirt. Maybe boxes has a button like "Migrate this VM to system" or so.
(In reply to comment #9) > (In reply to comment #7) > > Yeah exactly. I don't care about the underlying libvirt/qemu network > > configuration. I'd just like to be able to choose is reachable from the network > > (I can ssh the guest from the host for example) or if it's living in its own > > private network. > > Both of these things are the problems I constantly struggle with in vms: > > - Is it on the network ? Should be answered by the network UI in the guest. > - How do I ssh into it ? I think it's often better to do things like use a serial console for interactive access. The downside is it requires guest configuration, but that's something boxes could automate. For copying files in/out, 9P is nice (where supported, which unfortunately doesn't include RHEL).
(In reply to comment #33) > (In reply to comment #0) > > > * full network access (the box would be exposed on the network as if it was a > > real machine, this means address assigned by the LAN dhcpd, possible to connect > > to the VM, ...) > > I think bridged networking isn't a use case for Boxes. Such VMs should go in > the system libvirt. Why not? Besides we can't do NAT either as that only possible in system libvirt. > Maybe boxes has a button like "Migrate this VM to system" or so. We really should be discouraging all usage of system libvirt. All your VMs shared w/ every user and running privileged is wrong on many levels. Maybe it makes sense on servers etc but not on desktop at least. Boxes now even offers you to import all your VMs over to session libvirt.
> We really should be discouraging all usage of system libvirt. For Boxes, definitely. The approaches of comment 23 are still possible. On the QEMU side you can use QEMU's existing support for TAP helpers, but use a different executable than the default /usr/libexec/qemu-bridge-helper. This helper would not be privileged and would get a file descriptor from NM or the hypothetical virtnetworkd. > All your VMs shared w/ every user and running privileged is wrong on many > levels. However, note that the VMs do not run privileged. They run as user qemu, and each VM is separated from the others using MCS if SELinux is enabled.
(In reply to comment #36) > > We really should be discouraging all usage of system libvirt. > > For Boxes, definitely. The approaches of comment 23 are still possible. > > On the QEMU side you can use QEMU's existing support for TAP helpers, but use a > different executable than the default /usr/libexec/qemu-bridge-helper. This > helper would not be privileged and would get a file descriptor from NM or the > hypothetical virtnetworkd. If NM ever got that support, then we'd make unprivileged libvirtd use it instead of the setuid helper. > > > All your VMs shared w/ every user and running privileged is wrong on many > > levels. > > However, note that the VMs do not run privileged. They run as user qemu, and > each VM is separated from the others using MCS if SELinux is enabled. The key issue is really that the privileges / user are different. Experiance with virt-manager has shown that this causes no end of pain because the unprivileged 'qemu' can't see into user's home directories with their default restrictive permissison - you need to manually set group-execute perm bit on all directories leading upto the image. Supporting the system libvirtd in Boxes will just open you up to a whole new set of bug reports & user unhappiness.
Created attachment 263869 [details] [review] util: API to detect if libvirt bridge net is available We do this by checking if: * System libvirtd is running * Qemu is allowed to run a helper to access bridge net. WARNING: We only look at a hardcoded path currently so it will fail for qemu installed in custom prefix.
Created attachment 263870 [details] [review] installer: Use bridge net if system libvirt is available Use bridge networking in new VMs, if available. This network is much faster than the user networking. Unfortunately bridge networking being available implies running system libvirtd. However since system libvirtd is typically started on boot when libvirt is installed, we can perhaps rely on it being available at subsequent launch of the created VM in future? NB: We don't need to actually connect to system libvirt for this to work.
Created attachment 263871 [details] [review] libvirt-machine-props: Fix visibility of a few methods
Created attachment 263872 [details] [review] libvirt-machine-props: Allow changing network iface Allow changing network interface from properties view. Co-author: Alexander Larsson <alexl@redhat.com>
Pushing these already as this bug has been pending for a very long time now and its not like there is many other solutions available to us. Feel free to do post-commit reviews and I'll try my best to address the concerns. Attachment 263869 [details] pushed as 197d53b - util: API to detect if libvirt bridge net is available Attachment 263870 [details] pushed as 414d324 - installer: Use bridge net if system libvirt is available Attachment 263871 [details] pushed as 4fb18be - libvirt-machine-props: Fix visibility of a few methods Attachment 263872 [details] pushed as b2128c2 - libvirt-machine-props: Allow changing network iface
Upon further discussion with Christophe and Daniel on IRC, it turns out that the virbr0 bridge net we are using is essentially a NAT. Hence we should just use that if available by default and not have to provide any settings.
Filed a separate bug for removal of network option from properties: https://bugzilla.gnome.org/show_bug.cgi?id=720399