Date: Tue, 20 Oct 2020 13:39:00 +0200 From: "Patrick M. Hausen" <hausen@punkt.de> To: D'Arcy Cain <darcy@druid.net> Cc: freebsd-virtualization@freebsd.org Subject: Re: When is a switch not a switch? Message-ID: <A075C069-1F89-428C-BDB7-7A9F44A4E283@punkt.de> In-Reply-To: <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <CF189122-7D85-4BF1-9172-75D3EE0E77FB@punkt.de> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] Hi all, > Am 20.10.2020 um 12:50 schrieb D'Arcy Cain <darcy@druid.net>: > > On 10/20/20 5:36 AM, Patrick M. Hausen wrote: >>> I did see that. Does that mean that I don't even need to create switches at all? >> What is a switch in this context? I use bridge interfaces to connect jails via epair >> and VMs via tap. > > When I started I thought of a switch as analogous to a physical switch. If I am in an office with one ethernet jack but I have multiple devices I might connect a switch (or hub) to the jack and plug my devices into the switch. I don't need to create a separate network for my office. All of my devices are on the company network. OK, the "switch" interface in FreeBSD is bridge(4). Or to cite Radia Perlman: A bridge is a network device making forwarding decisions based on layer 2 addresses. A router is a network device making forwarding decisions based on layer 3 addresses. "Switch" is a marketing term meaning "faster or cheaper than the competition". > cloned_interfaces="bridge0" > ifconfig_bridge0="a.b.c.d.1 addm bge0 addm switch0 up" > Except that switch0 doesn't get created until vm-bhyve starts so it probably doesn't exist at that time. What is "switch0"? I suspect it is just a bridge interface that gets renamed by your VM management software. In that case manually creating bridge0 and all the things we discussed will not get you anywhere. >> If em0 does not have an IP address on the host and should be used >> exclusively for VMs, then the bridge does not need an IP address, either. >> Still you need to configure em0 "up". > > I can't imagine a scenario like that. You probably always need access to the host for maintenance. Well, there could be a second hardware interface for host communication ... And if one of the two is member of the bridge and the other one isn't it is perfectly valid to plug them into the same broadcast domain and get e.g. 1Gbit/s for the host and 1Gbit/s for all the jails or VMs. >> And additionally ... >> - you should disable all hardware acceleration features on the physical interface > > Like ASF? Real life example from our environment: ifconfig_igb0="-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vlanhwtso up" cloned_interfaces="bridge0" ifconfig_bridge0_name="inet0" ifconfig_inet0="addm igb0 up" ifconfig_inet0_alias0="inet 1.2.3.4/24" Then we configure iocage to attach the jails to bridge0. In your case you would have to tell your VM management tool to attach the VM tap interfaces to bridge0 instead of creating its own "switch0" - which I suspect is a bridge interface in disguise. As you can see above we rename all our Internet facing interfaces to "inet0" on all hosts. Then there are more like "mgmt0", "priv0", ... like that. So probably the bridge is renamed to "switch0". Tell the tool not to do that and use the preconfigured bridge0 instead. Another useful sysctl to get reproduceable static MAC addresses for the bridge itself accross reboots is: loader.conf: if_bridge_load="YES" sysctl.conf: net.link.bridge.inherit_mac=1 HTH, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Geschäftsführer: Jürgen Egeling, Daniel Lienert, Fabian Stein [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+OzFQACgkQkG8u4u3a iVwDrggAk1Ah6hgV4QhtC+Vt+pU8AWuZdazV9gsnIC/KwkJeoPb9VaIIgiFuWeAn qpUAZKQ/talEYkXiTaVqHc5ZZXqibUShjWdh2Bx9swCih/HakFgzxw3Jspwq1gI7 665ZtSC1YPnpbcECZ+/gDvw1z8wVHVgrbpa9WrVm9Wt1omPn2wSeQcHMGmlGShWF 49KfHqVWt/FjzReM1wAOqsDFuHzfltIqQPQV4ptvhbgtvnE3bdCIbEjMba2f7zmx A0tbHWR8SnnW2NnrqWLDHP8V8wZ9a/62J3wuatU3tBXS9jmYVRVytwS0Tvrosart EQmtmq8/rx4uZ1X1vM4at9zhAXEHzA== =gdBj -----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?A075C069-1F89-428C-BDB7-7A9F44A4E283>