From owner-freebsd-virtualization@freebsd.org Tue Oct 20 12:10:56 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAE8142FBBE for ; Tue, 20 Oct 2020 12:10:56 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFssW47Ylz4Fpm for ; Tue, 20 Oct 2020 12:10:55 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 8611438ACC; Tue, 20 Oct 2020 08:10:52 -0400 (EDT) To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> Date: Tue, 20 Oct 2020 08:10:51 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx" X-Rspamd-Queue-Id: 4CFssW47Ylz4Fpm X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-2.23 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.90)[-0.896]; NEURAL_HAM_LONG(-0.99)[-0.995]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; DMARC_NA(0.00)[druid.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.24)[-0.237]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 12:10:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx Content-Type: multipart/mixed; boundary="Nmm8MEmRt5QJUJkoInrxkJu5yVYeqPOlE"; protected-headers="v1" From: D'Arcy Cain To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org Message-ID: <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> In-Reply-To: --Nmm8MEmRt5QJUJkoInrxkJu5yVYeqPOlE Content-Type: multipart/mixed; boundary="------------5EBE8E1B42D6D408F7A9525D" Content-Language: en-US This is a multi-part message in MIME format. --------------5EBE8E1B42D6D408F7A9525D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/20/20 7:39 AM, Patrick M. Hausen wrote: >> When I started I thought of a switch as analogous to a physical switch= =2E If I am in an office with one ethernet jack but I have multiple devi= ces I might connect a switch (or hub) to the jack and plug my devices int= o the switch. I don't need to create a separate network for my office. A= ll of my devices are on the company network. >=20 > OK, the "switch" interface in FreeBSD is bridge(4). Understood. > Or to cite Radia Perlman: >=20 > A bridge is a network device making forwarding decisions based on layer= 2 addresses. > A router is a network device making forwarding decisions based on layer= 3 addresses. > "Switch" is a marketing term meaning "faster or cheaper than the compet= ition". I always thought that a switch was a hub with packet switching to avoid=20 collisions. >> cloned_interfaces=3D"bridge0" >> ifconfig_bridge0=3D"a.b.c.d.1 addm bge0 addm switch0 up" >=20 >> Except that switch0 doesn't get created until vm-bhyve starts so it pr= obably doesn't exist at that time. >=20 > What is "switch0"? I suspect it is just a bridge interface that gets re= named by > your VM management software. In that case manually creating bridge0 > and all the things we discussed will not get you anywhere. So in vm-bhyve I need to change; @@ -3,9 +3,9 @@ cpu=3D2 memory=3D2G network0_type=3D"virtio-net" -network0_switch=3D"public" +network0_switch=3D"bridge0" network1_type=3D"virtio-net" -network1_switch=3D"private" +network1_switch=3D"bridge1" disk0_type=3D"virtio-blk" disk0_name=3D"disk0.img" disk0_dev=3D"sparse-zvol" Or else rename the bridges to "public" and "private". > Real life example from our environment: >=20 > ifconfig_igb0=3D"-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vla= nhwtso up" ifconfig_bge0=3D"-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vlanh= wtso up" > cloned_interfaces=3D"bridge0" cloned_interfaces=3D"bridge0 bridge1" > ifconfig_bridge0_name=3D"inet0" ifconfig _bridge0_name=3D"public" ifconfig _bridge1_name=3D"private" > ifconfig_inet0=3D"addm igb0 up" ifconfig_public=3D"addm bge0 up" ifconfig_private=3D"addm bge1 up" > ifconfig_inet0_alias0=3D"inet 1.2.3.4/24" ifconfig_public_alias0=3D"inet 1.2.3.4/24" ifconfig_private_alias0=3D"192.168.151.4/14" So why alias? Wouldn't "ifconfig_public=3D" work? > Then we configure iocage to attach the jails to bridge0. >=20 > In your case you would have to tell your VM management tool to attach t= he > VM tap interfaces to bridge0 instead of creating its own "switch0" - wh= ich I > suspect is a bridge interface in disguise. As you can see above we rena= me > all our Internet facing interfaces to "inet0" on all hosts. Then there = are more > like "mgmt0", "priv0", ... like that. So probably the bridge is renamed= to "switch0". If I do the above I guess I can keep the names "public" and "private". >=20 > Tell the tool not to do that and use the preconfigured bridge0 instead.= Or public? inet0 in your example? >=20 > Another useful sysctl to get reproduceable static MAC addresses for the= bridge > itself accross reboots is: >=20 > loader.conf: if_bridge_load=3D"YES" > sysctl.conf: net.link.bridge.inherit_mac=3D1 Not sure I need this as long as arp works as it should. Do I really care= =20 what the MAC is? Cheers. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------5EBE8E1B42D6D408F7A9525D-- --Nmm8MEmRt5QJUJkoInrxkJu5yVYeqPOlE-- --QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX47TywUDAAAAAAAKCRA/5DDweYZnXZAW AQC7/IuVAoE76ZLgp/OJyLaFcxKucGak6cvBOCtSGL2hMgEA1X39WPgNMsyYzHRbOW4ia7UPZ1Qi CLxa3T6WTpCr2A4= =2BsT -----END PGP SIGNATURE----- --QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx--