From owner-freebsd-virtualization@freebsd.org Sun Oct 18 21:01:15 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5A1B043BFF3 for ; Sun, 18 Oct 2020 21:01:15 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 4CDskM1NHnz3T91 for ; Sun, 18 Oct 2020 21:01:15 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: by mailman.nyi.freebsd.org (Postfix) id 228E343BECE; Sun, 18 Oct 2020 21:01:15 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1699B43BECC for ; Sun, 18 Oct 2020 21:01:15 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CDskL5164z3TFT for ; Sun, 18 Oct 2020 21:01:14 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6199C18B34 for ; Sun, 18 Oct 2020 21:01:14 +0000 (UTC) (envelope-from bugzilla-noreply@FreeBSD.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 09IL1Edm047582 for ; Sun, 18 Oct 2020 21:01:14 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Received: (from bugzilla@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 09IL1EAk047579 for virtualization@FreeBSD.org; Sun, 18 Oct 2020 21:01:14 GMT (envelope-from bugzilla-noreply@FreeBSD.org) Message-Id: <202010182101.09IL1EAk047579@kenobi.freebsd.org> X-Authentication-Warning: kenobi.freebsd.org: bugzilla set sender to bugzilla-noreply@FreeBSD.org using -f From: bugzilla-noreply@FreeBSD.org To: virtualization@FreeBSD.org Subject: Problem reports for virtualization@FreeBSD.org that need special attention Date: Sun, 18 Oct 2020 21:01:14 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 18 Oct 2020 21:01:15 -0000 To view an individual PR, use: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=(Bug Id). The following is a listing of current problems submitted by FreeBSD users, which need special attention. These represent problem reports covering all versions including experimental development code and obsolete releases. Status | Bug Id | Description ------------+-----------+--------------------------------------------------- In Progress | 247208 | mpt(4): VMWare virtualized LSI controller panics New | 240945 | [hyper-v] [netvsc] hn network driver incorrectly 2 problems total for which you should take action. From owner-freebsd-virtualization@freebsd.org Tue Oct 20 02:02:57 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E89D43E294 for ; Tue, 20 Oct 2020 02:02:57 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFcMz5kVwz4Wpm for ; Tue, 20 Oct 2020 02:02:55 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 1B6B33812C for ; Mon, 19 Oct 2020 22:02:18 -0400 (EDT) To: freebsd-virtualization@freebsd.org From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: When is a switch not a switch? Message-ID: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> Date: Mon, 19 Oct 2020 22:02:17 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="jVzU7RkHyoBriBy8iuFhRVZkXbfYyLhb4" X-Rspamd-Queue-Id: 4CFcMz5kVwz4Wpm X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-0.28 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.994]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.20)[0.203]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_SHORT(0.61)[0.609]; DMARC_NA(0.00)[druid.net]; ARC_NA(0.00)[]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 02:02:57 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --jVzU7RkHyoBriBy8iuFhRVZkXbfYyLhb4 Content-Type: multipart/mixed; boundary="vjirb1z0fgPSLvGiZf0mNmArUiUwxwQNj"; protected-headers="v1" From: D'Arcy Cain To: freebsd-virtualization@freebsd.org Message-ID: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> Subject: When is a switch not a switch? --vjirb1z0fgPSLvGiZf0mNmArUiUwxwQNj Content-Type: multipart/mixed; boundary="------------5AA5242C1A18EB13CC2E233C" Content-Language: en-US This is a multi-part message in MIME format. --------------5AA5242C1A18EB13CC2E233C Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable I am using bhyve with vm-bhyve, I am trying to set up a virtual network = with multiple hosts. The idea is that a VM would be on the same virtual = network no matter which actual host it is on. Say I have a public network a.b.c.0/24. I thought I could create a switc= h=20 on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and = a.b.c.101. The idea would be that the VMs would appear on the real netwo= rk.=20 Then the 101 VM could migrate to a.b.c.2 and still be accessible. I=20 envisioned some sort of proxy arp would happen so that every VM would sim= ply=20 announce itself wherever it was. This did seem to work in that I could ping from the VM: # ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: icmp_seq=3D0 ttl=3D114 time=3D1.734 ms Even IPV6: # ping6 2605:2600:1001::4b PING6(56=3D40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b 16 bytes from 2605:2600:1001::4b, icmp_seq=3D0 hlim=3D64 time=3D0.960 ms 16 bytes from 2605:2600:1001::4b, icmp_seq=3D1 hlim=3D64 time=3D0.415 ms However TCP doesn't work. In fact, I could only ping by IP because the=20 system couldn't connect to the DNS server, to get an address even though = it=20 could ping it. I guess my first question is does this seem doable? If so, what am I=20 missing? Is it possible that a bhyve switch is more like a router? Thanks. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------5AA5242C1A18EB13CC2E233C-- --vjirb1z0fgPSLvGiZf0mNmArUiUwxwQNj-- --jVzU7RkHyoBriBy8iuFhRVZkXbfYyLhb4 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX45FKQUDAAAAAAAKCRA/5DDweYZnXZXp AP4lCdN3GC8dXFhmD6sxAqlzbNK1WG3R9xskwBduSZh66wD/Vb+p6N7tgOMaTueIYXDA1dSgI71r hxnO2UeGIdBvXQw= =dLiE -----END PGP SIGNATURE----- --jVzU7RkHyoBriBy8iuFhRVZkXbfYyLhb4-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 06:56:42 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A0973446CDB for ; Tue, 20 Oct 2020 06:56:42 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFktx4T6Sz3bc0 for ; Tue, 20 Oct 2020 06:56:41 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 09K6uU6v078799 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 19 Oct 2020 23:56:30 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 09K6uUBw078798; Mon, 19 Oct 2020 23:56:30 -0700 (PDT) (envelope-from jmg) Date: Mon, 19 Oct 2020 23:56:30 -0700 From: John-Mark Gurney To: "D'Arcy Cain" Cc: freebsd-virtualization@freebsd.org Subject: Re: When is a switch not a switch? Message-ID: <20201020065630.GE8272@funkthat.com> Mail-Followup-To: D'Arcy Cain , freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline In-Reply-To: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Mon, 19 Oct 2020 23:56:30 -0700 (PDT) X-Rspamd-Queue-Id: 4CFktx4T6Sz3bc0 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-0.94 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.08)[0.080]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.009]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.11)[-0.108]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_SPF_NA(0.00)[no SPF record]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 06:56:42 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable D'Arcy Cain wrote this message on Mon, Oct 19, 2020 at 22:02 -0400: > I am using bhyve with vm-bhyve, I am trying to set up a virtual network= =20 > with multiple hosts. The idea is that a VM would be on the same virtual= =20 > network no matter which actual host it is on. >=20 > Say I have a public network a.b.c.0/24. I thought I could create a switc= h=20 > on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and= =20 > a.b.c.101. The idea would be that the VMs would appear on the real netwo= rk.=20 > Then the 101 VM could migrate to a.b.c.2 and still be accessible. I=20 > envisioned some sort of proxy arp would happen so that every VM would sim= ply=20 > announce itself wherever it was. >=20 > This did seem to work in that I could ping from the VM: >=20 > # ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=3D0 ttl=3D114 time=3D1.734 ms >=20 > Even IPV6: >=20 > # ping6 2605:2600:1001::4b > PING6(56=3D40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D0 hlim=3D64 time=3D0.960 ms > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D1 hlim=3D64 time=3D0.415 ms >=20 > However TCP doesn't work. In fact, I could only ping by IP because the= =20 > system couldn't connect to the DNS server, to get an address even though = it=20 > could ping it. >=20 > I guess my first question is does this seem doable? If so, what am I=20 > missing? Is it possible that a bhyve switch is more like a router? By switch, do you mean use bridge? How specifically is the network configured? What you are describing sounds like what I do w/ bridge, but my use was slightly more complicated. Say your host has em0 as the main network, you would create a bridge0 interface, either via cloned_interfaces or via "ifconfig bridge0 create". Then you would put the em0 interface as a member of the bridge interface. You would also add the tap interfaces of the various bhyve vms as well (don't forget to make sure the tap interface is up on the host, net.link.tap.up_on_open helps w/ this)... I have heard (and that is the way I do that), that you have to put the host IPs on the bridge0 interface, and not the em0 interface. --=20 John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJfjoodXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2MEI1RTRGMTNDNzYyMDZDNjEyMDBCNjAy MDVGMEIzM0REMDA2QURBAAoJECBfCzPdAGraXcYP/0rzorXTZBfADZdRhm2DBuie NwdWSOIwgp1nHLwgLdAm58p2D7CzeDjrQ9CIqNGUeFRv/VIGpzJCuyTTeaWpJfWQ a5eqPwdpiPPBICS8JArk4ixm7Q5LZby8k2b1f6gM9KEkwjm4estrqNLd1w7tGa0K wIWscYaVEd5qj5eIMyD13XdeTKumGywXEtzN1oeNCJVF8ycoUkipTuY1ZKunICpp +N/2dLiJetFps2bNuz0G8SqevaAjn0NZ0YH2k1HlQPuIKzP+HnTxDL+k7bTTRbZj fRSIh/w/HIrpjjFQ6dLUiJVAvNSf4gM3A5XexgHCgiETOvUiF+c5SwaIZ0i/Gq3p dI9Bg7M9XIt88jGPoO6Zv/fW0gCpFGilq8jWNg+vO3BY6bZovzx91c9Np9o2TJig fIXKvIoGlC1Ba6+YrUqg5t3htA+ImA0nzo8TWQOOcghXLGrhfpFg3A/qRCiBvutL v+k9nDTUBp5IIQFiTB1uj583L1rZ8JNMfoYEGchun2PnURxuHW+QITBx8FQufjRA VKc3FvhLHe1N26J3+kBtjxYbUlzUAB5CKiKjaU9nNP7IvGy+UuWs7A6/ZQXea4ng jIVi7sD4XaTD27HpTVoaubMBBdDWof14+ZMPA/+zLTi2FWq/eCJ8M92V4NqRqxx4 MxA4GJA3kw+E1bGzDeiA =POR4 -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 08:36:37 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E578429AFD for ; Tue, 20 Oct 2020 08:36:37 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mail.punkt.de (mail.punkt.de [IPv6:2a00:b580:8000:11:1c6b:7032:35e9:5616]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFn6D2wmCz41YJ for ; Tue, 20 Oct 2020 08:36:36 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from [217.29.46.75] (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.punkt.de (Postfix) with ESMTPSA id 2CFAF1010D; Tue, 20 Oct 2020 10:36:27 +0200 (CEST) From: "Patrick M. Hausen" Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_29C0DE09-FC0C-4ADD-B68F-9A0A41973AF1"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Subject: Re: When is a switch not a switch? Date: Tue, 20 Oct 2020 10:36:08 +0200 In-Reply-To: <20201020065630.GE8272@funkthat.com> Cc: D'Arcy Cain , freebsd-virtualization@freebsd.org To: John-Mark Gurney References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> X-Mailer: Apple Mail (2.3445.104.17) X-Rspamd-Queue-Id: 4CFn6D2wmCz41YJ X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 2a00:b580:8000:11:1c6b:7032:35e9:5616 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-3.70 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:b580::/32]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; ARC_NA(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[punkt.de]; NEURAL_HAM_LONG(-1.00)[-1.001]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.82)[-0.816]; NEURAL_HAM_MEDIUM(-0.98)[-0.980]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16188, ipnet:2a00:b580::/32, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 08:36:37 -0000 --Apple-Mail=_29C0DE09-FC0C-4ADD-B68F-9A0A41973AF1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi all, > Am 20.10.2020 um 08:56 schrieb John-Mark Gurney : > I have heard (and that is the way I do that), that you have to put the > host IPs on the bridge0 interface, and not the em0 interface. It's officially documented here: = https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-bridgin= g.html "If the bridge host needs an IP address, set it on the bridge interface, = not on the member interfaces." Kind regards, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Gesch=C3=A4ftsf=C3=BChrer: J=C3=BCrgen Egeling, Daniel Lienert, Fabian = Stein --Apple-Mail=_29C0DE09-FC0C-4ADD-B68F-9A0A41973AF1 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+OoXgACgkQkG8u4u3a iVwrWAf7Bj8QzRCfRPJdet2hdHk3PdQ/WDK6/geh/kOj0E83Hzsi2E8NhLbp0UAL ANowa9SHNNdudfLQIVJc+IpvjIhCgrPomZ+x9MXmfePjdsOGr6Om0RmKxGuBUOS9 xNjZdTEDelPUdhlrW6JpMA9wy1uKHgsxBFMcr02JdwXzCLLmWBhL16j2uBDhysdS K+lqNVXMakgrMGqVNaTCUZP8KwVzGx9vJRGvpjLp3gJIfyi5rCSiuM5WLQdIxeem cJw3bCvSlYnjLxWuLbtfnYCLIsEoimGQA+4yfK2DEyTsS5EMHR2DLGho2Q/wb0v9 l/UOEaka2LOAgxctDuLtFkJxIIU1kQ== =Kf1Y -----END PGP SIGNATURE----- --Apple-Mail=_29C0DE09-FC0C-4ADD-B68F-9A0A41973AF1-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 09:23:01 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D98842B49A for ; Tue, 20 Oct 2020 09:23:01 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFp7m1dcHz43yd for ; Tue, 20 Oct 2020 09:22:59 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 6F1FE3874C for ; Tue, 20 Oct 2020 05:22:58 -0400 (EDT) To: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <5d2726bb-abb8-3648-1e56-9249a3493095@druid.net> Date: Tue, 20 Oct 2020 05:22:57 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <20201020065630.GE8272@funkthat.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="30I12hOzoiVtVZRliSK06CgeL4tZqDSMA" X-Rspamd-Queue-Id: 4CFp7m1dcHz43yd X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-1.01 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-0.99)[-0.993]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_MEDIUM(-0.19)[-0.189]; TO_DN_NONE(0.00)[]; NEURAL_SPAM_SHORT(0.27)[0.274]; DMARC_NA(0.00)[druid.net]; ARC_NA(0.00)[]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 09:23:01 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --30I12hOzoiVtVZRliSK06CgeL4tZqDSMA Content-Type: multipart/mixed; boundary="gdgEPREXL9TaObQP4DXAGnTStytJHIAQt"; protected-headers="v1" From: D'Arcy Cain To: freebsd-virtualization@freebsd.org Message-ID: <5d2726bb-abb8-3648-1e56-9249a3493095@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> In-Reply-To: <20201020065630.GE8272@funkthat.com> --gdgEPREXL9TaObQP4DXAGnTStytJHIAQt Content-Type: multipart/mixed; boundary="------------057A5AAF8CB944A1C1DA409B" Content-Language: en-US This is a multi-part message in MIME format. --------------057A5AAF8CB944A1C1DA409B Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable On 10/20/20 2:56 AM, John-Mark Gurney wrote: > By switch, do you mean use bridge? How specifically is the network > configured? Yes. I did try bridge first but I may not have understand all the nuance= s.=20 I also thought that "switch" meant the same thing as a physical switch = but=20 I guess it is more like a router. That was the point of my subject. > What you are describing sounds like what I do w/ bridge, but my use was= > slightly more complicated. >=20 > Say your host has em0 as the main network, you would create a bridge0 > interface, either via cloned_interfaces or via "ifconfig bridge0 create= ". > Then you would put the em0 interface as a member of the bridge "ifconfig bridge0 addm bge0" in my case but I also have a private network= so=20 "ifconfig bridge0 addm bge0 addm bge1" then. Or do I need two bridges? > interface. You would also add the tap interfaces of the various bhyve > vms as well (don't forget to make sure the tap interface is up on the > host, net.link.tap.up_on_open helps w/ this)... This is the part I am trying to automate so that VM can freely move betwe= en=20 hosts. Is there a way to make tap automatically add itself to a bridge? Thanks for your help. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------057A5AAF8CB944A1C1DA409B-- --gdgEPREXL9TaObQP4DXAGnTStytJHIAQt-- --30I12hOzoiVtVZRliSK06CgeL4tZqDSMA Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX46scQUDAAAAAAAKCRA/5DDweYZnXYbd AQCk3vPfkraXGv5YA1zbrH8GQGraOa/6q4bULnO/f8+rlgEA0wZFsP++/kA6Jc+vPtDRKfwyP2GH fKq5ApTRNXEFuwM= =62f0 -----END PGP SIGNATURE----- --30I12hOzoiVtVZRliSK06CgeL4tZqDSMA-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 09:28:45 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 57FF242B3CF for ; Tue, 20 Oct 2020 09:28:45 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFpGN4xkcz44Bd for ; Tue, 20 Oct 2020 09:28:44 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id B8A353875B; Tue, 20 Oct 2020 05:28:12 -0400 (EDT) To: "Patrick M. Hausen" , John-Mark Gurney Cc: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> Date: Tue, 20 Oct 2020 05:28:12 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="XZaHDU8BE4nVQEEuPfpcac4ZRLhpHehuA" X-Rspamd-Queue-Id: 4CFpGN4xkcz44Bd X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-2.16 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.997]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; DMARC_NA(0.00)[druid.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.21)[-0.212]; NEURAL_HAM_MEDIUM(-0.86)[-0.856]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 09:28:45 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --XZaHDU8BE4nVQEEuPfpcac4ZRLhpHehuA Content-Type: multipart/mixed; boundary="zr8W7hHxEaK1oUSYQwvvjOsG6khUiHsvO"; protected-headers="v1" From: D'Arcy Cain To: "Patrick M. Hausen" , John-Mark Gurney Cc: freebsd-virtualization@freebsd.org Message-ID: <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> In-Reply-To: --zr8W7hHxEaK1oUSYQwvvjOsG6khUiHsvO Content-Type: multipart/mixed; boundary="------------1E688918B39FB62EB98A8A74" Content-Language: en-US This is a multi-part message in MIME format. --------------1E688918B39FB62EB98A8A74 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/20/20 4:36 AM, Patrick M. Hausen wrote: > It's officially documented here: > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-brid= ging.html I did see that. Does that mean that I don't even need to create switches= at=20 all? > "If the bridge host needs an IP address, set it on the bridge interface= , not on the member interfaces." But I don't necessarily need an IP on the bridge itself, right? --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------1E688918B39FB62EB98A8A74-- --zr8W7hHxEaK1oUSYQwvvjOsG6khUiHsvO-- --XZaHDU8BE4nVQEEuPfpcac4ZRLhpHehuA Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX46trAUDAAAAAAAKCRA/5DDweYZnXbDs AQDfyOH+ieUFHk7Ucsx7nOb/DHjR3utNTZ2FoRQxTqMg4gD8CfmP4pO7JwLhsNIGaLrBA0OnqNsR FLzLGrkmPBfseQI= =u6TP -----END PGP SIGNATURE----- --XZaHDU8BE4nVQEEuPfpcac4ZRLhpHehuA-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 09:37:14 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 39EF042BA0B for ; Tue, 20 Oct 2020 09:37:14 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mail.punkt.de (mail.punkt.de [IPv6:2a00:b580:8000:11:1c6b:7032:35e9:5616]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFpS91s78z44fp for ; Tue, 20 Oct 2020 09:37:13 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from [217.29.46.75] (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.punkt.de (Postfix) with ESMTPSA id 7D3DB1024E for ; Tue, 20 Oct 2020 11:37:10 +0200 (CEST) Subject: Re: When is a switch not a switch? Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Content-Type: multipart/signed; boundary="Apple-Mail=_D80FB223-AB2E-4B54-8905-F9131EDC930A"; protocol="application/pgp-signature"; micalg=pgp-sha256 From: "Patrick M. Hausen" Resent-From: "Patrick M. Hausen" In-Reply-To: <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> Date: Tue, 20 Oct 2020 11:36:27 +0200 Resent-Date: Tue, 20 Oct 2020 11:37:10 +0200 Message-Id: <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> Resent-To: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> To: D'Arcy Cain X-Mailer: Apple Mail (2.3445.104.17) X-Rspamd-Queue-Id: 4CFpS91s78z44fp X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 2a00:b580:8000:11:1c6b:7032:35e9:5616 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-0.13 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FORGED_RECIPIENTS(2.00)[darcy@druid.net,freebsd-virtualization@freebsd.org]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:b580::/32:c]; MV_CASE(0.50)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; HAS_ATTACHMENT(0.00)[]; DMARC_NA(0.00)[punkt.de]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_HAM_LONG(-1.01)[-1.013]; NEURAL_HAM_MEDIUM(-0.68)[-0.675]; TO_DN_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_SPAM_SHORT(0.46)[0.458]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16188, ipnet:2a00:b580::/32, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 09:37:14 -0000 --Apple-Mail=_D80FB223-AB2E-4B54-8905-F9131EDC930A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi all, > Am 20.10.2020 um 11:28 schrieb D'Arcy Cain : >=20 > On 10/20/20 4:36 AM, Patrick M. Hausen wrote: >> It's officially documented here: >> = https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-bridgin= g.html >=20 > I did see that. Does that mean that I don't even need to create = switches at all? What is a switch in this context? I use bridge interfaces to connect = jails via epair and VMs via tap. >> "If the bridge host needs an IP address, set it on the bridge = interface, not on the member interfaces." >=20 > But I don't necessarily need an IP on the bridge itself, right? Depends ;-) If the host has got e.g. em0 with an IP address and you want to make that physical interface part of e.g. bridge0 as well as all the VMs so = they can communicate on the wire ... you *must* move the IP address config from em0 to bridge0 and configure em0 "up". If em0 does not have an IP address on the host and should be used exclusively for VMs, then the bridge does not need an IP address, = either. Still you need to configure em0 "up". And additionally ... - you should disable all hardware acceleration features on the physical = interface - if you are using pf you should move the rule processing from the = members to the bridge like so: sysctl net.link.bridge.pfil_member=3D0 sysctl net.link.bridge.pfil_bridge=3D1 HTH, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Gesch=C3=A4ftsf=C3=BChrer: J=C3=BCrgen Egeling, Daniel Lienert, Fabian = Stein --Apple-Mail=_D80FB223-AB2E-4B54-8905-F9131EDC930A Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+Or8YACgkQkG8u4u3a iVyFjAf/T88YLm/BFeinpFnSV6yM2I1dXYG82Ife51J+E49k/oqQL8fo6I3wayQ4 hQLyuhldm3OiCJqWFzY+64hFdvEZ7GkJQDmiyRQZXpK1hUuJAvizC/xnv9AOfjss UevXItXoSfr+gDSJ1zEzyBYvNpI9EVe9dMsmafWorzBAc38+S8DPPIEn1hLbyMv/ A5oz6baX0P5bWY68mYNjiieeL07tQzbzfCMPXtmObxW5u+rpLd7d+qqnsvjCm0Hr XMm0ETCN9ktDbw02CSeSH0qbaNI17n09eMknejFz61xvhGTJbGSk2myKZ6JoBCw+ /Ywg1OZfGC3Kl4h0i0ONWjOca6zDXQ== =7nL1 -----END PGP SIGNATURE----- --Apple-Mail=_D80FB223-AB2E-4B54-8905-F9131EDC930A-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 11:39:05 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8D8EB42E2AD for ; Tue, 20 Oct 2020 11:39:05 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mail.punkt.de (mail.punkt.de [IPv6:2a00:b580:8000:11:1c6b:7032:35e9:5616]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFs8m5Zmlz4Cp1 for ; Tue, 20 Oct 2020 11:39:04 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from [217.29.46.75] (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.punkt.de (Postfix) with ESMTPSA id 9DF0310B0A; Tue, 20 Oct 2020 13:39:01 +0200 (CEST) From: "Patrick M. Hausen" Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_E71AA62E-07B7-4771-9C33-60105090E42C"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Subject: Re: When is a switch not a switch? Date: Tue, 20 Oct 2020 13:39:00 +0200 In-Reply-To: <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> Cc: freebsd-virtualization@freebsd.org To: D'Arcy Cain References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> X-Mailer: Apple Mail (2.3445.104.17) X-Rspamd-Queue-Id: 4CFs8m5Zmlz4Cp1 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 2a00:b580:8000:11:1c6b:7032:35e9:5616 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-3.61 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:b580::/32:c]; MV_CASE(0.50)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; HAS_ATTACHMENT(0.00)[]; DMARC_NA(0.00)[punkt.de]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.007]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.72)[-0.718]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.98)[-0.984]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16188, ipnet:2a00:b580::/32, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 11:39:05 -0000 --Apple-Mail=_E71AA62E-07B7-4771-9C33-60105090E42C Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi all, > Am 20.10.2020 um 12:50 schrieb D'Arcy Cain : >=20 > On 10/20/20 5:36 AM, Patrick M. Hausen wrote: >>> I did see that. Does that mean that I don't even need to create = switches at all? >> What is a switch in this context? I use bridge interfaces to connect = jails via epair >> and VMs via tap. >=20 > When I started I thought of a switch as analogous to a physical = switch. If I am in an office with one ethernet jack but I have multiple = devices I might connect a switch (or hub) to the jack and plug my = devices into the switch. I don't need to create a separate network for = my office. All of my devices are on the company network. OK, the "switch" interface in FreeBSD is bridge(4). Or to cite Radia Perlman: A bridge is a network device making forwarding decisions based on layer = 2 addresses. A router is a network device making forwarding decisions based on layer = 3 addresses. "Switch" is a marketing term meaning "faster or cheaper than the = competition". > cloned_interfaces=3D"bridge0" > ifconfig_bridge0=3D"a.b.c.d.1 addm bge0 addm switch0 up" > Except that switch0 doesn't get created until vm-bhyve starts so it = probably doesn't exist at that time. What is "switch0"? I suspect it is just a bridge interface that gets = renamed by your VM management software. In that case manually creating bridge0 and all the things we discussed will not get you anywhere. >> If em0 does not have an IP address on the host and should be used >> exclusively for VMs, then the bridge does not need an IP address, = either. >> Still you need to configure em0 "up". >=20 > I can't imagine a scenario like that. You probably always need access = to the host for maintenance. Well, there could be a second hardware interface for host communication = ... And if one of the two is member of the bridge and the other one isn't it = is perfectly valid to plug them into the same broadcast domain and get e.g. 1Gbit/s for the host and 1Gbit/s for all the jails or VMs. >> And additionally ... >> - you should disable all hardware acceleration features on the = physical interface >=20 > Like ASF? Real life example from our environment: ifconfig_igb0=3D"-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag = -vlanhwtso up" cloned_interfaces=3D"bridge0" ifconfig_bridge0_name=3D"inet0" ifconfig_inet0=3D"addm igb0 up" ifconfig_inet0_alias0=3D"inet 1.2.3.4/24" Then we configure iocage to attach the jails to bridge0. In your case you would have to tell your VM management tool to attach = the VM tap interfaces to bridge0 instead of creating its own "switch0" - = which I suspect is a bridge interface in disguise. As you can see above we = rename all our Internet facing interfaces to "inet0" on all hosts. Then there = are more like "mgmt0", "priv0", ... like that. So probably the bridge is renamed = to "switch0". Tell the tool not to do that and use the preconfigured bridge0 instead. Another useful sysctl to get reproduceable static MAC addresses for the = bridge itself accross reboots is: loader.conf: if_bridge_load=3D"YES" sysctl.conf: net.link.bridge.inherit_mac=3D1 HTH, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Gesch=C3=A4ftsf=C3=BChrer: J=C3=BCrgen Egeling, Daniel Lienert, Fabian = Stein --Apple-Mail=_E71AA62E-07B7-4771-9C33-60105090E42C Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+OzFQACgkQkG8u4u3a iVwDrggAk1Ah6hgV4QhtC+Vt+pU8AWuZdazV9gsnIC/KwkJeoPb9VaIIgiFuWeAn qpUAZKQ/talEYkXiTaVqHc5ZZXqibUShjWdh2Bx9swCih/HakFgzxw3Jspwq1gI7 665ZtSC1YPnpbcECZ+/gDvw1z8wVHVgrbpa9WrVm9Wt1omPn2wSeQcHMGmlGShWF 49KfHqVWt/FjzReM1wAOqsDFuHzfltIqQPQV4ptvhbgtvnE3bdCIbEjMba2f7zmx A0tbHWR8SnnW2NnrqWLDHP8V8wZ9a/62J3wuatU3tBXS9jmYVRVytwS0Tvrosart EQmtmq8/rx4uZ1X1vM4at9zhAXEHzA== =gdBj -----END PGP SIGNATURE----- --Apple-Mail=_E71AA62E-07B7-4771-9C33-60105090E42C-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 11:52:29 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AD0A542EA06 for ; Tue, 20 Oct 2020 11:52:29 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mail.punkt.de (mail.punkt.de [IPv6:2a00:b580:8000:11:1c6b:7032:35e9:5616]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFsSF0DtDz4Dy8 for ; Tue, 20 Oct 2020 11:52:28 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from [217.29.46.75] (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.punkt.de (Postfix) with ESMTPSA id 9D70F10C70; Tue, 20 Oct 2020 13:52:27 +0200 (CEST) From: "Patrick M. Hausen" Message-Id: <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> Content-Type: multipart/signed; boundary="Apple-Mail=_1BBA6985-B70D-4CC7-9FC2-EB75D5704DD8"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Subject: Re: When is a switch not a switch? Date: Tue, 20 Oct 2020 13:52:26 +0200 In-Reply-To: Cc: freebsd-virtualization@freebsd.org To: D'Arcy Cain References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> X-Mailer: Apple Mail (2.3445.104.17) X-Rspamd-Queue-Id: 4CFsSF0DtDz4Dy8 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 2a00:b580:8000:11:1c6b:7032:35e9:5616 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-3.62 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:b580::/32:c]; MV_CASE(0.50)[]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[punkt.de]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.007]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.73)[-0.731]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.98)[-0.983]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16188, ipnet:2a00:b580::/32, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 11:52:29 -0000 --Apple-Mail=_1BBA6985-B70D-4CC7-9FC2-EB75D5704DD8 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 A short P.S. I just looked shortly into vm-bhyve. Whatever this tool does with = respect to the "switches", possibly just turning off all the hardware acceleration features for = your network card can solve your problems without further messing around with bridge(4) and = friends. What you need to configure depends on your hardware. I made a table for the various interfaces we use at our place: em: -rxcsum -txcsum -lro -vlanmtu -vlanhwcsum -vlanhwfilter = -vlanhwtag up igb: -rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag = -vlanhwtso up ix: -rxcsum -rxcsum6 -txcsum -txcsum6 -tso -lro -vlanhwtag = -vlanhwtso up ixl: -rxcsum -rxcsum6 -txcsum -txcsum6 -tso -lro -vlanhwtag = -vlanhwtso up bnxt: -rxcsum -rxcsum6 -txcsum -txcsum6 -tso -lro -vlanhwtag = -vlanhwtso -vlanhwfilter up HTH, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Gesch=C3=A4ftsf=C3=BChrer: J=C3=BCrgen Egeling, Daniel Lienert, Fabian = Stein --Apple-Mail=_1BBA6985-B70D-4CC7-9FC2-EB75D5704DD8 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+Oz3oACgkQkG8u4u3a iVwEjQf/ZGMfztrzgZoCZrx2+qb3mzaNzs9Z8vc1G9WxSPP2/GzTrl3RMXDAbGjV jy3qfBSogrVfMrSYwoE4kDRjIktcI6ZS0JnN8lkE6wapc5ccXL7X7scNHppAdiwo btZgZgYvm+kkt0sukubEbZLyUMCT7RFHaZ1sRTOrOAwkoyioj2HCmdYaBAj+Lcp9 NSp1fbZu1VfH3F7UnBMJ4UqKKsQ78RnQi2o8zku+Xf7mfuV4e86On9veU9wInXMu fQ9+crlAA8AGyqoKHqs9rdQkHZDoIMnjQn+S84UHIzId982hP2evZeE3FIKF8Wy0 VagUTOEBbm0OI9xCZMBCIVflDjV/QQ== =zgKn -----END PGP SIGNATURE----- --Apple-Mail=_1BBA6985-B70D-4CC7-9FC2-EB75D5704DD8-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 12:10:56 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CAE8142FBBE for ; Tue, 20 Oct 2020 12:10:56 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFssW47Ylz4Fpm for ; Tue, 20 Oct 2020 12:10:55 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 8611438ACC; Tue, 20 Oct 2020 08:10:52 -0400 (EDT) To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> Date: Tue, 20 Oct 2020 08:10:51 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx" X-Rspamd-Queue-Id: 4CFssW47Ylz4Fpm X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-2.23 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.90)[-0.896]; NEURAL_HAM_LONG(-0.99)[-0.995]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; DMARC_NA(0.00)[druid.net]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.24)[-0.237]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 12:10:56 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx Content-Type: multipart/mixed; boundary="Nmm8MEmRt5QJUJkoInrxkJu5yVYeqPOlE"; protected-headers="v1" From: D'Arcy Cain To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org Message-ID: <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> In-Reply-To: --Nmm8MEmRt5QJUJkoInrxkJu5yVYeqPOlE Content-Type: multipart/mixed; boundary="------------5EBE8E1B42D6D408F7A9525D" Content-Language: en-US This is a multi-part message in MIME format. --------------5EBE8E1B42D6D408F7A9525D Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/20/20 7:39 AM, Patrick M. Hausen wrote: >> When I started I thought of a switch as analogous to a physical switch= =2E If I am in an office with one ethernet jack but I have multiple devi= ces I might connect a switch (or hub) to the jack and plug my devices int= o the switch. I don't need to create a separate network for my office. A= ll of my devices are on the company network. >=20 > OK, the "switch" interface in FreeBSD is bridge(4). Understood. > Or to cite Radia Perlman: >=20 > A bridge is a network device making forwarding decisions based on layer= 2 addresses. > A router is a network device making forwarding decisions based on layer= 3 addresses. > "Switch" is a marketing term meaning "faster or cheaper than the compet= ition". I always thought that a switch was a hub with packet switching to avoid=20 collisions. >> cloned_interfaces=3D"bridge0" >> ifconfig_bridge0=3D"a.b.c.d.1 addm bge0 addm switch0 up" >=20 >> Except that switch0 doesn't get created until vm-bhyve starts so it pr= obably doesn't exist at that time. >=20 > What is "switch0"? I suspect it is just a bridge interface that gets re= named by > your VM management software. In that case manually creating bridge0 > and all the things we discussed will not get you anywhere. So in vm-bhyve I need to change; @@ -3,9 +3,9 @@ cpu=3D2 memory=3D2G network0_type=3D"virtio-net" -network0_switch=3D"public" +network0_switch=3D"bridge0" network1_type=3D"virtio-net" -network1_switch=3D"private" +network1_switch=3D"bridge1" disk0_type=3D"virtio-blk" disk0_name=3D"disk0.img" disk0_dev=3D"sparse-zvol" Or else rename the bridges to "public" and "private". > Real life example from our environment: >=20 > ifconfig_igb0=3D"-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vla= nhwtso up" ifconfig_bge0=3D"-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vlanh= wtso up" > cloned_interfaces=3D"bridge0" cloned_interfaces=3D"bridge0 bridge1" > ifconfig_bridge0_name=3D"inet0" ifconfig _bridge0_name=3D"public" ifconfig _bridge1_name=3D"private" > ifconfig_inet0=3D"addm igb0 up" ifconfig_public=3D"addm bge0 up" ifconfig_private=3D"addm bge1 up" > ifconfig_inet0_alias0=3D"inet 1.2.3.4/24" ifconfig_public_alias0=3D"inet 1.2.3.4/24" ifconfig_private_alias0=3D"192.168.151.4/14" So why alias? Wouldn't "ifconfig_public=3D" work? > Then we configure iocage to attach the jails to bridge0. >=20 > In your case you would have to tell your VM management tool to attach t= he > VM tap interfaces to bridge0 instead of creating its own "switch0" - wh= ich I > suspect is a bridge interface in disguise. As you can see above we rena= me > all our Internet facing interfaces to "inet0" on all hosts. Then there = are more > like "mgmt0", "priv0", ... like that. So probably the bridge is renamed= to "switch0". If I do the above I guess I can keep the names "public" and "private". >=20 > Tell the tool not to do that and use the preconfigured bridge0 instead.= Or public? inet0 in your example? >=20 > Another useful sysctl to get reproduceable static MAC addresses for the= bridge > itself accross reboots is: >=20 > loader.conf: if_bridge_load=3D"YES" > sysctl.conf: net.link.bridge.inherit_mac=3D1 Not sure I need this as long as arp works as it should. Do I really care= =20 what the MAC is? Cheers. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------5EBE8E1B42D6D408F7A9525D-- --Nmm8MEmRt5QJUJkoInrxkJu5yVYeqPOlE-- --QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX47TywUDAAAAAAAKCRA/5DDweYZnXZAW AQC7/IuVAoE76ZLgp/OJyLaFcxKucGak6cvBOCtSGL2hMgEA1X39WPgNMsyYzHRbOW4ia7UPZ1Qi CLxa3T6WTpCr2A4= =2BsT -----END PGP SIGNATURE----- --QO3B3PQM2HdSARNFZ6CcIOW3zcI98BQnx-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 12:35:28 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E4BD64304C6 for ; Tue, 20 Oct 2020 12:35:28 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mail.punkt.de (mail.punkt.de [IPv6:2a00:b580:8000:11:1c6b:7032:35e9:5616]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFtPr33R6z4H7W for ; Tue, 20 Oct 2020 12:35:28 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from [217.29.46.75] (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.punkt.de (Postfix) with ESMTPSA id E6BF510EBF; Tue, 20 Oct 2020 14:35:26 +0200 (CEST) From: "Patrick M. Hausen" Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_7BBBCBFB-692C-4BEB-96A9-692E04966C41"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Subject: Re: When is a switch not a switch? Date: Tue, 20 Oct 2020 14:35:25 +0200 In-Reply-To: <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> Cc: freebsd-virtualization@freebsd.org To: D'Arcy Cain References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> X-Mailer: Apple Mail (2.3445.104.17) X-Rspamd-Queue-Id: 4CFtPr33R6z4H7W X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 2a00:b580:8000:11:1c6b:7032:35e9:5616 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-3.83 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:b580::/32:c]; MV_CASE(0.50)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; HAS_ATTACHMENT(0.00)[]; DMARC_NA(0.00)[punkt.de]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.007]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.94)[-0.941]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.98)[-0.984]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16188, ipnet:2a00:b580::/32, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 12:35:29 -0000 --Apple-Mail=_7BBBCBFB-692C-4BEB-96A9-692E04966C41 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi! > Am 20.10.2020 um 14:10 schrieb D'Arcy Cain : >=20 > On 10/20/20 7:39 AM, Patrick M. Hausen wrote: >>> When I started I thought of a switch as analogous to a physical = switch. If I am in an office with one ethernet jack but I have multiple = devices I might connect a switch (or hub) to the jack and plug my = devices into the switch. I don't need to create a separate network for = my office. All of my devices are on the company network. >> OK, the "switch" interface in FreeBSD is bridge(4). >=20 > Understood. >=20 >> Or to cite Radia Perlman: >> A bridge is a network device making forwarding decisions based on = layer 2 addresses. >> A router is a network device making forwarding decisions based on = layer 3 addresses. >> "Switch" is a marketing term meaning "faster or cheaper than the = competition". >=20 > I always thought that a switch was a hub with packet switching to = avoid collisions. That is a bridge. A switch simply is a multiport bridge. And a layer 3 = switch is a router. > Or else rename the bridges to "public" and "private". Yep, probably. >> ifconfig_inet0=3D"addm igb0 up" > ifconfig_public=3D"addm bge0 up" > ifconfig_private=3D"addm bge1 up" >=20 >> ifconfig_inet0_alias0=3D"inet 1.2.3.4/24" > ifconfig_public_alias0=3D"inet 1.2.3.4/24" > ifconfig_private_alias0=3D"192.168.151.4/14" >=20 > So why alias? Wouldn't "ifconfig_public=3D" work? We already have ifconfig_public=3D"addm bge0 up" Adding ifconfig_public=3D"inet 1.2.3.4/24" on another line would overwrite the first one. These are just variable assignments not executable code. You cannot have more than one ifconfig_public line. If you need more than one they have to be named ifconfig_public ifconfig_public_alias0 ifconfig_public_alias1 ... Execution stops at the first undefined one, so no gaps, either. > Not sure I need this as long as arp works as it should. Do I really = care what the MAC is? Well, the ARP timeouts specifically of Cisco gear can be enervatingly = long so hosts are not reachable after reboot for minutes ... these settings fix that. HTH, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Gesch=C3=A4ftsf=C3=BChrer: J=C3=BCrgen Egeling, Daniel Lienert, Fabian = Stein --Apple-Mail=_7BBBCBFB-692C-4BEB-96A9-692E04966C41 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+O2Y0ACgkQkG8u4u3a iVxAawf6Avy0WuiR8v3nSAk/5/9/8RE1aaFG5yKyoZe9402/fu0+77c8kdbR5zOy yIcHJxX1hLy+YCo484WSOC9ac3upkZyySBFFhzLjvUGObAzVRtNpKwfE3UO85Yst om2fS3NRDJIYWx5Z3HzGsxbtDyFVYXA1uodq8dwq1rCnVaZtE9gTuU5XpPSt/mdB /OQsFfvmQVfbTLFOrCqqZ8ExR5Zi/sU7KQuMum825K5u9H8TjiPmeg5o9uk+YJyZ blQNdwRiSAaqgEVE5EgR/kPq7jIjCUDc2nef5BkpXwJ1+KORdHAOj7jLnOOkYhOI uJebS8wgkDZZtfr5XFbqh0BMiW+cfQ== =3P8P -----END PGP SIGNATURE----- --Apple-Mail=_7BBBCBFB-692C-4BEB-96A9-692E04966C41-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 12:45:01 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9B5BE43055D for ; Tue, 20 Oct 2020 12:45:01 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFtcr4WBYz4HFH for ; Tue, 20 Oct 2020 12:45:00 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-io1-xd32.google.com with SMTP id q25so3023982ioh.4 for ; Tue, 20 Oct 2020 05:45:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=A/q2UrZ6RoCIwP9CDhw6hYWrsaHMilvqdCmvf3wQIq0=; b=WJH55OUc7lqOK+yWZCFS+op3TtashFSWIwTmtAlTS4OzoHtNSbpqX3yLfjbk7iqr4v Es5Kaq5JUZp9G/dQLV2bKUmDVzWtk0X0raF+OeRf53iHq1oVjJDVQ3B89cMz7KbsnFyK k4i1zmXm5Aq/6wGPPYblgyFdvIaMwKcRr6cu8zebprflYdoaGG5oX0oeMw/bOXy1cb79 rYR+ZGt+HobxkitgS7V8WqpwDwWkXp1GbQhv1QUCv+XpoC/lDZr9KUWS1X5h5oHNpFQr bDLV3idKdMfnw41VqAs0HzG4QqBYiXn4CfayGnuYgj07HlHVBBPmlkHlMXzJOcWH7xYc vPZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=A/q2UrZ6RoCIwP9CDhw6hYWrsaHMilvqdCmvf3wQIq0=; b=TUGGVosReudcnrw7JNNkBHD6vO11Gkrct3deS4ZAh8DTygvj+o8g5L9GKFV20wTO9U yVB/LQyCI7EliuBLbe/H0jfvHWYpQj3yRrkpSPWwFPDdo4igVjlCL0SQwtRMACZfTK1r C5h4rBT3cjhECo1/gXbcvskwGauT4diAjVSIJ37q8ax1113d8YTkxmrqolFixBUigzd5 OSpQdFYHz55W7sp2F46gW8TDmuXmIp3+C0jFOjpu7XXateXA9r2TmkwXLdruToxvn146 23hU+f0ytMuRC0lb9OeXnYkOZ3sNRqDBWhL7CbZYKpJiosOUCjFdsKgPEvgj54zlGO59 6GPw== X-Gm-Message-State: AOAM530evjGNFnUqaw5dZe1YaxqC2pa2mvhzhXuK+dzWTz8TFBSaCocV iUHXDWe0oJ+Lt9U/ge3HGzuYdiWwMZ+ILW0G X-Google-Smtp-Source: ABdhPJxJ3XUUkBrPAN3LisHexvU8ADPSrf8oLWHqcoGte1TWsFbimLplnBFxWNhobipzej2dHcLSUg== X-Received: by 2002:a5e:c90c:: with SMTP id z12mr1921406iol.150.1603197898090; Tue, 20 Oct 2020 05:44:58 -0700 (PDT) Received: from mutt-hbsd (pool-100-16-222-53.bltmmd.fios.verizon.net. [100.16.222.53]) by smtp.gmail.com with ESMTPSA id p198sm1501239iod.15.2020.10.20.05.44.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Oct 2020 05:44:57 -0700 (PDT) Date: Tue, 20 Oct 2020 08:44:56 -0400 From: Shawn Webb To: D'Arcy Cain Cc: freebsd-virtualization@freebsd.org Subject: Re: When is a switch not a switch? Message-ID: <20201020124456.kyvlhus3qj4o7gtp@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 13.0-CURRENT-HBSD FreeBSD 13.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0xFF2E67A277F8E1FA References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="e3lyc3bdulbpnglp" Content-Disposition: inline In-Reply-To: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> X-Rspamd-Queue-Id: 4CFtcr4WBYz4HFH X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=WJH55OUc; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::d32 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-3.47 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.37)[-0.373]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RECEIVED_SPAMHAUS_PBL(0.00)[100.16.222.53:received]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.992]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.007]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[hardenedbsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d32:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 12:45:01 -0000 --e3lyc3bdulbpnglp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 19, 2020 at 10:02:17PM -0400, D'Arcy Cain wrote: > I am using bhyve with vm-bhyve, I am trying to set up a virtual network > with multiple hosts. The idea is that a VM would be on the same virtual > network no matter which actual host it is on. >=20 > Say I have a public network a.b.c.0/24. I thought I could create a switch > on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and > a.b.c.101. The idea would be that the VMs would appear on the real netwo= rk. > Then the 101 VM could migrate to a.b.c.2 and still be accessible. I > envisioned some sort of proxy arp would happen so that every VM would sim= ply > announce itself wherever it was. >=20 > This did seem to work in that I could ping from the VM: >=20 > # ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=3D0 ttl=3D114 time=3D1.734 ms >=20 > Even IPV6: >=20 > # ping6 2605:2600:1001::4b > PING6(56=3D40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D0 hlim=3D64 time=3D0.960 ms > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D1 hlim=3D64 time=3D0.415 ms >=20 > However TCP doesn't work. In fact, I could only ping by IP because the > system couldn't connect to the DNS server, to get an address even though = it > could ping it. >=20 > I guess my first question is does this seem doable? If so, what am I > missing? Is it possible that a bhyve switch is more like a router? >=20 > Thanks. >=20 > --=20 > D'Arcy J.M. Cain | Democracy is three wolves > http://www.druid.net/darcy/ | and a sheep voting on > +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. > IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net >=20 > Disclaimer: By sending an email to ANY of my addresses you > are agreeing that: >=20 > 1. I am by definition, "the intended recipient". > 2. All information in the email is mine to do with as I see > fit and make such financial profit, political mileage, or > good joke as it lends itself to. In particular, I may quote > it where I please. > 3. I may take the contents as representing the views of > your company if I so wish. > 4. This overrides any disclaimer or statement of > confidentiality that may be included or implied in > your message. I usually configure my bridgeN device to have an IP and subnet that I know won't be on any of the physical networks I care about. I'll then add only the tapN..M devices that the bhyve VMs will use to that bridgeN. I'll then use pf to NAT from that private network on bridgeN to the real world. =3D=3D=3D=3D BEGIN rc.conf =3D=3D=3D=3D cloned_interfaces=3D"bridge0 tap0 tap1" ifconfig_bridge0=3D"inet 192.168.254.1 subnet mask 255.255.255.0" ifconfig_bridge0=3D"${ifconfig_bridge0} addm tap0 addm tap1" =3D=3D=3D=3D END rc.conf =3D=3D=3D=3D =3D=3D=3D=3D BEGIN pf.conf =3D=3D=3D=3D table counters { \ 192.168.254.0/24 \ } scrub in all nat on em0 from {} to any -> (em0) nat on wlan0 from {} to any -> (wlan0) pass in all pass out all =3D=3D=3D=3D END pf.conf =3D=3D=3D=3D Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD GPG Key ID: 0xFF2E67A277F8E1FA GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Sha= wn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --e3lyc3bdulbpnglp Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAl+O28UACgkQ/y5nonf4 4fpp9BAAmvrWeKaI9Jrbp7wZ3pcG7x6vlwF0Mp08FqOhbz+mKsMbbPibr7WTWy/x 8CRNo4EwgLD9spWyq+g6YFIvQ0KmIB4kFgMJ9913Ztr9NvaYa2D7DQ/5a/3/31yg MWCpbq3jbwD6ej9SrclURwMcM8vAEUqJOZh20FQgTJ6k6I7TncmBqEjCa0FKJuYP 8kDkenEjk8hk8rmhQTVx4ATyqWcWDRu7vw959MXO1rdenJn+caXRU9eAhzXU8NHd M66fJ8Qyl+EhM/camJm2hFBwJtZ7+BjmW5uXBzjiGTgSOeLRSi1vEQ3FDD1f0in7 upX7LCdPAEJGclWZHncu/H+hxqUx51byqaGIYx8SZC1j9jjk5sIFUDIyntjWHlmx 2e6FKaRDoXeMtsbvotcVGBZZ/20rcf5cfMfAoajtJO6YBDX88mAb2EOdkRz+Ssxi ZacH+OmmtQmfXyURxp89rtsmGHkpYhk3ZaAr4jkVDQDdk5Q+nkpMmxzgGFQ+tQ45 XN2mSKujMyIA76SAfdUgid/hqFoZyK2cJuJrGfmrv0yHKX/D3RjB/86jS3FpNj8O +otD32fxi5bsBGoHn3HSOLpRPAMcMGOJxPD0j5TNH6Ge2dDMc9KoVQDNVvmL1fbJ bwI994sMaQzKjmNjxALP4MWH1e5BmYsnQPMUUmJwgfRt8qmtPec= =+sm/ -----END PGP SIGNATURE----- --e3lyc3bdulbpnglp-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 13:00:32 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 958BF430AE0 for ; Tue, 20 Oct 2020 13:00:32 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFtyl47F4z4Hdw for ; Tue, 20 Oct 2020 13:00:31 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id BD0DE38C3D; Tue, 20 Oct 2020 09:00:29 -0400 (EDT) To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <2a0f2d6e-6a45-ee2a-4170-0e6ea5bd0225@druid.net> Date: Tue, 20 Oct 2020 09:00:29 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="fJMWZozn9ihuOi93JTrTlnGIX4tx2cDfs" X-Rspamd-Queue-Id: 4CFtyl47F4z4Hdw X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-1.95 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.90)[-0.896]; NEURAL_HAM_LONG(-0.99)[-0.995]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; HAS_ATTACHMENT(0.00)[]; DMARC_NA(0.00)[druid.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.04)[0.043]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 13:00:32 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --fJMWZozn9ihuOi93JTrTlnGIX4tx2cDfs Content-Type: multipart/mixed; boundary="VbfbP71EBxZXwprLf6o0U2mIygoOaxT9Y"; protected-headers="v1" From: D'Arcy Cain To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org Message-ID: <2a0f2d6e-6a45-ee2a-4170-0e6ea5bd0225@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <309e9df2-51e0-ff71-15ef-e42d0418f193@druid.net> In-Reply-To: --VbfbP71EBxZXwprLf6o0U2mIygoOaxT9Y Content-Type: multipart/mixed; boundary="------------2323F034BE323CD794BB1F91" Content-Language: en-US This is a multi-part message in MIME format. --------------2323F034BE323CD794BB1F91 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/20/20 8:35 AM, Patrick M. Hausen wrote: >> So why alias? Wouldn't "ifconfig_public=3D" work? >=20 > We already have >=20 > ifconfig_public=3D"addm bge0 up" >=20 > Adding >=20 > ifconfig_public=3D"inet 1.2.3.4/24" >=20 > on another line would overwrite the first one. These are just > variable assignments not executable code. You cannot have > more than one Doh! Of course. I would have known that after the next coffee. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------2323F034BE323CD794BB1F91-- --VbfbP71EBxZXwprLf6o0U2mIygoOaxT9Y-- --fJMWZozn9ihuOi93JTrTlnGIX4tx2cDfs Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX47fbQUDAAAAAAAKCRA/5DDweYZnXYoS AQCrhWLoCCEU/IEDjYhNQIi9rRSmoarLr79YSJ9OuEDI7AEA2BeYBMHeJGpDyGygLkh4m+YyfrdV ctHAS66P1XASqAQ= =wpRr -----END PGP SIGNATURE----- --fJMWZozn9ihuOi93JTrTlnGIX4tx2cDfs-- From owner-freebsd-virtualization@freebsd.org Tue Oct 20 16:34:47 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7CFF84357C1 for ; Tue, 20 Oct 2020 16:34:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (mailman.nyi.freebsd.org [IPv6:2610:1c1:1:606c::50:13]) by mx1.freebsd.org (Postfix) with ESMTP id 4CFzjz2pnXz4VPn for ; Tue, 20 Oct 2020 16:34:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id 607EC4354D7; Tue, 20 Oct 2020 16:34:47 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 604C94356F0 for ; Tue, 20 Oct 2020 16:34:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFzjz1znWz4VVC for ; Tue, 20 Oct 2020 16:34:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2816317D67 for ; Tue, 20 Oct 2020 16:34:47 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 09KGYlPr078940 for ; Tue, 20 Oct 2020 16:34:47 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 09KGYlpv078939 for virtualization@FreeBSD.org; Tue, 20 Oct 2020 16:34:47 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 203874] [patch] MSI/MSI-X interrupts don't work in VMware virtual machines Date: Tue, 20 Oct 2020 16:34:46 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: kern X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: patch X-Bugzilla-Severity: Affects Many People X-Bugzilla-Who: registrazioni@kiokoman.eu.org X-Bugzilla-Status: Closed X-Bugzilla-Resolution: FIXED X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 16:34:47 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D203874 Manuel P. changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |registrazioni@kiokoman.eu.o | |rg --- Comment #32 from Manuel P. --- I only see it in the master branch on FreeBSD i think that it would be useful to see it also for FreeBSD 12 if possible Thanks, Manuel --=20 You are receiving this mail because: You are the assignee for the bug.= From owner-freebsd-virtualization@freebsd.org Tue Oct 20 23:51:44 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A638F43D3F9 for ; Tue, 20 Oct 2020 23:51:44 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG9Q74gRdz4ssw for ; Tue, 20 Oct 2020 23:51:43 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-wr1-f51.google.com with SMTP id n15so407469wrq.2 for ; Tue, 20 Oct 2020 16:51:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=888P/dW1GbixsbZuO9QGhfX+89ARGrScYBh0kUpzxK8=; b=AI9kVfHWl0yUBlaQ+1tHuoETXbml7HfJ/yL+2kHhGWeg8rY59U4Oc38N2MzeS3n5wP 2p42d8e75wuoP/yG158guXI2CPZ7KOgSVkJKgju5AXniJkCAn4eNz3rMsmEEdml/tDtv TYwOHewxuH3hVDNrkAoxkCVjUPXdwlyAPgtJ+qi3WID9fgsV4Q1CJ1Ma0kcPxtsZ1V2N bWZNx7861yApCJFGrgdGeJz632McqoQPo/lbLZsCfPl/6HvmWtZUKBTKHR6Jcq3vo1xr tovLvM3ojD/EYTKuRFlZJHh28qgQy6j3QFpeWRp7AGUdSZgpn8Kz/qP06VUUEgYh8O1c 49fw== X-Gm-Message-State: AOAM5333soqY6BLlAEBlHvQm3zfXp49cbBjPNcvtVq5CCEu+EzQttzL5 kKcQWxJ9o67BLGeU7OfvPSfGLi2f9CWZc7KjM1vHz7zS X-Google-Smtp-Source: ABdhPJyvXOvl+dOUJxdJ/qevrOnBwAmBPWfFw+kdmNvTl6gKX2kglUx1S14VtAlq49Uch9jXBAIZpokPW3QYUDgbwWk= X-Received: by 2002:a1c:6643:: with SMTP id a64mr453257wmc.142.1603237901737; Tue, 20 Oct 2020 16:51:41 -0700 (PDT) MIME-Version: 1.0 References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> In-Reply-To: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> From: Jason Tubnor Date: Wed, 21 Oct 2020 10:51:30 +1100 Message-ID: Subject: Re: When is a switch not a switch? To: "D'Arcy Cain" Cc: "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 4CG9Q74gRdz4ssw X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [0.80 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; URI_COUNT_ODD(1.00)[1]; NEURAL_HAM_SHORT(-0.64)[-0.638]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.67)[-0.667]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.90)[-0.900]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[tubnor.net]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.51:from]; HTTP_TO_IP(1.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.51:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 23:51:44 -0000 Hi, On Tue, 20 Oct 2020 at 13:02, D'Arcy Cain wrote: > I am using bhyve with vm-bhyve, I am trying to set up a virtual network > with multiple hosts. The idea is that a VM would be on the same virtual > network no matter which actual host it is on. > > Say I have a public network a.b.c.0/24. I thought I could create a switch > on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and > a.b.c.101. The idea would be that the VMs would appear on the real > network. > Then the 101 VM could migrate to a.b.c.2 and still be accessible. I > envisioned some sort of proxy arp would happen so that every VM would > simply > announce itself wherever it was. > It looks like you are over complicating this. When using vm-bhyve, as long as each host has the same vswitch (bridge) then the tap will automagically be inserted correctly on guest startup (as long as the conf file follows your guest storage). Let vm-bhyve manage bridge creation. Only use /etc/rc.conf to bring up the interface. If you are running > 11.4 then you must turn LRO off (-lro) when you bring up the interface. The other settings in this thread can be left on. The problem you are experiencing is packet fragmentation that the guest has to deal with because LRO is enabled (off by default in 11, enabled in 12 and above). LRO should be disabled automatically when an interface (or child VLAN) is added to a bridge. I have tried to get the network guys to fix this but no such luck. > This did seem to work in that I could ping from the VM: > > # ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=114 time=1.734 ms > > Even IPV6: > > # ping6 2605:2600:1001::4b > PING6(56=40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b > 16 bytes from 2605:2600:1001::4b, icmp_seq=0 hlim=64 time=0.960 ms > 16 bytes from 2605:2600:1001::4b, icmp_seq=1 hlim=64 time=0.415 ms > > However TCP doesn't work. In fact, I could only ping by IP because the > system couldn't connect to the DNS server, to get an address even though > it > could ping it. > > I guess my first question is does this seem doable? If so, what am I > missing? Is it possible that a bhyve switch is more like a router? From owner-freebsd-virtualization@freebsd.org Wed Oct 21 19:35:59 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 23AA642BA96 for ; Wed, 21 Oct 2020 19:35:59 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGghY5YY6z3SMN for ; Wed, 21 Oct 2020 19:35:57 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 09LJZejq064740 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Wed, 21 Oct 2020 12:35:40 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 09LJZdsf064739; Wed, 21 Oct 2020 12:35:39 -0700 (PDT) (envelope-from jmg) Date: Wed, 21 Oct 2020 12:35:39 -0700 From: John-Mark Gurney To: "D'Arcy Cain" Cc: "Patrick M. Hausen" , freebsd-virtualization@freebsd.org Subject: Re: When is a switch not a switch? Message-ID: <20201021193539.GG8272@funkthat.com> Mail-Followup-To: D'Arcy Cain , "Patrick M. Hausen" , freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="MfFXiAuoTsnnDAfZ" Content-Disposition: inline In-Reply-To: <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Wed, 21 Oct 2020 12:35:40 -0700 (PDT) X-Rspamd-Queue-Id: 4CGghY5YY6z3SMN X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-1.82 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-0.94)[-0.944]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.21)[-0.213]; NEURAL_HAM_MEDIUM(-0.76)[-0.764]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_SPF_NA(0.00)[no SPF record]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Oct 2020 19:35:59 -0000 --MfFXiAuoTsnnDAfZ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable D'Arcy Cain wrote this message on Tue, Oct 20, 2020 at 05:28 -0400: > On 10/20/20 4:36 AM, Patrick M. Hausen wrote: > > It's officially documented here: > > https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/network-brid= ging.html >=20 > I did see that. Does that mean that I don't even need to create switches= at=20 > all? >=20 > > "If the bridge host needs an IP address, set it on the bridge interface= , not on the member interfaces." >=20 > But I don't necessarily need an IP on the bridge itself, right? I can't say w/o more info in exactly how your system is setup... If you could provide more concrete information about what interfaces you are using, and what IPs are configured on what interfaces (or VMs), that'd be helpful. --=20 John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --MfFXiAuoTsnnDAfZ Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJfkI2KXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2MEI1RTRGMTNDNzYyMDZDNjEyMDBCNjAy MDVGMEIzM0REMDA2QURBAAoJECBfCzPdAGrakJYQAJyOeL1yvUVVGRgNm1KTy8qk wKTwp4pm9i5jVvoU4fIj/rrMsmXslSy/mObV+EEJZarQCid4r4g4n9sFEc0e4lYd L/cykoL8XFCLsWAiWQ6ImBb9yKvbGE28YyyjHHlA+f4o9OlCp8a4i1QvA4V5UQdb PsVOXB6RvsO4Zsc5sA8c9Qp91rWA8VsGLACQ2ZbW7gRCt1rxzFARhwo4ejQTaY7i coy2Xrp5rAdRRZFqbWZzy0UobbDZ5aQKadZsahoZZM7sffritYc3tTntwACDw3bI d51htTjAoH2CH+telhklJxzgnIi9ETnFsTRJPCjkF/qCce0XoN16TLdZ75C0a3md MPrIQq3nCEoRGCvGQjMoZurm5q0srgYDNxTzu5eMepywxcTkZVLauJLwCyZkg4wp zx55xQgdnU+iVOeW9Z3SLOppEM8GH3y6Gb4TAWjL6vgKn9z79aEd+3L6thTjYG/Z gPldm40a1cqgExAxk4x4NPGMZZOZcyO4Uwo71MKeQ6Gtt93mtwJmSf+tkknMHIxF gtwMCrElgjZz/1ost4gInAyqHDz1rK0H869oiEpU95dbyv9fR2ljL1Gu0vxZvA64 uUx31CaoQLrgY8aYEkMUovgmlPmGWI+L0R48kOYValgwA6IWYeOLUO+Essd+5hCL f1jrslIMTJ6UVnhF8q80 =hol4 -----END PGP SIGNATURE----- --MfFXiAuoTsnnDAfZ-- From owner-freebsd-virtualization@freebsd.org Thu Oct 22 02:47:57 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 491D943B86D for ; Thu, 22 Oct 2020 02:47:57 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CGsGz6KJxz4DZm for ; Thu, 22 Oct 2020 02:47:55 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 9FE733CCE4; Wed, 21 Oct 2020 22:47:47 -0400 (EDT) To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> Date: Wed, 21 Oct 2020 22:47:46 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="9VWmIojpVNM5X2IX12FC5MMzT25Cz4Atj" X-Rspamd-Queue-Id: 4CGsGz6KJxz4DZm X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-1.65 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; NEURAL_HAM_MEDIUM(-0.69)[-0.685]; NEURAL_HAM_LONG(-0.98)[-0.978]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; DMARC_NA(0.00)[druid.net]; AUTH_NA(1.00)[]; NEURAL_SPAM_SHORT(0.12)[0.116]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 02:47:57 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --9VWmIojpVNM5X2IX12FC5MMzT25Cz4Atj Content-Type: multipart/mixed; boundary="yd7Jtb4AgJcLURycmM9kV28eMgqUwZlUx"; protected-headers="v1" From: D'Arcy Cain To: "Patrick M. Hausen" Cc: freebsd-virtualization@freebsd.org Message-ID: <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> In-Reply-To: <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> --yd7Jtb4AgJcLURycmM9kV28eMgqUwZlUx Content-Type: multipart/mixed; boundary="------------2417412F780BCE09EB82A2CF" Content-Language: en-US This is a multi-part message in MIME format. --------------2417412F780BCE09EB82A2CF Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/20/20 7:52 AM, Patrick M. Hausen wrote: > What you need to configure depends on your hardware. > I made a table for the various interfaces we use at our place: >=20 > em: -rxcsum -txcsum -lro -vlanmtu -vlanhwcsum -vlanhwfilter -vlanhwtag= up > igb: -rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag -vlanhwtso up I have bge(4) cards. This seems to be the correct one for me. It still doesn't work. Here is the relevant lines from my rc.conf. I ma= y=20 as well use the actual values. This is verbatim. ifconfig_eth0=3D"-rxcsum -rxcsum6 -txcsum -txcsum6 -tso -vlanhwtag=20 -vlanhwtso up" cloned_interfaces=3D"bridge0 bridge1" ifconfig_bridge0_name=3D"public" ifconfig_public=3D"addm eth0 up" ifconfig_public_alias0=3D"inet 0x629e8b${me}/27" ifconfig_public_alias0_ipv6=3D"inet6 2605:2600:1001::${me}/64" defaultrouter=3D"98.158.139.94" ipv6_defaultrouter=3D2605:2600:1001::1 Note that "me" is set to the hex value of the last octet, 65 in this case= =2E Here is what the interfaces look like: eth0: flags=3D8943 metric= 0=20 mtu 1500 options=3D80088 ether 14:02:ec:31:60:d0 media: Ethernet autoselect (1000baseT ) status: active nd6 options=3D29 public: flags=3D8843 metric 0 mtu= 1500 ether 02:9d:b2:b8:78:00 inet 98.158.139.65 netmask 0xffffffe0 broadcast 98.158.139.95 id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: eth0 flags=3D143 ifmaxaddr 0 port 1 priority 128 path cost 55 groups: bridge nd6 options=3D9 tap0: flags=3D8843 metric 0 mtu 1= 500 description: vmnet-BASE-0-public options=3D80000 ether 00:bd:5f:56:f8:00 groups: tap vm-port media: Ethernet autoselect status: active nd6 options=3D29 Opened by PID 3679 In the VM I set the IP to 71 on the same network. Here is what that look= s like: vtnet0: flags=3D8943 metr= ic 0=20 mtu 1500 options=3D80028 ether 22:22:22:22:22:00 inet 98.158.139.71 netmask 0xffffffe0 broadcast 98.158.139.95 media: Ethernet 10Gbase-T status: active nd6 options=3D29 Everything looks correct but I can't even ping between the host and the V= M. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------2417412F780BCE09EB82A2CF-- --yd7Jtb4AgJcLURycmM9kV28eMgqUwZlUx-- --9VWmIojpVNM5X2IX12FC5MMzT25Cz4Atj Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX5Dy0gUDAAAAAAAKCRA/5DDweYZnXcj7 AQDFzKjc7jNBPjQIuy/Z5sijlKJgg4xUazq4vugZR+sDhQEAnuXV1aknXHs8IZr9Ws37gPrHiJZ5 uVS4J4is5xT5AAM= =n+GF -----END PGP SIGNATURE----- --9VWmIojpVNM5X2IX12FC5MMzT25Cz4Atj-- From owner-freebsd-virtualization@freebsd.org Thu Oct 22 03:16:14 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7197543C3C0 for ; Thu, 22 Oct 2020 03:16:14 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGsvd4cVQz4GGL for ; Thu, 22 Oct 2020 03:16:13 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-wr1-f42.google.com with SMTP id g12so162023wrp.10 for ; Wed, 21 Oct 2020 20:16:13 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=5EEqDDsCW+4fyGnmPcvjs99vb0LcNOt3DSvzBzQ/MDw=; b=Orj5wp3hzX0CXu6ypTig3qpdWPLKoZdmTMIFph1K0uhvRBIYa/DDKmL+ezkvTUATnz 04tquQDtKmvkYY2cJnQhL3sibW+mf+zOJMklMvK8CpdSQjYBrjtqydxh/J2f675Z2r+Z uhVtE3Y6gHhJdNSU0PNeCgZ9FuDJBRMFRaDvd7Oa6AkJ9W0bPrCfMJms9TXMycUB2WSP dAaWByL6PzLjggnF4FD0k/WNtO1wzH9kf0Yc1UPa/PC15otMPjvW7KswEHQpNOAcJsIO yy2fcQlcAz0ppNDsgSkuo6iCSgsFPFd0c6ePGwpuYpOfePfrzsNTq/qWARZHJn8o4ASo K0mw== X-Gm-Message-State: AOAM533BfRfKQ7y+JVM13airlW+PI9fbNGlLv5DMi+DSmzdto8op9O7d B7bZMnBY1LKWhzLXmGQobpIuIm8MtiWm+sSoj27DAMgYkKI= X-Google-Smtp-Source: ABdhPJzSuFTdcHpUCNoyAIZCN8B37sQcI1bij3kt/oRRKjPgO4rn3ZZqt8axEphkgAe4DM57PmP5VqTrDNYSR0TPjs0= X-Received: by 2002:adf:d849:: with SMTP id k9mr308382wrl.332.1603336571782; Wed, 21 Oct 2020 20:16:11 -0700 (PDT) MIME-Version: 1.0 References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> In-Reply-To: <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> From: Jason Tubnor Date: Thu, 22 Oct 2020 14:16:00 +1100 Message-ID: Subject: Re: When is a switch not a switch? To: "D'Arcy Cain" Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 4CGsvd4cVQz4GGL X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.221.42 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [-0.89 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; ARC_NA(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-0.97)[-0.965]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.02)[0.021]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.42:from]; NEURAL_HAM_MEDIUM(-0.95)[-0.948]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; DMARC_NA(0.00)[tubnor.net]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.42:from]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 03:16:14 -0000 On Thu, 22 Oct 2020 at 13:47, D'Arcy Cain wrote: > > Here is what the interfaces look like: > > eth0: flags=8943 metric 0 > mtu 1500 > options=80088 > ether 14:02:ec:31:60:d0 > media: Ethernet autoselect (1000baseT ) > status: active > nd6 options=29 > public: flags=8843 metric 0 mtu > 1500 > ether 02:9d:b2:b8:78:00 > inet 98.158.139.65 netmask 0xffffffe0 broadcast 98.158.139.95 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: eth0 flags=143 > ifmaxaddr 0 port 1 priority 128 path cost 55 > groups: bridge > nd6 options=9 > tap0: flags=8843 metric 0 mtu 1500 > description: vmnet-BASE-0-public > options=80000 > ether 00:bd:5f:56:f8:00 > groups: tap vm-port > media: Ethernet autoselect > status: active > nd6 options=29 > Opened by PID 3679 > tap0 is not a member of public so has no way to get to eth0 or public. I'd avoid creating the 'public' bridge and let vm-bhyve create it. Something like: vm switch create public vm switch add public eth0 Then for the guest: vm add -d network -s public If you have IPv4/v6 addresses on eth0, you will be able to ping those from the guest. Cheers, Jason. In the VM I set the IP to 71 on the same network. Here is what that looks > like: > > vtnet0: flags=8943 metric > 0 > mtu 1500 > options=80028 > ether 22:22:22:22:22:00 > inet 98.158.139.71 netmask 0xffffffe0 broadcast 98.158.139.95 > media: Ethernet 10Gbase-T > status: active > nd6 options=29 > > Everything looks correct but I can't even ping between the host and the VM. From owner-freebsd-virtualization@freebsd.org Thu Oct 22 07:56:44 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 491E0441E03 for ; Thu, 22 Oct 2020 07:56:44 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from mail.punkt.de (mail.punkt.de [IPv6:2a00:b580:8000:11:1c6b:7032:35e9:5616]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CH07H1n46z4W6C for ; Thu, 22 Oct 2020 07:56:43 +0000 (UTC) (envelope-from hausen@punkt.de) Received: from [217.29.46.74] (kagate.punkt.de [217.29.33.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.punkt.de (Postfix) with ESMTPSA id B8559149B1; Thu, 22 Oct 2020 09:56:40 +0200 (CEST) From: "Patrick M. Hausen" Message-Id: Content-Type: multipart/signed; boundary="Apple-Mail=_A83E54F4-A6CC-4CAB-ADD4-1DE2A8FE37F2"; protocol="application/pgp-signature"; micalg=pgp-sha256 Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.17\)) Subject: Re: When is a switch not a switch? Date: Thu, 22 Oct 2020 09:56:39 +0200 In-Reply-To: <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> Cc: freebsd-virtualization@freebsd.org To: D'Arcy Cain References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> X-Mailer: Apple Mail (2.3445.104.17) X-Rspamd-Queue-Id: 4CH07H1n46z4W6C X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of hausen@punkt.de designates 2a00:b580:8000:11:1c6b:7032:35e9:5616 as permitted sender) smtp.mailfrom=hausen@punkt.de X-Spamd-Result: default: False [-3.28 / 15.00]; RCVD_TLS_ALL(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:b580::/32]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; HAS_ATTACHMENT(0.00)[]; ARC_NA(0.00)[]; DMARC_NA(0.00)[punkt.de]; NEURAL_HAM_LONG(-1.00)[-0.998]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.39)[-0.386]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_MEDIUM(-0.99)[-0.995]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16188, ipnet:2a00:b580::/32, country:DE]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 07:56:44 -0000 --Apple-Mail=_A83E54F4-A6CC-4CAB-ADD4-1DE2A8FE37F2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Hi! > Am 22.10.2020 um 04:47 schrieb D'Arcy Cain : > public: flags=3D8843 metric 0 = mtu 1500 > ether 02:9d:b2:b8:78:00 > inet 98.158.139.65 netmask 0xffffffe0 broadcast 98.158.139.95 > id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200 > root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > member: eth0 flags=3D143 > ifmaxaddr 0 port 1 priority 128 path cost 55 > groups: bridge > nd6 options=3D9 > tap0: [...] tap0 is not a member of your bridge. With the VM running you can try ifconfig public addm tap0 and check if that changes things. Then go back to the drawing board and probably let vm-bhyve manage all that stuff. Just make sure to configure your physical interface with the hardware acceleration features disabled. You *can* put the IP address on the physical interface and have vm-bhyve create the bridge. I honestly don't know why the documentation explicitly states that you should not. FreeNAS has been running like = this for years and only supports the "correct" configuration since 11.3 or = so. On the other hand coming from Cisco and friends putting the address on the topmost layer 3 interface does make perfect sense to me - so e.g. on a Cisco switch you have physical ports that are members of a VLAN and if you run anything layer 3 on that box, of course the address goes on the VLAN, not the port ... But give vm-bhyve a spin with the address on the physical. Or use two different physical interfaces - one for the host, one for the "public" = bridge. HTH, Patrick -- punkt.de GmbH Patrick M. Hausen .infrastructure Kaiserallee 13a 76133 Karlsruhe Tel. +49 721 9109500 https://infrastructure.punkt.de info@punkt.de AG Mannheim 108285 Gesch=C3=A4ftsf=C3=BChrer: J=C3=BCrgen Egeling, Daniel Lienert, Fabian = Stein --Apple-Mail=_A83E54F4-A6CC-4CAB-ADD4-1DE2A8FE37F2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEgzqrjO/mj9CSsTg2kG8u4u3aiVwFAl+ROzcACgkQkG8u4u3a iVygIAgAsfdcJLK9HQXeSTjdcckL50UsVuMWXIPat029dNlfm8GUyPlDoU3/axfg m6xsDj0cy4UGW20ZHjl+7vsF5UvgjSU7CNg1iF08Z8X4PscIiUj7oX6b034HG52U 6NQ/H9ma22pum+ae6/73hU9+sgftHqXuMo4Akbis4qRTsMtviBaaW++B/pVt40nS 6aAKN/75srHfcL7lE9rkAk3fku+wGwje3X7s6ZYuSRZcXotuvwrMOEyIcaUlSONF a1UfL4vWsBWddFt4qx7gO40C6nDIxK31qaQQEReY9Pgbd+7Ds/04mKTigt+lU+oZ Qti3b41klkPvrf6p99IiX1nG4tGT2g== =JDnA -----END PGP SIGNATURE----- --Apple-Mail=_A83E54F4-A6CC-4CAB-ADD4-1DE2A8FE37F2-- From owner-freebsd-virtualization@freebsd.org Thu Oct 22 14:03:06 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 262954498C5 for ; Thu, 22 Oct 2020 14:03:06 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CH8G11D9xz3gG1 for ; Thu, 22 Oct 2020 14:03:04 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 8C84E3DA9C; Thu, 22 Oct 2020 10:03:03 -0400 (EDT) To: Jason Tubnor Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: Date: Thu, 22 Oct 2020 10:03:02 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="FL3DnYvzZ43pBiL4XQdoqBeRISABlVMxZ" X-Rspamd-Queue-Id: 4CH8G11D9xz3gG1 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-1.37 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.008]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; ARC_NA(0.00)[]; AUTH_NA(1.00)[]; DMARC_NA(0.00)[druid.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_SHORT(0.30)[0.298]; NEURAL_HAM_MEDIUM(-0.56)[-0.558]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 14:03:06 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --FL3DnYvzZ43pBiL4XQdoqBeRISABlVMxZ Content-Type: multipart/mixed; boundary="Iuj4QjErnR6RKdLFKD9UAonvqgIFjeGVy"; protected-headers="v1" From: D'Arcy Cain To: Jason Tubnor Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" Message-ID: Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> In-Reply-To: --Iuj4QjErnR6RKdLFKD9UAonvqgIFjeGVy Content-Type: multipart/mixed; boundary="------------47019A699F3C4F200A021DDF" Content-Language: en-US This is a multi-part message in MIME format. --------------47019A699F3C4F200A021DDF Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/21/20 11:16 PM, Jason Tubnor wrote: > public: flags=3D8843 metric= 0 mtu 1500 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ether 02:9d:b2:b8:78:00 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0inet 98.158.139.65 netmask 0xfff= fffe0 broadcast 98.158.139.95 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0id 00:00:00:00:00:00 priority 32= 768 hellotime 2 fwddelay 15 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0maxage 20 holdcnt 6 proto rstp m= axaddr 2000 timeout 1200 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0root id 00:00:00:00:00:00 priori= ty 32768 ifcost 0 port 0 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0member: eth0 flags=3D143 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ifma= xaddr 0 port 1 priority 128 path cost 55 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0groups: bridge > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nd6 options=3D9 > tap0: flags=3D8843 metric 0= mtu 1500 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0description: vmnet-BASE-0-public= > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0options=3D80000 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0ether 00:bd:5f:56:f8:00 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0groups: tap vm-port > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0media: Ethernet autoselect > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0status: active > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0nd6 options=3D29 > =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0Opened by PID 3679 >=20 >=20 > tap0 is not a member of public so has no way to get to eth0 or public. Odd. I thought that vm-bhyve did that when the VM came up. Hmm. It looks like tap0 gets attached to vm-public if it is configured t= o=20 attach to public. Perhaps I just need to name the bridge that way. > I'd avoid creating the 'public' bridge and let vm-bhyve create it. =20 > Something like: >=20 > vm switch create public > vm switch add public eth0 But then I can't create the IP on the host until too late in the boot. I= =20 assume that you mean to do that in rc.local. Are you sure that the above= is=20 any different than what I did? > Then for the guest: >=20 > vm add -d network -s public Hard to automate when VM can migrate to another host. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------47019A699F3C4F200A021DDF-- --Iuj4QjErnR6RKdLFKD9UAonvqgIFjeGVy-- --FL3DnYvzZ43pBiL4XQdoqBeRISABlVMxZ Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX5GRFgUDAAAAAAAKCRA/5DDweYZnXabT AP47hAaylY2roGill+T1YL2IeZ/JM4EmTYaPxadW3aUwJwEArz75nXo5cKfrP1FgH/kKn263O7M+ pVSUf7EuHn2mZwU= =JFds -----END PGP SIGNATURE----- --FL3DnYvzZ43pBiL4XQdoqBeRISABlVMxZ-- From owner-freebsd-virtualization@freebsd.org Thu Oct 22 22:50:25 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BAC4542C169 for ; Thu, 22 Oct 2020 22:50:25 +0000 (UTC) (envelope-from paul@redbarn.org) Received: from family.redbarn.org (family.redbarn.org [IPv6:2001:559:8000:cd::5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHMyS3NvYz4bDC for ; Thu, 22 Oct 2020 22:50:24 +0000 (UTC) (envelope-from paul@redbarn.org) Received: from [IPv6:2001:559:8000:c9:c9bf:4c9b:a373:d564] (unknown [IPv6:2001:559:8000:c9:c9bf:4c9b:a373:d564]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by family.redbarn.org (Postfix) with ESMTPSA id BE65DC3F19; Thu, 22 Oct 2020 22:50:14 +0000 (UTC) Subject: Re: When is a switch not a switch? To: D'Arcy Cain Cc: Jason Tubnor , "freebsd-virtualization@freebsd.org" References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> From: Paul Vixie Message-ID: <42e3d3f2-7cf3-568e-8714-086c699e9c33@redbarn.org> Date: Thu, 22 Oct 2020 15:50:12 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 PostboxApp/7.0.34 MIME-Version: 1.0 In-Reply-To: Content-Language: en-US X-Rspamd-Queue-Id: 4CHMyS3NvYz4bDC X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of paul@redbarn.org designates 2001:559:8000:cd::5 as permitted sender) smtp.mailfrom=paul@redbarn.org X-Spamd-Result: default: False [-2.25 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FREEFALL_USER(0.00)[paul]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; R_SPF_ALLOW(-0.20)[+ip6:2001:559:8000::/48]; ARC_NA(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.005]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.989]; DMARC_NA(0.00)[redbarn.org]; NEURAL_HAM_MEDIUM(-0.95)[-0.954]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:33651, ipnet:2001:559:8000::/48, country:US]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 22:50:25 -0000 D'Arcy Cain wrote on 2020-10-22 07:03: > On 10/21/20 11:16 PM, Jason Tubnor wrote: >>     public: flags=8843 metric >> 0 mtu 1500 >>       ... >>               groups: bridge >>               nd6 options=9 >>     tap0: flags=8843 metric 0 >> mtu 1500 >> ... >>               groups: tap vm-port >>               media: Ethernet autoselect >>               ... >> >> >> tap0 is not a member of public so has no way to get to eth0 or public. > > Odd.  I thought that vm-bhyve did that when the VM came up. here's how i do it, per years-ago documentation to this effect: > autobridge_interfaces="bridge0"                                                                                         > autobridge_bridge0="tap* > igb1"                                                                                          > ifconfig_igb1="up media 1000baseTX fib > 1"                                                                               > cloned_interfaces="bridge0 tap0 tap1 tap2 tap3 tap4 tap5 tap6 tap7" > ifconfig_bridge0="inet 24.104.150.210/27 fib > 1"                                                                         > ifconfig_bridge0_ipv6="inet6 2001:559:8000:cd::2/64 fib 1 > auto_linklocal up" > ifconfig_tap0="up fib 1" > ifconfig_tap1="up fib 1" > ifconfig_tap2="up fib 1" > ifconfig_tap3="up fib 1" > ifconfig_tap4="up fib 1" > ifconfig_tap5="up fib 1" > ifconfig_tap6="up fib 1" > ifconfig_tap7="up fib 1" note, fib 1 is a detail here, just gives me a different default route for the virtual machines. > > Hmm.  It looks like tap0 gets attached to vm-public if it is > configured to attach to public.  Perhaps I just need to name the > bridge that way. autobridge_* (rc.conf(5)) is what you'll need for that. > >> I'd avoid creating the 'public' bridge and let vm-bhyve create it.  >> Something like: >> >> vm switch create public >> vm switch add public eth0 > > But then I can't create the IP on the host until too late in the > boot.  I assume that you mean to do that in rc.local.  Are you sure > that the above is any different than what I did? because my management interface for the milking machine is on a different subnet, i have this: > vlans_igb0="201 203" > ifconfig_igb0="up vlanmtu media > 1000baseTX"                                                                             > ifconfig_igb0_201="inet > 24.104.150.130/26"                                                                              > ifconfig_igb0_201_ipv6="inet6 2001:559:8000:c9::3/64" > ifconfig_igb0_203="inet > 24.104.150.16/27"                                                                               > ifconfig_igb0_203_ipv6="inet6 2001:559:8000:cb::16/64" however, you could do away with that and just autobridge your connected interface (mine is igb1, see above.) > >> Then for the guest: >> >> vm add -d network -s public > > Hard to automate when VM can migrate to another host. hard to do any of this if you're not doing it by hand. (i don't use the "vm" command.) -- Sent from Postbox From owner-freebsd-virtualization@freebsd.org Fri Oct 23 04:11:52 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2917E43C16E for ; Fri, 23 Oct 2020 04:11:52 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CHW5M1hxBz4r0b for ; Fri, 23 Oct 2020 04:11:50 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-wm1-f48.google.com with SMTP id c77so8474wmd.4 for ; Thu, 22 Oct 2020 21:11:50 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=xTblUZ9E2SdQxy4nLuGg3GCS6EyACA5vU6UwS4URmWI=; b=KYYsTqCNJ0kVaG5hCsvuw6+96h02WvqXnc5axFvvUkuXGYG33R23yhAT+k2EecVq0g JH53rf5YiK2m02lx2R/3msKfKelg0CI2dAIselTdRAAw1BEBcOCSYlsVy8b+0fzzRkxR fZdVh+oqqC22av1ZGZnPahq8x/8WhJkkD3z9zxEZDmvNDcFYgDl5CgWQouSV58Fv8puj bIkcpJHPxj6N9Xadz5YNTh5LHThBq+s9xt4QL5brlLWoeWXDzgrGpNAS17WanRqC7EdJ Z+9KEJRfV1XbzY0PKShwW5VxH2THT13Kr521SmmNbtOA6q71eD7SY9UaIgT16MpRXYwb Xb7g== X-Gm-Message-State: AOAM5322je+hBfX5HzaogZjuqSBHRVLz1nd00VZdsyjyhuJRTwzjfDen s6QvoremI32Ac4Nbkrk2DApon2xvj3dTah/SReE= X-Google-Smtp-Source: ABdhPJy5kog5JH8idrtJ56yavtJtVXv+kbkgMwicHkJgxvzTKO5g/gfqtgIwkTmMYfhvONyrk3XfJ1nR90ib/tv+fkU= X-Received: by 2002:a1c:417:: with SMTP id 23mr233905wme.1.1603426309451; Thu, 22 Oct 2020 21:11:49 -0700 (PDT) MIME-Version: 1.0 References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> In-Reply-To: From: Jason Tubnor Date: Fri, 23 Oct 2020 15:11:39 +1100 Message-ID: Subject: Re: When is a switch not a switch? To: "D'Arcy Cain" Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 4CHW5M1hxBz4r0b X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.128.48 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [-1.00 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; NEURAL_HAM_LONG(-0.97)[-0.967]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[tubnor.net]; TO_DN_SOME(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.09)[-0.086]; RCVD_IN_DNSWL_NONE(0.00)[209.85.128.48:from]; NEURAL_HAM_MEDIUM(-0.94)[-0.944]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.128.48:from]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; MAILMAN_DEST(0.00)[freebsd-virtualization] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 04:11:52 -0000 On Fri, 23 Oct 2020 at 01:03, D'Arcy Cain wrote: > > > > > > > tap0 is not a member of public so has no way to get to eth0 or public. > > Odd. I thought that vm-bhyve did that when the VM came up. > > Hmm. It looks like tap0 gets attached to vm-public if it is configured to > attach to public. Perhaps I just need to name the bridge that way. > If you create the virtual switch (bridge) for vm-bhyve using: vm switch create -b Then for the guest: vm add -d network -s When you start your guest up, it will attach to the existing bridge you defined in rc.conf, each time. > > > vm add -d network -s public > > Hard to automate when VM can migrate to another host. > > If both your hosts are configured using the same naming conventions for bridges and vm-bhyve switches, migration should be pretty simple and painless. Cheers, Jason. From owner-freebsd-virtualization@freebsd.org Fri Oct 23 12:40:58 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 945F6445E3F for ; Fri, 23 Oct 2020 12:40:58 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CHkNn56s0z45Ch for ; Fri, 23 Oct 2020 12:40:57 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 319E33F964; Fri, 23 Oct 2020 08:40:50 -0400 (EDT) To: Paul Vixie Cc: Jason Tubnor , "freebsd-virtualization@freebsd.org" References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> <42e3d3f2-7cf3-568e-8714-086c699e9c33@redbarn.org> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: Date: Fri, 23 Oct 2020 08:40:49 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: <42e3d3f2-7cf3-568e-8714-086c699e9c33@redbarn.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="ORceTUgHN3bXMRtDXkn9hbBpzQi3eKQYr" X-Rspamd-Queue-Id: 4CHkNn56s0z45Ch X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-2.41 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.92)[-0.919]; HAS_ATTACHMENT(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; ARC_NA(0.00)[]; AUTH_NA(1.00)[]; DMARC_NA(0.00)[druid.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.51)[-0.514]; NEURAL_HAM_MEDIUM(-0.88)[-0.878]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 12:40:58 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --ORceTUgHN3bXMRtDXkn9hbBpzQi3eKQYr Content-Type: multipart/mixed; boundary="BZFN6xPDtys8019u320h7O067Cyy26GQs"; protected-headers="v1" From: D'Arcy Cain To: Paul Vixie Cc: Jason Tubnor , "freebsd-virtualization@freebsd.org" Message-ID: Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> <42e3d3f2-7cf3-568e-8714-086c699e9c33@redbarn.org> In-Reply-To: <42e3d3f2-7cf3-568e-8714-086c699e9c33@redbarn.org> --BZFN6xPDtys8019u320h7O067Cyy26GQs Content-Type: multipart/mixed; boundary="------------1D50B990EDBAF03DA8D636CB" Content-Language: en-US This is a multi-part message in MIME format. --------------1D50B990EDBAF03DA8D636CB Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/22/20 6:50 PM, Paul Vixie wrote: > here's how i do it, per years-ago documentation to this effect: >=20 >> autobridge_interfaces=3D"bridge0" autobridge_bridge0=3D"tap* igb1"=20 Will that work if I have two networks? autobridge_interfaces=3D"bridge0" autobridge_bridge0=3D"tap* bge0" autobridge_interfaces=3D"bridge1" autobridge_bridge1=3D"tap* bge1" Seems like it would get confused about which tap to assign to which bridg= e. >> ifconfig_igb1=3D"up media 1000baseTX fib 1" cloned_interfaces=3D"bridg= e0 tap0=20 >> tap1 tap2 tap3 tap4 tap5 tap6 tap7" Again, they aren't all assigned to one bridge. Probably the even taps wo= uld=20 go to one and the odd taps to the other but that's not guaranteed. >> ifconfig_bridge0=3D"inet 24.104.150.210/27 fib 1"=20 >> ifconfig_bridge0_ipv6=3D"inet6 2001:559:8000:cd::2/64 fib 1 auto_linkl= ocal up" >> ifconfig_tap0=3D"up fib 1" >> ifconfig_tap1=3D"up fib 1" >> ifconfig_tap2=3D"up fib 1" >> ifconfig_tap3=3D"up fib 1" >> ifconfig_tap4=3D"up fib 1" >> ifconfig_tap5=3D"up fib 1" >> ifconfig_tap6=3D"up fib 1" >> ifconfig_tap7=3D"up fib 1" >=20 > note, fib 1 is a detail here, just gives me a different default route f= or=20 > the virtual machines. In my case I want to make the default route the same as the host's. > autobridge_* (rc.conf(5)) is what you'll need for that. See above. --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------1D50B990EDBAF03DA8D636CB-- --BZFN6xPDtys8019u320h7O067Cyy26GQs-- --ORceTUgHN3bXMRtDXkn9hbBpzQi3eKQYr Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX5LPUQUDAAAAAAAKCRA/5DDweYZnXVUU AP9RbWav2skChlghoGaU3WlMxa399TQqY2bB8s//1ipxQwD+JuY2yZM7cHxsLGuAsz7Axb1GPsIl L/wo82xiP8kyiQg= =2iKu -----END PGP SIGNATURE----- --ORceTUgHN3bXMRtDXkn9hbBpzQi3eKQYr-- From owner-freebsd-virtualization@freebsd.org Fri Oct 23 12:46:37 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E9429446254 for ; Fri, 23 Oct 2020 12:46:37 +0000 (UTC) (envelope-from darcy@druid.net) Received: from mail.vex.net (mail.vex.net [98.158.139.68]) by mx1.freebsd.org (Postfix) with ESMTP id 4CHkWJ5DBGz45kL for ; Fri, 23 Oct 2020 12:46:36 +0000 (UTC) (envelope-from darcy@druid.net) Received: from imp.druid.net (unknown [98.158.128.36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) (Authenticated sender: darcy) by mail.vex.net (Postfix) with ESMTPSA id 6C9883F99F; Fri, 23 Oct 2020 08:46:34 -0400 (EDT) To: Jason Tubnor Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> From: D'Arcy Cain Autocrypt: addr=darcy@druid.net; keydata= xjMEXu32OxYJKwYBBAHaRw8BAQdAD839OEQVMqCs45KVwLKg4srvl51WsVhMpTGOd9z3Ym3N HUQnQXJjeSBDYWluIDxkYXJjeUBkcnVpZC5uZXQ+wpYEExYIAD4WIQSQJTNYM0vv3aTmBCs/ 5DDweYZnXQUCXu32OwIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRA/5DDw eYZnXWAxAP9cDL1j4koUFRBNQyC4iMp/cx5KnozmtsFtiTrWdZBrIwD6A2rLFx60tJryd/qp SZpXk7UPDLH/PY1hstjx9WUbXQjOOARe7fY7EgorBgEEAZdVAQUBAQdAbK0SOaTRrkI2QAWz rCz29D2RDOgGTvEbDpyWiyA5RhQDAQgHwn4EGBYIACYWIQSQJTNYM0vv3aTmBCs/5DDweYZn XQUCXu32OwIbDAUJCWYBgAAKCRA/5DDweYZnXaNOAQDC77ymNjoMZQVgDCcmgZEk6IZxn45k nyW17OYCpRctvgD9Fg3aocAbHK7V9AhmSbDPxLNQygQUPjjU7Cyn97b1cQs= Subject: Re: When is a switch not a switch? Message-ID: <27a8f88b-1d78-e9df-9c5d-f9fa66c09769@druid.net> Date: Fri, 23 Oct 2020 08:46:33 -0400 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.3.2 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="v5N68DEhNRixEnAy1n5Mdm0oVGfJYn6kq" X-Rspamd-Queue-Id: 4CHkWJ5DBGz45kL X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of darcy@druid.net has no SPF policy when checking 98.158.139.68) smtp.mailfrom=darcy@druid.net X-Spamd-Result: default: False [-2.41 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.92)[-0.919]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain,application/pgp-keys]; HAS_ATTACHMENT(0.00)[]; ARC_NA(0.00)[]; AUTH_NA(1.00)[]; DMARC_NA(0.00)[druid.net]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.51)[-0.513]; NEURAL_HAM_MEDIUM(-0.88)[-0.877]; SIGNED_PGP(-2.00)[]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:+,4:+,5:~]; ASN(0.00)[asn:19842, ipnet:98.158.139.0/24, country:CA]; SUBJECT_ENDS_QUESTION(1.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 12:46:38 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --v5N68DEhNRixEnAy1n5Mdm0oVGfJYn6kq Content-Type: multipart/mixed; boundary="xfDptYKmzVv5RIoyNSWtjlC4Ux6VUcfZk"; protected-headers="v1" From: D'Arcy Cain To: Jason Tubnor Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" Message-ID: <27a8f88b-1d78-e9df-9c5d-f9fa66c09769@druid.net> Subject: Re: When is a switch not a switch? References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> In-Reply-To: --xfDptYKmzVv5RIoyNSWtjlC4Ux6VUcfZk Content-Type: multipart/mixed; boundary="------------5E24E9995C8275E9B306C5EB" Content-Language: en-US This is a multi-part message in MIME format. --------------5E24E9995C8275E9B306C5EB Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable On 10/23/20 12:11 AM, Jason Tubnor wrote: > If both your hosts are configured using the same naming conventions for= =20 > bridges and vm-bhyve switches, migration should be pretty simple and pa= inless. I had to modify vm-bhyve to get it to use my switch names so that works n= ow.=20 It still didn't add it automatically so I had to run "ifconfig public a= ddm=20 tap0" manually. As a result I am right back to where I was. I can ping any IP address on= =20 the net but I cannot make a TCP connection. I can't even use domain name= s=20 because I can't connect to the DNS server in my own network. Actually, I can make a TCP connection from the VM to the host. Do I have= to=20 set up some sort of proxy arp? How would I do that? --=20 D'Arcy J.M. Cain | Democracy is three wolves http://www.druid.net/darcy/ | and a sheep voting on +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net Disclaimer: By sending an email to ANY of my addresses you are agreeing that: 1. I am by definition, "the intended recipient". 2. All information in the email is mine to do with as I see fit and make such financial profit, political mileage, or good joke as it lends itself to. In particular, I may quote it where I please. 3. I may take the contents as representing the views of your company if I so wish. 4. This overrides any disclaimer or statement of confidentiality that may be included or implied in your message. --------------5E24E9995C8275E9B306C5EB-- --xfDptYKmzVv5RIoyNSWtjlC4Ux6VUcfZk-- --v5N68DEhNRixEnAy1n5Mdm0oVGfJYn6kq Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQSQJTNYM0vv3aTmBCs/5DDweYZnXQUCX5LQqQUDAAAAAAAKCRA/5DDweYZnXUyl AP99eCgBayi5uA/14dgka2LO/1HlAJqi1chLo+J0UJZHmgD/bzNtojZvLY8Daszs3sXVe70fjF4m AhcZAcY5UMqMZwU= =eiI5 -----END PGP SIGNATURE----- --v5N68DEhNRixEnAy1n5Mdm0oVGfJYn6kq-- From owner-freebsd-virtualization@freebsd.org Fri Oct 23 23:41:35 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E5A454292A3 for ; Fri, 23 Oct 2020 23:41:35 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-wm1-f50.google.com (mail-wm1-f50.google.com [209.85.128.50]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CJ1330NLWz3TkP for ; Fri, 23 Oct 2020 23:41:34 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-wm1-f50.google.com with SMTP id c194so3915365wme.2 for ; Fri, 23 Oct 2020 16:41:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=vzwHNq0BG2e0FjLVjLmtniipsdfSNxkz/iElXN93sF4=; b=biCfCnLRPXM8rK7m8ez3o3zM/Umfwse+NjohTp7Wz4nE2VyElLZIiGZoz6/DWxN8c2 U0pCtoNZnWTAnbFAk+VLvj2vY4sx/w/mI9I0R9KjjdgqLSmfq3yGem78W14EAiOyofaY MMldKI5Cu+y5if+KPFSLDqC2E1VEGgJ6pZ1z8shFxDuMEMEKndvVeXH5ruYc/23PNJTL +V+mhb8B1+71fVeY1j+2NYJbJMl4HiAXONTsp6xXKtc28s4cp+8VokDQtzl7WCUoYBKB 8CGNndS4LbPFU+BK+LroFps52yqGrTDaUmx34MDDBr0pqMktiMQu3ZTVX6ICzakL5JKf zmMw== X-Gm-Message-State: AOAM530vfDwO41xtyWswsrWFvnaMgBemzYbgzARwPN4TJHaSZ6Jqf2Em BApZNPsu2U52xHjoSmjanEVkv02ngwYB2ESCTKw= X-Google-Smtp-Source: ABdhPJwHJmUu8gXniQVhKoK3x3RbGPxB/ere4IkqFdwLzy+MVsLRS0+x9xsCiinSo6VkDbADDCfkBBpyaqKpouGhI2E= X-Received: by 2002:a1c:9641:: with SMTP id y62mr4666335wmd.145.1603496493206; Fri, 23 Oct 2020 16:41:33 -0700 (PDT) MIME-Version: 1.0 References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> <20201020065630.GE8272@funkthat.com> <3ed627e2-d99a-107e-4135-8aef1ad4ec71@druid.net> <30A67F82-312E-4651-A5E7-2E2AD926FF24@punkt.de> <973b1b56-817f-6976-e5d3-34cfbc373b13@druid.net> <723CA318-80E9-4A6B-91ED-E791A40CC2C0@punkt.de> <6267599d-bc0c-6d63-bb19-1fc695e53997@druid.net> <27a8f88b-1d78-e9df-9c5d-f9fa66c09769@druid.net> In-Reply-To: <27a8f88b-1d78-e9df-9c5d-f9fa66c09769@druid.net> From: Jason Tubnor Date: Sat, 24 Oct 2020 10:41:21 +1100 Message-ID: Subject: Re: When is a switch not a switch? To: "D'Arcy Cain" Cc: "Patrick M. Hausen" , "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 4CJ1330NLWz3TkP X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.128.50 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [-2.01 / 15.00]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; ARC_NA(0.00)[]; DMARC_NA(0.00)[tubnor.net]; NEURAL_HAM_LONG(-0.97)[-0.968]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-1.07)[-1.068]; RCVD_IN_DNSWL_NONE(0.00)[209.85.128.50:from]; NEURAL_HAM_MEDIUM(-0.98)[-0.978]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.128.50:from]; SUBJECT_ENDS_QUESTION(1.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; R_DKIM_NA(0.00)[]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; MAILMAN_DEST(0.00)[freebsd-virtualization] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Oct 2020 23:41:36 -0000 On Fri, 23 Oct 2020 at 23:46, D'Arcy Cain wrote: > > > Actually, I can make a TCP connection from the VM to the host. Do I have > to > set up some sort of proxy arp? How would I do that? > > > In 12.1/2, we experienced issues similar, turning off LRO (-lro) on the physical interface fixed this for us. LRO causes packet fragmentation and I am sure you'd get ICMP to fail if you increased the payload. Cheers, Jason. From owner-freebsd-virtualization@freebsd.org Sat Oct 24 16:15:51 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8E6D544DD82 for ; Sat, 24 Oct 2020 16:15:51 +0000 (UTC) (envelope-from xavier@xavierhumbert.net) Received: from aragorn.amdh.fr (aragorn.groumpf.org [176.31.180.205]) by mx1.freebsd.org (Postfix) with UTF8SMTP id 4CJR6G4pw1z3Vmt for ; Sat, 24 Oct 2020 16:15:50 +0000 (UTC) (envelope-from xavier@xavierhumbert.net) Received: from numenor.groumpf.org (numenor [78.251.129.10]) by aragorn.amdh.fr (Postfix) with UTF8SMTP id 2AED3DD6C1 for ; Sat, 24 Oct 2020 18:15:43 +0200 (CEST) Received: from numenor.groumpf.org (localhost [127.0.0.1]) by numenor.groumpf.org (Postfix) with UTF8SMTP id 0B4BB201F7 for ; Sat, 24 Oct 2020 18:15:43 +0200 (CEST) X-Virus-Scanned: amavisd-new at groumpf.org Received: from numenor.groumpf.org ([127.0.0.1]) by numenor.groumpf.org (ns3.groumpf.org [127.0.0.1]) (amavisd-new, port 10024) with UTF8LMTP id Pib32yghUHno for ; Sat, 24 Oct 2020 18:15:36 +0200 (CEST) Received: from [192.168.100.25] (feanor.groumpf.org [192.168.100.25]) by numenor.groumpf.org (Postfix) with UTF8SMTPSA id 2C17420227 for ; Sat, 24 Oct 2020 18:15:36 +0200 (CEST) To: freebsd-virtualization@freebsd.org From: Xavier Humbert Subject: Problem setting up Centos 7 Message-ID: Date: Sat, 24 Oct 2020 18:15:09 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Thunderbird/83.0 MIME-Version: 1.0 Content-Language: fr X-Rspamd-Queue-Id: 4CJR6G4pw1z3Vmt X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of xavier@xavierhumbert.net has no SPF policy when checking 176.31.180.205) smtp.mailfrom=xavier@xavierhumbert.net X-Spamd-Result: default: False [2.32 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.66)[0.657]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.55)[0.547]; DMARC_NA(0.00)[xavierhumbert.net]; NEURAL_SPAM_LONG(0.12)[0.118]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:16276, ipnet:176.31.0.0/16, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RECEIVED_SPAMHAUS_PBL(0.00)[78.251.129.10:received] Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2020 16:15:51 -0000 Hi, I haven't posted to this list for a while. I'm trying to set up Centos=20 with vm-bhyve on 12STABLE. I keep getting > error: disk `hd0,msdos1' not found. > error: you need to load the kernel first. My config file is=C2=A0 this one : > loader=3D"grub" > cpu=3D1 > memory=3D512M > network0_type=3D"virtio-net" > network0_switch=3D"public" > disk0_type=3D"virtio-blk" > disk0_name=3D"disk0.img" > grub_install0=3D"linux /isolinux/vmlinuz LANG=3Den_US.UTF-8 KEYTABLE=3D= us=20 > SYSFONT=3Dlatarcyrheb-sun16 console=3DttyS0" > grub_install1=3D"initrd /isolinux/initrd.img" > grub_run0=3D"linux /vmlinuz-3.10.0-957.el7.x86_64=20 > root=3D/dev/mapper/centos-root LANG=3Den_US.UTF-8 KEYTABLE=3Dus=20 > SYSFONT=3Dlatarcyrheb-sun16 console=3DttyS0" > grub_run1=3D"initrd /initramfs-3.10.0-957.el7.x86_64.img" Yes, I ran strings(1) on vmlinux to get the exact version. What is wrong with my VM ? Thanks in advance, Xavier --=20 Xavier HUMBERT Linux/FreeBSD Senior Sysadmin From owner-freebsd-virtualization@freebsd.org Sat Oct 24 21:18:53 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7D033454BE2 for ; Sat, 24 Oct 2020 21:18:53 +0000 (UTC) (envelope-from xavier@xavierhumbert.net) Received: from aragorn.amdh.fr (aragorn.groumpf.org [176.31.180.205]) by mx1.freebsd.org (Postfix) with UTF8SMTP id 4CJYqw1SXPz46bg for ; Sat, 24 Oct 2020 21:18:51 +0000 (UTC) (envelope-from xavier@xavierhumbert.net) Received: from numenor.groumpf.org (numenor [78.251.129.10]) by aragorn.amdh.fr (Postfix) with UTF8SMTP id 767F5DEA09 for ; Sat, 24 Oct 2020 23:18:49 +0200 (CEST) Received: from numenor.groumpf.org (localhost [127.0.0.1]) by numenor.groumpf.org (Postfix) with UTF8SMTP id 1D5A0200DD for ; Sat, 24 Oct 2020 23:18:49 +0200 (CEST) X-Virus-Scanned: amavisd-new at groumpf.org Received: from numenor.groumpf.org ([127.0.0.1]) by numenor.groumpf.org (ns3.groumpf.org [127.0.0.1]) (amavisd-new, port 10024) with UTF8LMTP id Ztvlm2PaN469 for ; Sat, 24 Oct 2020 23:18:37 +0200 (CEST) Received: from [192.168.100.25] (feanor.groumpf.org [192.168.100.25]) by numenor.groumpf.org (Postfix) with UTF8SMTPSA id 64A722015C for ; Sat, 24 Oct 2020 23:18:37 +0200 (CEST) From: Xavier Humbert To: freebsd-virtualization@freebsd.org References: Subject: Re: Problem setting up Centos 7 Message-ID: Date: Sat, 24 Oct 2020 23:18:30 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:83.0) Gecko/20100101 Thunderbird/83.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable Content-Language: fr X-Rspamd-Queue-Id: 4CJYqw1SXPz46bg X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of xavier@xavierhumbert.net has no SPF policy when checking 176.31.180.205) smtp.mailfrom=xavier@xavierhumbert.net X-Spamd-Result: default: False [2.21 / 15.00]; ARC_NA(0.00)[]; RCVD_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_SPAM_SHORT(0.54)[0.536]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; TO_DN_NONE(0.00)[]; AUTH_NA(1.00)[]; RCPT_COUNT_ONE(0.00)[1]; NEURAL_SPAM_MEDIUM(0.55)[0.548]; DMARC_NA(0.00)[xavierhumbert.net]; NEURAL_SPAM_LONG(0.13)[0.128]; R_SPF_NA(0.00)[no SPF record]; RCVD_NO_TLS_LAST(0.10)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:16276, ipnet:176.31.0.0/16, country:FR]; MID_RHS_MATCH_FROM(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RECEIVED_SPAMHAUS_PBL(0.00)[78.251.129.10:received] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Oct 2020 21:18:53 -0000 On 24/10/2020 18:15, Xavier Humbert wrote: > Hi, > > I haven't posted to this list for a while. I'm trying to set up Centos = > with vm-bhyve on 12STABLE. > > I keep getting > >> error: disk `hd0,msdos1' not found. >> error: you need to load the kernel first. > My config file is=C2=A0 this one : >> loader=3D"grub" >> cpu=3D1 >> memory=3D512M >> network0_type=3D"virtio-net" >> network0_switch=3D"public" >> disk0_type=3D"virtio-blk" >> disk0_name=3D"disk0.img" >> grub_install0=3D"linux /isolinux/vmlinuz LANG=3Den_US.UTF-8 KEYTABLE=3D= us=20 >> SYSFONT=3Dlatarcyrheb-sun16 console=3DttyS0" >> grub_install1=3D"initrd /isolinux/initrd.img" >> grub_run0=3D"linux /vmlinuz-3.10.0-957.el7.x86_64=20 >> root=3D/dev/mapper/centos-root LANG=3Den_US.UTF-8 KEYTABLE=3Dus=20 >> SYSFONT=3Dlatarcyrheb-sun16 console=3DttyS0" >> grub_run1=3D"initrd /initramfs-3.10.0-957.el7.x86_64.img" > Yes, I ran strings(1) on vmlinux to get the exact version. > > What is wrong with my VM ? > > Thanks in advance, > > Xavier OK, I destroyed and recreated the VM, now the installer launches with=20 this very config. Please note that UEFI loader doesn't work, it angs Now I run into the infamous XFS problem... The text installer doesn't allow to choose the filesystem type. And VNC=20 won't launch (network not activated) I'm lost and don't want to install another Linux, I've to install a RPM=20 based appliance... Googled 3 hours with no success. Any idea ? Thanks, Xavier --=20 Xavier HUMBERT Linux/FreeBSD Senior Sysadmin