From owner-freebsd-virtualization@freebsd.org Tue Oct 20 06:56:42 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A0973446CDB for ; Tue, 20 Oct 2020 06:56:42 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFktx4T6Sz3bc0 for ; Tue, 20 Oct 2020 06:56:41 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 09K6uU6v078799 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 19 Oct 2020 23:56:30 -0700 (PDT) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 09K6uUBw078798; Mon, 19 Oct 2020 23:56:30 -0700 (PDT) (envelope-from jmg) Date: Mon, 19 Oct 2020 23:56:30 -0700 From: John-Mark Gurney To: "D'Arcy Cain" Cc: freebsd-virtualization@freebsd.org Subject: Re: When is a switch not a switch? Message-ID: <20201020065630.GE8272@funkthat.com> Mail-Followup-To: D'Arcy Cain , freebsd-virtualization@freebsd.org References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline In-Reply-To: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Mon, 19 Oct 2020 23:56:30 -0700 (PDT) X-Rspamd-Queue-Id: 4CFktx4T6Sz3bc0 X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of jmg@gold.funkthat.com has no SPF policy when checking 208.87.223.18) smtp.mailfrom=jmg@gold.funkthat.com X-Spamd-Result: default: False [-0.94 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; ARC_NA(0.00)[]; FREEFALL_USER(0.00)[jmg]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_TLS_ALL(0.00)[]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; DMARC_NA(0.00)[funkthat.com]; AUTH_NA(1.00)[]; NEURAL_SPAM_MEDIUM(0.08)[0.080]; MID_RHS_MATCH_FROM(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.009]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.11)[-0.108]; RCPT_COUNT_TWO(0.00)[2]; SIGNED_PGP(-2.00)[]; FORGED_SENDER(0.30)[jmg@funkthat.com,jmg@gold.funkthat.com]; R_SPF_NA(0.00)[no SPF record]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:32354, ipnet:208.87.216.0/21, country:US]; FROM_NEQ_ENVFROM(0.00)[jmg@funkthat.com,jmg@gold.funkthat.com]; MAILMAN_DEST(0.00)[freebsd-virtualization]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 06:56:42 -0000 --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable D'Arcy Cain wrote this message on Mon, Oct 19, 2020 at 22:02 -0400: > I am using bhyve with vm-bhyve, I am trying to set up a virtual network= =20 > with multiple hosts. The idea is that a VM would be on the same virtual= =20 > network no matter which actual host it is on. >=20 > Say I have a public network a.b.c.0/24. I thought I could create a switc= h=20 > on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and= =20 > a.b.c.101. The idea would be that the VMs would appear on the real netwo= rk.=20 > Then the 101 VM could migrate to a.b.c.2 and still be accessible. I=20 > envisioned some sort of proxy arp would happen so that every VM would sim= ply=20 > announce itself wherever it was. >=20 > This did seem to work in that I could ping from the VM: >=20 > # ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=3D0 ttl=3D114 time=3D1.734 ms >=20 > Even IPV6: >=20 > # ping6 2605:2600:1001::4b > PING6(56=3D40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D0 hlim=3D64 time=3D0.960 ms > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D1 hlim=3D64 time=3D0.415 ms >=20 > However TCP doesn't work. In fact, I could only ping by IP because the= =20 > system couldn't connect to the DNS server, to get an address even though = it=20 > could ping it. >=20 > I guess my first question is does this seem doable? If so, what am I=20 > missing? Is it possible that a bhyve switch is more like a router? By switch, do you mean use bridge? How specifically is the network configured? What you are describing sounds like what I do w/ bridge, but my use was slightly more complicated. Say your host has em0 as the main network, you would create a bridge0 interface, either via cloned_interfaces or via "ifconfig bridge0 create". Then you would put the em0 interface as a member of the bridge interface. You would also add the tap interfaces of the various bhyve vms as well (don't forget to make sure the tap interface is up on the host, net.link.tap.up_on_open helps w/ this)... I have heard (and that is the way I do that), that you have to put the host IPs on the bridge0 interface, and not the em0 interface. --=20 John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not." --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQJ8BAEBCgBmBQJfjoodXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2MEI1RTRGMTNDNzYyMDZDNjEyMDBCNjAy MDVGMEIzM0REMDA2QURBAAoJECBfCzPdAGraXcYP/0rzorXTZBfADZdRhm2DBuie NwdWSOIwgp1nHLwgLdAm58p2D7CzeDjrQ9CIqNGUeFRv/VIGpzJCuyTTeaWpJfWQ a5eqPwdpiPPBICS8JArk4ixm7Q5LZby8k2b1f6gM9KEkwjm4estrqNLd1w7tGa0K wIWscYaVEd5qj5eIMyD13XdeTKumGywXEtzN1oeNCJVF8ycoUkipTuY1ZKunICpp +N/2dLiJetFps2bNuz0G8SqevaAjn0NZ0YH2k1HlQPuIKzP+HnTxDL+k7bTTRbZj fRSIh/w/HIrpjjFQ6dLUiJVAvNSf4gM3A5XexgHCgiETOvUiF+c5SwaIZ0i/Gq3p dI9Bg7M9XIt88jGPoO6Zv/fW0gCpFGilq8jWNg+vO3BY6bZovzx91c9Np9o2TJig fIXKvIoGlC1Ba6+YrUqg5t3htA+ImA0nzo8TWQOOcghXLGrhfpFg3A/qRCiBvutL v+k9nDTUBp5IIQFiTB1uj583L1rZ8JNMfoYEGchun2PnURxuHW+QITBx8FQufjRA VKc3FvhLHe1N26J3+kBtjxYbUlzUAB5CKiKjaU9nNP7IvGy+UuWs7A6/ZQXea4ng jIVi7sD4XaTD27HpTVoaubMBBdDWof14+ZMPA/+zLTi2FWq/eCJ8M92V4NqRqxx4 MxA4GJA3kw+E1bGzDeiA =POR4 -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA--