From owner-freebsd-virtualization@freebsd.org Tue Oct 20 12:45:01 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9B5BE43055D for ; Tue, 20 Oct 2020 12:45:01 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-io1-xd32.google.com (mail-io1-xd32.google.com [IPv6:2607:f8b0:4864:20::d32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CFtcr4WBYz4HFH for ; Tue, 20 Oct 2020 12:45:00 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: by mail-io1-xd32.google.com with SMTP id q25so3023982ioh.4 for ; Tue, 20 Oct 2020 05:45:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=A/q2UrZ6RoCIwP9CDhw6hYWrsaHMilvqdCmvf3wQIq0=; b=WJH55OUc7lqOK+yWZCFS+op3TtashFSWIwTmtAlTS4OzoHtNSbpqX3yLfjbk7iqr4v Es5Kaq5JUZp9G/dQLV2bKUmDVzWtk0X0raF+OeRf53iHq1oVjJDVQ3B89cMz7KbsnFyK k4i1zmXm5Aq/6wGPPYblgyFdvIaMwKcRr6cu8zebprflYdoaGG5oX0oeMw/bOXy1cb79 rYR+ZGt+HobxkitgS7V8WqpwDwWkXp1GbQhv1QUCv+XpoC/lDZr9KUWS1X5h5oHNpFQr bDLV3idKdMfnw41VqAs0HzG4QqBYiXn4CfayGnuYgj07HlHVBBPmlkHlMXzJOcWH7xYc vPZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=A/q2UrZ6RoCIwP9CDhw6hYWrsaHMilvqdCmvf3wQIq0=; b=TUGGVosReudcnrw7JNNkBHD6vO11Gkrct3deS4ZAh8DTygvj+o8g5L9GKFV20wTO9U yVB/LQyCI7EliuBLbe/H0jfvHWYpQj3yRrkpSPWwFPDdo4igVjlCL0SQwtRMACZfTK1r C5h4rBT3cjhECo1/gXbcvskwGauT4diAjVSIJ37q8ax1113d8YTkxmrqolFixBUigzd5 OSpQdFYHz55W7sp2F46gW8TDmuXmIp3+C0jFOjpu7XXateXA9r2TmkwXLdruToxvn146 23hU+f0ytMuRC0lb9OeXnYkOZ3sNRqDBWhL7CbZYKpJiosOUCjFdsKgPEvgj54zlGO59 6GPw== X-Gm-Message-State: AOAM530evjGNFnUqaw5dZe1YaxqC2pa2mvhzhXuK+dzWTz8TFBSaCocV iUHXDWe0oJ+Lt9U/ge3HGzuYdiWwMZ+ILW0G X-Google-Smtp-Source: ABdhPJxJ3XUUkBrPAN3LisHexvU8ADPSrf8oLWHqcoGte1TWsFbimLplnBFxWNhobipzej2dHcLSUg== X-Received: by 2002:a5e:c90c:: with SMTP id z12mr1921406iol.150.1603197898090; Tue, 20 Oct 2020 05:44:58 -0700 (PDT) Received: from mutt-hbsd (pool-100-16-222-53.bltmmd.fios.verizon.net. [100.16.222.53]) by smtp.gmail.com with ESMTPSA id p198sm1501239iod.15.2020.10.20.05.44.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Oct 2020 05:44:57 -0700 (PDT) Date: Tue, 20 Oct 2020 08:44:56 -0400 From: Shawn Webb To: D'Arcy Cain Cc: freebsd-virtualization@freebsd.org Subject: Re: When is a switch not a switch? Message-ID: <20201020124456.kyvlhus3qj4o7gtp@mutt-hbsd> X-Operating-System: FreeBSD mutt-hbsd 13.0-CURRENT-HBSD FreeBSD 13.0-CURRENT-HBSD X-PGP-Key: http://pgp.mit.edu/pks/lookup?op=vindex&search=0xFF2E67A277F8E1FA References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="e3lyc3bdulbpnglp" Content-Disposition: inline In-Reply-To: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> X-Rspamd-Queue-Id: 4CFtcr4WBYz4HFH X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=WJH55OUc; dmarc=none; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::d32 as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org X-Spamd-Result: default: False [-3.47 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[hardenedbsd.org:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.37)[-0.373]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; SUBJECT_ENDS_QUESTION(1.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RECEIVED_SPAMHAUS_PBL(0.00)[100.16.222.53:received]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.992]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.01)[-1.007]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[hardenedbsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::d32:from]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 12:45:01 -0000 --e3lyc3bdulbpnglp Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Oct 19, 2020 at 10:02:17PM -0400, D'Arcy Cain wrote: > I am using bhyve with vm-bhyve, I am trying to set up a virtual network > with multiple hosts. The idea is that a VM would be on the same virtual > network no matter which actual host it is on. >=20 > Say I have a public network a.b.c.0/24. I thought I could create a switch > on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and > a.b.c.101. The idea would be that the VMs would appear on the real netwo= rk. > Then the 101 VM could migrate to a.b.c.2 and still be accessible. I > envisioned some sort of proxy arp would happen so that every VM would sim= ply > announce itself wherever it was. >=20 > This did seem to work in that I could ping from the VM: >=20 > # ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=3D0 ttl=3D114 time=3D1.734 ms >=20 > Even IPV6: >=20 > # ping6 2605:2600:1001::4b > PING6(56=3D40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D0 hlim=3D64 time=3D0.960 ms > 16 bytes from 2605:2600:1001::4b, icmp_seq=3D1 hlim=3D64 time=3D0.415 ms >=20 > However TCP doesn't work. In fact, I could only ping by IP because the > system couldn't connect to the DNS server, to get an address even though = it > could ping it. >=20 > I guess my first question is does this seem doable? If so, what am I > missing? Is it possible that a bhyve switch is more like a router? >=20 > Thanks. >=20 > --=20 > D'Arcy J.M. Cain | Democracy is three wolves > http://www.druid.net/darcy/ | and a sheep voting on > +1 416 788 2246 (DoD#0082) (eNTP) | what's for dinner. > IM: darcy@VybeNetworks.com, VoIP: sip:darcy@druid.net >=20 > Disclaimer: By sending an email to ANY of my addresses you > are agreeing that: >=20 > 1. I am by definition, "the intended recipient". > 2. All information in the email is mine to do with as I see > fit and make such financial profit, political mileage, or > good joke as it lends itself to. In particular, I may quote > it where I please. > 3. I may take the contents as representing the views of > your company if I so wish. > 4. This overrides any disclaimer or statement of > confidentiality that may be included or implied in > your message. I usually configure my bridgeN device to have an IP and subnet that I know won't be on any of the physical networks I care about. I'll then add only the tapN..M devices that the bhyve VMs will use to that bridgeN. I'll then use pf to NAT from that private network on bridgeN to the real world. =3D=3D=3D=3D BEGIN rc.conf =3D=3D=3D=3D cloned_interfaces=3D"bridge0 tap0 tap1" ifconfig_bridge0=3D"inet 192.168.254.1 subnet mask 255.255.255.0" ifconfig_bridge0=3D"${ifconfig_bridge0} addm tap0 addm tap1" =3D=3D=3D=3D END rc.conf =3D=3D=3D=3D =3D=3D=3D=3D BEGIN pf.conf =3D=3D=3D=3D table counters { \ 192.168.254.0/24 \ } scrub in all nat on em0 from {} to any -> (em0) nat on wlan0 from {} to any -> (wlan0) pass in all pass out all =3D=3D=3D=3D END pf.conf =3D=3D=3D=3D Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD GPG Key ID: 0xFF2E67A277F8E1FA GPG Key Fingerprint: D206 BB45 15E0 9C49 0CF9 3633 C85B 0AF8 AB23 0FB2 https://git-01.md.hardenedbsd.org/HardenedBSD/pubkeys/src/branch/master/Sha= wn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --e3lyc3bdulbpnglp Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAl+O28UACgkQ/y5nonf4 4fpp9BAAmvrWeKaI9Jrbp7wZ3pcG7x6vlwF0Mp08FqOhbz+mKsMbbPibr7WTWy/x 8CRNo4EwgLD9spWyq+g6YFIvQ0KmIB4kFgMJ9913Ztr9NvaYa2D7DQ/5a/3/31yg MWCpbq3jbwD6ej9SrclURwMcM8vAEUqJOZh20FQgTJ6k6I7TncmBqEjCa0FKJuYP 8kDkenEjk8hk8rmhQTVx4ATyqWcWDRu7vw959MXO1rdenJn+caXRU9eAhzXU8NHd M66fJ8Qyl+EhM/camJm2hFBwJtZ7+BjmW5uXBzjiGTgSOeLRSi1vEQ3FDD1f0in7 upX7LCdPAEJGclWZHncu/H+hxqUx51byqaGIYx8SZC1j9jjk5sIFUDIyntjWHlmx 2e6FKaRDoXeMtsbvotcVGBZZ/20rcf5cfMfAoajtJO6YBDX88mAb2EOdkRz+Ssxi ZacH+OmmtQmfXyURxp89rtsmGHkpYhk3ZaAr4jkVDQDdk5Q+nkpMmxzgGFQ+tQ45 XN2mSKujMyIA76SAfdUgid/hqFoZyK2cJuJrGfmrv0yHKX/D3RjB/86jS3FpNj8O +otD32fxi5bsBGoHn3HSOLpRPAMcMGOJxPD0j5TNH6Ge2dDMc9KoVQDNVvmL1fbJ bwI994sMaQzKjmNjxALP4MWH1e5BmYsnQPMUUmJwgfRt8qmtPec= =+sm/ -----END PGP SIGNATURE----- --e3lyc3bdulbpnglp--