From owner-freebsd-virtualization@freebsd.org Tue Oct 20 23:51:44 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A638F43D3F9 for ; Tue, 20 Oct 2020 23:51:44 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CG9Q74gRdz4ssw for ; Tue, 20 Oct 2020 23:51:43 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-wr1-f51.google.com with SMTP id n15so407469wrq.2 for ; Tue, 20 Oct 2020 16:51:43 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=888P/dW1GbixsbZuO9QGhfX+89ARGrScYBh0kUpzxK8=; b=AI9kVfHWl0yUBlaQ+1tHuoETXbml7HfJ/yL+2kHhGWeg8rY59U4Oc38N2MzeS3n5wP 2p42d8e75wuoP/yG158guXI2CPZ7KOgSVkJKgju5AXniJkCAn4eNz3rMsmEEdml/tDtv TYwOHewxuH3hVDNrkAoxkCVjUPXdwlyAPgtJ+qi3WID9fgsV4Q1CJ1Ma0kcPxtsZ1V2N bWZNx7861yApCJFGrgdGeJz632McqoQPo/lbLZsCfPl/6HvmWtZUKBTKHR6Jcq3vo1xr tovLvM3ojD/EYTKuRFlZJHh28qgQy6j3QFpeWRp7AGUdSZgpn8Kz/qP06VUUEgYh8O1c 49fw== X-Gm-Message-State: AOAM5333soqY6BLlAEBlHvQm3zfXp49cbBjPNcvtVq5CCEu+EzQttzL5 kKcQWxJ9o67BLGeU7OfvPSfGLi2f9CWZc7KjM1vHz7zS X-Google-Smtp-Source: ABdhPJyvXOvl+dOUJxdJ/qevrOnBwAmBPWfFw+kdmNvTl6gKX2kglUx1S14VtAlq49Uch9jXBAIZpokPW3QYUDgbwWk= X-Received: by 2002:a1c:6643:: with SMTP id a64mr453257wmc.142.1603237901737; Tue, 20 Oct 2020 16:51:41 -0700 (PDT) MIME-Version: 1.0 References: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> In-Reply-To: <57c32e6d-5572-3d3b-1a57-f3064bee7dc2@druid.net> From: Jason Tubnor Date: Wed, 21 Oct 2020 10:51:30 +1100 Message-ID: Subject: Re: When is a switch not a switch? To: "D'Arcy Cain" Cc: "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 4CG9Q74gRdz4ssw X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.221.51 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [0.80 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17:c]; URI_COUNT_ODD(1.00)[1]; NEURAL_HAM_SHORT(-0.64)[-0.638]; RCPT_COUNT_TWO(0.00)[2]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.67)[-0.667]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-0.90)[-0.900]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[tubnor.net]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.221.51:from]; HTTP_TO_IP(1.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.221.51:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-virtualization] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 Oct 2020 23:51:44 -0000 Hi, On Tue, 20 Oct 2020 at 13:02, D'Arcy Cain wrote: > I am using bhyve with vm-bhyve, I am trying to set up a virtual network > with multiple hosts. The idea is that a VM would be on the same virtual > network no matter which actual host it is on. > > Say I have a public network a.b.c.0/24. I thought I could create a switch > on a host. The host would be a.b.c.1 and the VMs would be a.b.c.100 and > a.b.c.101. The idea would be that the VMs would appear on the real > network. > Then the 101 VM could migrate to a.b.c.2 and still be accessible. I > envisioned some sort of proxy arp would happen so that every VM would > simply > announce itself wherever it was. > It looks like you are over complicating this. When using vm-bhyve, as long as each host has the same vswitch (bridge) then the tap will automagically be inserted correctly on guest startup (as long as the conf file follows your guest storage). Let vm-bhyve manage bridge creation. Only use /etc/rc.conf to bring up the interface. If you are running > 11.4 then you must turn LRO off (-lro) when you bring up the interface. The other settings in this thread can be left on. The problem you are experiencing is packet fragmentation that the guest has to deal with because LRO is enabled (off by default in 11, enabled in 12 and above). LRO should be disabled automatically when an interface (or child VLAN) is added to a bridge. I have tried to get the network guys to fix this but no such luck. > This did seem to work in that I could ping from the VM: > > # ping 8.8.8.8 > PING 8.8.8.8 (8.8.8.8): 56 data bytes > 64 bytes from 8.8.8.8: icmp_seq=0 ttl=114 time=1.734 ms > > Even IPV6: > > # ping6 2605:2600:1001::4b > PING6(56=40+8+8 bytes) 2605:2600:1001::4 --> 2605:2600:1001::4b > 16 bytes from 2605:2600:1001::4b, icmp_seq=0 hlim=64 time=0.960 ms > 16 bytes from 2605:2600:1001::4b, icmp_seq=1 hlim=64 time=0.415 ms > > However TCP doesn't work. In fact, I could only ping by IP because the > system couldn't connect to the DNS server, to get an address even though > it > could ping it. > > I guess my first question is does this seem doable? If so, what am I > missing? Is it possible that a bhyve switch is more like a router?