From owner-freebsd-virtualization@freebsd.org Wed Nov 4 21:53:06 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B7084679A5 for ; Wed, 4 Nov 2020 21:53:06 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CRL4K5fwsz4LZD for ; Wed, 4 Nov 2020 21:53:05 +0000 (UTC) (envelope-from jtubnor@gmail.com) Received: by mail-wm1-f42.google.com with SMTP id h62so3737349wme.3 for ; Wed, 04 Nov 2020 13:53:05 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LsV1a2V2/wfgVY1QrUSLfkuE+iyz0ovPq5dFp/juuvE=; b=Bmq2Q4cG07UFdRwI/MTpaW0VpMkZZgkRkGEQ+ct2PhGnXKYKprQ0dzE4dzJvNLVwQB jQrbDa92+0O5JEqq/Secz69I7egj6A2ZKj0G9FyG6M6UagKyv9YYa9HMDawCjW7G+mTH 7IXbiALYV7rAPfGl/CBTUK125qB3JbgVna53fw39BHU2FwJd7gS5nXv31w2HEHNAaxnO 5bwy64JmbXh+ShpruloicRbQiwL/bt1ppmcgfOqDtQbC8BU2KrFcw6wtcdWDaKMP0FN1 llYIs2K3BXHgfZ4GcneagTlUk3U9Sfe6wEzQ3BZ8Dy3aCp0WixuAm93T9+/7la6795Wn UM/A== X-Gm-Message-State: AOAM5316aO/JFjMUSlbN2LXUuyJoau4UTodd0SnCiR9BKPS7dfBOZ5zO pNWm2XeuQ6hpVBZBW6a0WpwD/ciw65YgNwI0ABykpU44 X-Google-Smtp-Source: ABdhPJwzWhGEh6Y3DCftao+p2DXGFD7l9YoJBixSSc0m8iRnvETlk1Y40s3YtTdS1QpgAY4nKRRxLVnmnAGZvGiFEoA= X-Received: by 2002:a7b:c2ef:: with SMTP id e15mr7155691wmk.180.1604526783667; Wed, 04 Nov 2020 13:53:03 -0800 (PST) MIME-Version: 1.0 References: <01000175941a2783-79804ed8-eafa-4f80-92d4-3f500e9d7993-000000@email.amazonses.com> In-Reply-To: <01000175941a2783-79804ed8-eafa-4f80-92d4-3f500e9d7993-000000@email.amazonses.com> From: Jason Tubnor Date: Thu, 5 Nov 2020 08:52:52 +1100 Message-ID: Subject: Re: Using OpenBSD guest as PF firewall To: lausts@acm.org Cc: "freebsd-virtualization@freebsd.org" X-Rspamd-Queue-Id: 4CRL4K5fwsz4LZD X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of jtubnor@gmail.com designates 209.85.128.42 as permitted sender) smtp.mailfrom=jtubnor@gmail.com X-Spamd-Result: default: False [-1.98 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; NEURAL_HAM_LONG(-1.01)[-1.008]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; DMARC_NA(0.00)[tubnor.net]; NEURAL_SPAM_SHORT(0.02)[0.020]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[209.85.128.42:from]; NEURAL_HAM_MEDIUM(-1.00)[-0.995]; FORGED_SENDER(0.30)[jason@tubnor.net,jtubnor@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.128.42:from]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_NEQ_ENVFROM(0.00)[jason@tubnor.net,jtubnor@gmail.com]; MAILMAN_DEST(0.00)[freebsd-virtualization] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Nov 2020 21:53:06 -0000 On Thu, 5 Nov 2020 at 03:32, Thomas Laus wrote: > Is there a How-To or a Handbook article for using an OpenBSD guest as a > firewall for a FreeBSD host? I have enabled pci-passthru and the > OpenBSD guest can use the functional FreeBSD NIC hardware and has a > hostname.vio0 configured with an IP address and netmask. I have created > I think you are getting a few things mixed up here. If you pass through the adaptor to OpenBSD, then you'll address it by the real device name and not use the vio driver. Once you pass it through, the host will not be able to communicate with the guest via the same path, it will be via some other physical connection back to a switch. If you want the guest and host to communicate over the same wire, then you need to bridge the physical interface at the host level and then add a tap to that bridge that the guest will then use. a public switch on the FreeBSD side and have added tap0. I can connect > both from and to the OpenBSD / FreeBSD host by their respective IP > addresses. These addresses both use the same subnet. I can't connect > anywhere else from the FreeBSD host. The OpenBSD guest has an open > pf.conf file to pass all packets to from vio0 to my re0 NIC. > > The OpenBSD system is version 6.8 and the FreeBSD is Current r367054. > It looks like I need to create a bridge somewhere, but can't find the > proper commands to make one. I have read a few instructions, but none > of them use commands from the vm-bhyve port. I found that is always a > good idea to not 'mix and match' these methods. > We use the bridge/vio/tap configuration extensively. Testing on -HEAD shows that two OpenBSD guests can communicate with each other and the rest of the network at 3.5Gb/s with bridge. We see even faster with netmap/VALE (17Gb/s) but OpenBSD vio driver has checksum issues that I need to sort out. We don't pass-thru any hardware as it exceeds the level of comfort that we are happy with with deployed remote hosts. Cheers, Jason.