From owner-freebsd-arch@freebsd.org Sat Jan 9 02:24:16 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AACF84DF3C4 for ; Sat, 9 Jan 2021 02:24:16 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (gate2.funkthat.com [208.87.223.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "gate2.funkthat.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCP1D3N7Dz3tPy; Sat, 9 Jan 2021 02:24:15 +0000 (UTC) (envelope-from jmg@gold.funkthat.com) Received: from gold.funkthat.com (localhost [127.0.0.1]) by gold.funkthat.com (8.15.2/8.15.2) with ESMTPS id 1092OAo1073711 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 8 Jan 2021 18:24:10 -0800 (PST) (envelope-from jmg@gold.funkthat.com) Received: (from jmg@localhost) by gold.funkthat.com (8.15.2/8.15.2/Submit) id 1092O9EA073710; Fri, 8 Jan 2021 18:24:09 -0800 (PST) (envelope-from jmg) Date: Fri, 8 Jan 2021 18:24:09 -0800 From: John-Mark Gurney To: John Baldwin Cc: Andrew Gallatin , freebsd-arch@FreeBSD.org, Rick Macklem , Allan Jude Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? Message-ID: <20210109022409.GL31099@funkthat.com> Mail-Followup-To: John Baldwin , Andrew Gallatin , freebsd-arch@FreeBSD.org, Rick Macklem , Allan Jude References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> <20210108214446.GJ31099@funkthat.com> <4fe4a57c-8c43-a677-4872-d0671104c414@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4fe4a57c-8c43-a677-4872-d0671104c414@FreeBSD.org> X-Operating-System: FreeBSD 11.3-STABLE amd64 X-PGP-Fingerprint: D87A 235F FB71 1F3F 55B7 ED9B D5FF 5A51 C0AC 3D65 X-Files: The truth is out there X-URL: https://www.funkthat.com/ X-Resume: https://www.funkthat.com/~jmg/resume.html X-TipJar: bitcoin:13Qmb6AeTgQecazTWph4XasEsP7nGRbAPE X-to-the-FBI-CIA-and-NSA: HI! HOW YA DOIN? can i haz chizburger? User-Agent: Mutt/1.6.1 (2016-04-27) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (gold.funkthat.com [127.0.0.1]); Fri, 08 Jan 2021 18:24:10 -0800 (PST) X-Rspamd-Queue-Id: 4DCP1D3N7Dz3tPy X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2021 02:24:16 -0000 John Baldwin wrote this message on Fri, Jan 08, 2021 at 17:03 -0800: > On 1/8/21 1:44 PM, John-Mark Gurney wrote: > > Andrew Gallatin wrote this message on Fri, Jan 08, 2021 at 12:26 -0500: > >> Kernel TLS (KTLS) support was added roughly a year ago, and provides > >> an efficient software or hardware accelerated path to have the kernel > >> (or the NIC) handle TLS crypto. This is quite useful for web and > >> NFS servers, and provides a huge (2x -> 5x) efficiency gain by > >> avoiding data copies into userspace for crypto, and potentially > >> offloading the crypto to hardware. > >> > >> > >> KTLS is well tested on amd64, having been used in production at Netflix > >> for nearly 4 years. The vast majority of Netflix video has been served > >> via KTLS for the last few years. Its what has allowed us to serve > >> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve > >> nearly 400Gb/s on AMD servers with NICs which support crypto offload. > >> > >> I have received a few requests to enable it by default in GENERIC, and > >> I'd like to get some opinions. > >> > >> There are essentially 3 options > >> > >> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and > >> flipping kern.ipc.tls.enable=1 > >> > >> The advantage of this is that it "just works" out of the box for users, > >> and for reviewers. > >> > >> The drawback is that new code is thrust on unsuspecting users, > >> potentially exposing them to bugs that we have not found in our > >> somewhat limited web serving workload. > > > > This is my vote. > > > > I assume that the in tree and ports tree OpenSSL libraries will make > > use of it when present? Does this mean fetch and the like will also > > use it when talking w/ https website? (that's a nice benefit). > > In tree OpenSSL does not support KTLS. OpenSSL considers KTLS support > too large of a feature to officially backport to the 1.1.1 branch, so > if we add it in base, it will mean keeping it as a local diff. > > OTOH, I do maintain a backport of KTLS to 1.1.1 and there is a KTLS > option for the security/openssl port (not on by default, it perhaps > should be on 13?) which includes KTLS support. security/openssl-devel > (which tracks OpenSSL 3) also has a KTLS option that probably should > be enabled by default on 13 as it only consists of enabling the > option without requiring patches to the port. > > I can raise the issue again with secteam about importing KTLS into the > base OpenSSL. I think the main issue is the risk of getting a merge > conflict when merging in an SA, though from my experience maintaining > the KTLS patchset against 1.1.1 for the past year or so, I expect that > risk to be fairly low. Considering that 1.1.1 support will end long before the support time of 13-current ends, that's only two+ years of work to merge supported patches, then we're on our own anyways.. > Personally, it would make my life a bit happier as a developer using > KTLS for it to at least be in GENERIC by default, but that's a pretty > narrow use case. :) I forget about the OpenSSL status in ports, do all ports that use OpenSSL use ports OpenSSL? I guess not, because git-lite didn't install OpenSSL, but supports https... If none(almost none) of the FreeBSD software (or ports) uses it by default, then my vote changes to 3, which is to not enable it. If you have to do all this work to get software to use KTLS, checking out the ports tree and compiling custom ports, etc, then you're already far enough along the path that recompiling the kernel isn't that big of a stretch, and it saves the kernel code space, and the security risk of another API... Compiling a kernel w/ it really isn't THAT hard... cat > sys/amd64/conf/KTLS <