From owner-freebsd-arch@freebsd.org Sat Jan 9 22:37:25 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 908644E50D0 for ; Sat, 9 Jan 2021 22:37:25 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCvx064bqz4b8D; Sat, 9 Jan 2021 22:37:24 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from [192.168.1.2] (pool-74-110-137-7.rcmdva.fios.verizon.net [74.110.137.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: gallatin) by duke.cs.duke.edu (Postfix) with ESMTPSA id E92B927000BE; Sat, 9 Jan 2021 17:37:23 -0500 (EST) DMARC-Filter: OpenDMARC Filter v1.3.1 duke.cs.duke.edu E92B927000BE DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail0816; t=1610231844; bh=Vm3ajYWPDycBO0K0zKoi7pd5uQEgEgILkeZzFa3BpXQ=; h=Subject:To:From:Date:From; b=K0/qvazpW3QyBlHAeL2LcjvVtxngePuEkVmKIeh7HxnrmZrj8a1BTJNJvONhd94N+ K/XDR8FI8DBsQZ1l7DVJqlfgkOBu7BsmtZC4ZBvhUh57PmxK/5RYq1j1xRFaoxZsVR qeCT1kkyr+HAe62Vtqz6VZEZiOS6Dkgio6vror3/x7R/uqcBMtsnCaco0dv4RV33Si gJEuoAos5vrznES0yOzZrd4LVy1Fq6X8Mr4/CdpV0pHGNIGR+ansbCph3qZ2ajjJuu XzyA6g/fM857xHwQ1+ew9VYCA8avaCb8bhfRdPuSEJ1pVO9XcE09DZSOYX04A0+2hb W8sxQBH5xNwaA== Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: John Baldwin , freebsd-arch@FreeBSD.org, Rick Macklem , Allan Jude References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> <20210108214446.GJ31099@funkthat.com> <4fe4a57c-8c43-a677-4872-d0671104c414@FreeBSD.org> <20210109022409.GL31099@funkthat.com> From: Andrew Gallatin Message-ID: <993ebe97-d4b4-fe59-5b4f-9d607bb5e698@cs.duke.edu> Date: Sat, 9 Jan 2021 17:37:23 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <20210109022409.GL31099@funkthat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DCvx064bqz4b8D X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.duke.edu header.s=mail0816 header.b=K0/qvazp; dmarc=pass (policy=none) header.from=cs.duke.edu; spf=pass (mx1.freebsd.org: domain of gallatin@cs.duke.edu designates 152.3.140.1 as permitted sender) smtp.mailfrom=gallatin@cs.duke.edu X-Spamd-Result: default: False [-4.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:152.3.140.0/23]; DKIM_TRACE(0.00)[cs.duke.edu:+]; DMARC_POLICY_ALLOW(-0.50)[cs.duke.edu,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[74.110.137.7:received]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[152.3.140.1:from]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:13371, ipnet:152.3.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[cs.duke.edu:s=mail0816]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; SPAMHAUS_ZRD(0.00)[152.3.140.1:from:127.0.2.255]; DWL_DNSWL_LOW(-1.00)[duke.edu:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[152.3.140.1:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arch] X-Mailman-Approved-At: Sun, 10 Jan 2021 09:01:02 +0000 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2021 22:37:25 -0000 On 1/8/21 9:24 PM, John-Mark Gurney wrote: > John Baldwin wrote this message on Fri, Jan 08, 2021 at 17:03 -0800: >> On 1/8/21 1:44 PM, John-Mark Gurney wrote: >>> Andrew Gallatin wrote this message on Fri, Jan 08, 2021 at 12:26 -0500: >>>> Kernel TLS (KTLS) support was added roughly a year ago, and provides >>>> an efficient software or hardware accelerated path to have the kernel >>>> (or the NIC) handle TLS crypto. This is quite useful for web and >>>> NFS servers, and provides a huge (2x -> 5x) efficiency gain by >>>> avoiding data copies into userspace for crypto, and potentially >>>> offloading the crypto to hardware. >>>> >>>> >>>> KTLS is well tested on amd64, having been used in production at Netflix >>>> for nearly 4 years. The vast majority of Netflix video has been served >>>> via KTLS for the last few years. Its what has allowed us to serve >>>> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve >>>> nearly 400Gb/s on AMD servers with NICs which support crypto offload. >>>> >>>> I have received a few requests to enable it by default in GENERIC, and >>>> I'd like to get some opinions. >>>> >>>> There are essentially 3 options >>>> >>>> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and >>>> flipping kern.ipc.tls.enable=1 >>>> >>>> The advantage of this is that it "just works" out of the box for users, >>>> and for reviewers. >>>> >>>> The drawback is that new code is thrust on unsuspecting users, >>>> potentially exposing them to bugs that we have not found in our >>>> somewhat limited web serving workload. >>> >>> This is my vote. >>> >>> I assume that the in tree and ports tree OpenSSL libraries will make >>> use of it when present? Does this mean fetch and the like will also >>> use it when talking w/ https website? (that's a nice benefit). >> >> In tree OpenSSL does not support KTLS. OpenSSL considers KTLS support >> too large of a feature to officially backport to the 1.1.1 branch, so >> if we add it in base, it will mean keeping it as a local diff. >> >> OTOH, I do maintain a backport of KTLS to 1.1.1 and there is a KTLS >> option for the security/openssl port (not on by default, it perhaps >> should be on 13?) which includes KTLS support. security/openssl-devel >> (which tracks OpenSSL 3) also has a KTLS option that probably should >> be enabled by default on 13 as it only consists of enabling the >> option without requiring patches to the port. >> >> I can raise the issue again with secteam about importing KTLS into the >> base OpenSSL. I think the main issue is the risk of getting a merge >> conflict when merging in an SA, though from my experience maintaining >> the KTLS patchset against 1.1.1 for the past year or so, I expect that >> risk to be fairly low. > > Considering that 1.1.1 support will end long before the support time of > 13-current ends, that's only two+ years of work to merge supported > patches, then we're on our own anyways.. We (Netflix) have maintained patches to base openssl for several years, and I can recall only one tricky merge. But I think this ship has sailed. I'm not about to ask to make somebody else's life more difficult. >> Personally, it would make my life a bit happier as a developer using >> KTLS for it to at least be in GENERIC by default, but that's a pretty >> narrow use case. :) > > I forget about the OpenSSL status in ports, do all ports that use > OpenSSL use ports OpenSSL? I guess not, because git-lite didn't > install OpenSSL, but supports https... > > If none(almost none) of the FreeBSD software (or ports) uses it by > default, then my vote changes to 3, which is to not enable it. If you > have to do all this work to get software to use KTLS, checking out the > ports tree and compiling custom ports, etc, then you're already far > enough along the path that recompiling the kernel isn't that big of a > stretch, and it saves the kernel code space, and the security risk > of another API... > > Compiling a kernel w/ it really isn't THAT hard... > > cat > sys/amd64/conf/KTLS < include GENERIC > > options KERN_TLS > EOF > make buildkernel KERNCONF=KTLS -j 4 > make installkernel KERNCONF=KTLS -j 4 >