From owner-freebsd-arch@freebsd.org Sat Jan 9 00:51:58 2021 Return-Path: Delivered-To: freebsd-arch@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B7E834DB8AD for ; Sat, 9 Jan 2021 00:51:58 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4DCLyj57Sqz3m7G; Sat, 9 Jan 2021 00:51:57 +0000 (UTC) (envelope-from gallatin@cs.duke.edu) Received: from [192.168.1.2] (pool-74-110-137-7.rcmdva.fios.verizon.net [74.110.137.7]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: gallatin) by duke.cs.duke.edu (Postfix) with ESMTPSA id EA6412700131; Fri, 8 Jan 2021 19:51:55 -0500 (EST) DMARC-Filter: OpenDMARC Filter v1.3.1 duke.cs.duke.edu EA6412700131 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=cs.duke.edu; s=mail0816; t=1610153517; bh=QaMBv4m1XlUBfbAaIm/mijIPJhQUYAhnpTLXWuJ7tNY=; h=Subject:To:From:Date:From; b=mQxLYUrIKXAwZ3BXGYL/GewJzgFQILOMtqZL3QwkSYDnMEKdtOuzxBwK6ys+iSjMC HXZU2ZrXtHtqKnEwy97U6ys1eBjj6LrIZsjMpr8n0ddmIGtc2aR7Eb7A9pJPtH0QBg gawlEA85CvY6buXwiXs4NYXMvjfEgp1xt5Gvp+Z864+lSfaheUA3RHUJXZzaVPmeQL 0M8JKD7YkUaFC87L524uUWlMnLHJ8hAbKf3NjbMKcMZyc1VwJWLiS1GljGvpYdfWhj fEpX/jKC4ula1c4SfPWFFDOaQvq7AgJR2cGyuHZyli5ZDauxvPWW1ysDE6KsUG1pSQ oSR55Z69+4eKw== Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? To: freebsd-arch@FreeBSD.org, Rick Macklem , Allan Jude References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> <20210108214446.GJ31099@funkthat.com> From: Andrew Gallatin Message-ID: Date: Fri, 8 Jan 2021 19:51:55 -0500 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:78.0) Gecko/20100101 Thunderbird/78.6.0 MIME-Version: 1.0 In-Reply-To: <20210108214446.GJ31099@funkthat.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4DCLyj57Sqz3m7G X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cs.duke.edu header.s=mail0816 header.b=mQxLYUrI; dmarc=pass (policy=none) header.from=cs.duke.edu; spf=pass (mx1.freebsd.org: domain of gallatin@cs.duke.edu designates 152.3.140.1 as permitted sender) smtp.mailfrom=gallatin@cs.duke.edu X-Spamd-Result: default: False [-4.10 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:152.3.140.0/23]; DKIM_TRACE(0.00)[cs.duke.edu:+]; DMARC_POLICY_ALLOW(-0.50)[cs.duke.edu,none]; NEURAL_HAM_SHORT(-1.00)[-1.000]; RECEIVED_SPAMHAUS_PBL(0.00)[74.110.137.7:received]; SUBJECT_ENDS_QUESTION(1.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[152.3.140.1:from]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:13371, ipnet:152.3.128.0/17, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[cs.duke.edu:s=mail0816]; FREEFALL_USER(0.00)[gallatin]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; DWL_DNSWL_LOW(-1.00)[duke.edu:dkim]; SPAMHAUS_ZRD(0.00)[152.3.140.1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_LOW(-0.10)[152.3.140.1:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-arch] X-Mailman-Approved-At: Sat, 09 Jan 2021 07:53:29 +0000 X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2021 00:51:58 -0000 On 1/8/21 4:44 PM, John-Mark Gurney wrote: > Andrew Gallatin wrote this message on Fri, Jan 08, 2021 at 12:26 -0500: >> Kernel TLS (KTLS) support was added roughly a year ago, and provides >> an efficient software or hardware accelerated path to have the kernel >> (or the NIC) handle TLS crypto. This is quite useful for web and >> NFS servers, and provides a huge (2x -> 5x) efficiency gain by >> avoiding data copies into userspace for crypto, and potentially >> offloading the crypto to hardware. >> >> >> KTLS is well tested on amd64, having been used in production at Netflix >> for nearly 4 years. The vast majority of Netflix video has been served >> via KTLS for the last few years. Its what has allowed us to serve >> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve >> nearly 400Gb/s on AMD servers with NICs which support crypto offload. >> >> I have received a few requests to enable it by default in GENERIC, and >> I'd like to get some opinions. >> >> There are essentially 3 options >> >> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and >> flipping kern.ipc.tls.enable=1 >> >> The advantage of this is that it "just works" out of the box for users, >> and for reviewers. >> >> The drawback is that new code is thrust on unsuspecting users, >> potentially exposing them to bugs that we have not found in our >> somewhat limited web serving workload. > > This is my vote. > > I assume that the in tree and ports tree OpenSSL libraries will make > use of it when present? Does this mean fetch and the like will also > use it when talking w/ https website? (that's a nice benefit). It is enabled in the ports openssl. We (Netflix) have patches to the base openssl, but I believe that it was decided we don't want to diverge, so its not in base. > IMO, this is the best option for at least 13-current (we can revisit > this before 13.0-R happens), and preferably for 13.0-R. W/ both a > kernel option and a sysctl to disable, we have a way to address issues, > and getting code being used and tested is the best way to make it > stable, and shaking out any remaining bugs. > Awesome, thanks for the feedback! Drew