Date: Sun, 10 Jan 2021 11:43:34 -0500 From: Allan Jude <allanjude@freebsd.org> To: Andrew Gallatin <gallatin@cs.duke.edu>, freebsd-arch@FreeBSD.org Cc: John Baldwin <jhb@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca> Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? Message-ID: <db6dcef1-a71b-f7ca-33a7-5e15b357ffed@freebsd.org> In-Reply-To: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2021-01-08 12:26, Andrew Gallatin wrote: > > Kernel TLS (KTLS) support was added roughly a year ago, and provides > an efficient software or hardware accelerated path to have the kernel > (or the NIC) handle TLS crypto. This is quite useful for web and > NFS servers, and provides a huge (2x -> 5x) efficiency gain by > avoiding data copies into userspace for crypto, and potentially > offloading the crypto to hardware. > > > KTLS is well tested on amd64, having been used in production at Netflix > for nearly 4 years. The vast majority of Netflix video has been served > via KTLS for the last few years. Its what has allowed us to serve > 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve > nearly 400Gb/s on AMD servers with NICs which support crypto offload. > > I have received a few requests to enable it by default in GENERIC, and > I'd like to get some opinions. > > There are essentially 3 options > > 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and > flipping kern.ipc.tls.enable=1 > > The advantage of this is that it "just works" out of the box for users, > and for reviewers. > > The drawback is that new code is thrust on unsuspecting users, > potentially exposing them to bugs that we have not found in our > somewhat limited web serving workload. > I am in favour of just enabling it. As mentioned elsewhere in the thread, if it does cause a problem, we can switch the sysctl to default to off. I think we want to get this in ASAP, and then we can decide on the default value for the sysctl after we get broader user testing closer to 13.0-RELEASE > 2) Enable KTLS in GENERIC, but leave it turned off by default. > > This option allows users to enable ktls without a rebuild of GENERIC, > but does not enable it by default. So they can enable it if they > know about it, but are protected from bugs. > > The disadvantages of this are that it increases the kernel size > by ~20K, starts up one thread per core on every amd64 machine, > and it adds more required tuning to get good performance from FreeBSD. > > > 3) Continue along with KTLS disabled in GENERIC > > This is the lowest risk, but adds a higher bar for users wanting > to use ktls. > > > > Note that the discussion is focused on amd64 only, as KTLS will > only work on 64-bit platforms which use a direct map. It has > not been tested at all on ppc64, and currently causes a > panic-at-boot on arm64 due to what are suspected to be problems > in the arm64 PCB setup. See: > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945 > > Drew > -- Allan Jude
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?db6dcef1-a71b-f7ca-33a7-5e15b357ffed>