Date: Fri, 8 Jan 2021 12:26:38 -0500 From: Andrew Gallatin <gallatin@cs.duke.edu> To: freebsd-arch@FreeBSD.org Cc: Allan Jude <allanjude@freebsd.org>, John Baldwin <jhb@freebsd.org>, Rick Macklem <rmacklem@uoguelph.ca> Subject: Should we enable KERN_TLS on amd64 for FreeBSD 13? Message-ID: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu>
next in thread | raw e-mail | index | archive | help
Kernel TLS (KTLS) support was added roughly a year ago, and provides an efficient software or hardware accelerated path to have the kernel (or the NIC) handle TLS crypto. This is quite useful for web and NFS servers, and provides a huge (2x -> 5x) efficiency gain by avoiding data copies into userspace for crypto, and potentially offloading the crypto to hardware. KTLS is well tested on amd64, having been used in production at Netflix for nearly 4 years. The vast majority of Netflix video has been served via KTLS for the last few years. Its what has allowed us to serve 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve nearly 400Gb/s on AMD servers with NICs which support crypto offload. I have received a few requests to enable it by default in GENERIC, and I'd like to get some opinions. There are essentially 3 options 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and flipping kern.ipc.tls.enable=1 The advantage of this is that it "just works" out of the box for users, and for reviewers. The drawback is that new code is thrust on unsuspecting users, potentially exposing them to bugs that we have not found in our somewhat limited web serving workload. 2) Enable KTLS in GENERIC, but leave it turned off by default. This option allows users to enable ktls without a rebuild of GENERIC, but does not enable it by default. So they can enable it if they know about it, but are protected from bugs. The disadvantages of this are that it increases the kernel size by ~20K, starts up one thread per core on every amd64 machine, and it adds more required tuning to get good performance from FreeBSD. 3) Continue along with KTLS disabled in GENERIC This is the lowest risk, but adds a higher bar for users wanting to use ktls. Note that the discussion is focused on amd64 only, as KTLS will only work on 64-bit platforms which use a direct map. It has not been tested at all on ppc64, and currently causes a panic-at-boot on arm64 due to what are suspected to be problems in the arm64 PCB setup. See: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247945 Drew
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8eff83e5-49bc-d410-626e-603c03877b80>