Date: Fri, 8 Jan 2021 18:24:09 -0800 From: John-Mark Gurney <jmg@funkthat.com> To: John Baldwin <jhb@FreeBSD.org> Cc: Andrew Gallatin <gallatin@cs.duke.edu>, freebsd-arch@FreeBSD.org, Rick Macklem <rmacklem@uoguelph.ca>, Allan Jude <allanjude@freebsd.org> Subject: Re: Should we enable KERN_TLS on amd64 for FreeBSD 13? Message-ID: <20210109022409.GL31099@funkthat.com> In-Reply-To: <4fe4a57c-8c43-a677-4872-d0671104c414@FreeBSD.org> References: <8eff83e5-49bc-d410-626e-603c03877b80@cs.duke.edu> <20210108214446.GJ31099@funkthat.com> <4fe4a57c-8c43-a677-4872-d0671104c414@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
John Baldwin wrote this message on Fri, Jan 08, 2021 at 17:03 -0800:
> On 1/8/21 1:44 PM, John-Mark Gurney wrote:
> > Andrew Gallatin wrote this message on Fri, Jan 08, 2021 at 12:26 -0500:
> >> Kernel TLS (KTLS) support was added roughly a year ago, and provides
> >> an efficient software or hardware accelerated path to have the kernel
> >> (or the NIC) handle TLS crypto. This is quite useful for web and
> >> NFS servers, and provides a huge (2x -> 5x) efficiency gain by
> >> avoiding data copies into userspace for crypto, and potentially
> >> offloading the crypto to hardware.
> >>
> >>
> >> KTLS is well tested on amd64, having been used in production at Netflix
> >> for nearly 4 years. The vast majority of Netflix video has been served
> >> via KTLS for the last few years. Its what has allowed us to serve
> >> 100Gb/s on Xeon 2697A cpus for years, and what allows us to serve
> >> nearly 400Gb/s on AMD servers with NICs which support crypto offload.
> >>
> >> I have received a few requests to enable it by default in GENERIC, and
> >> I'd like to get some opinions.
> >>
> >> There are essentially 3 options
> >>
> >> 1) Fully enable KTLS by adding 'options KERN_TLS' to GENERIC, and
> >> flipping kern.ipc.tls.enable=1
> >>
> >> The advantage of this is that it "just works" out of the box for users,
> >> and for reviewers.
> >>
> >> The drawback is that new code is thrust on unsuspecting users,
> >> potentially exposing them to bugs that we have not found in our
> >> somewhat limited web serving workload.
> >
> > This is my vote.
> >
> > I assume that the in tree and ports tree OpenSSL libraries will make
> > use of it when present? Does this mean fetch and the like will also
> > use it when talking w/ https website? (that's a nice benefit).
>
> In tree OpenSSL does not support KTLS. OpenSSL considers KTLS support
> too large of a feature to officially backport to the 1.1.1 branch, so
> if we add it in base, it will mean keeping it as a local diff.
>
> OTOH, I do maintain a backport of KTLS to 1.1.1 and there is a KTLS
> option for the security/openssl port (not on by default, it perhaps
> should be on 13?) which includes KTLS support. security/openssl-devel
> (which tracks OpenSSL 3) also has a KTLS option that probably should
> be enabled by default on 13 as it only consists of enabling the
> option without requiring patches to the port.
>
> I can raise the issue again with secteam about importing KTLS into the
> base OpenSSL. I think the main issue is the risk of getting a merge
> conflict when merging in an SA, though from my experience maintaining
> the KTLS patchset against 1.1.1 for the past year or so, I expect that
> risk to be fairly low.
Considering that 1.1.1 support will end long before the support time of
13-current ends, that's only two+ years of work to merge supported
patches, then we're on our own anyways..
> Personally, it would make my life a bit happier as a developer using
> KTLS for it to at least be in GENERIC by default, but that's a pretty
> narrow use case. :)
I forget about the OpenSSL status in ports, do all ports that use
OpenSSL use ports OpenSSL? I guess not, because git-lite didn't
install OpenSSL, but supports https...
If none(almost none) of the FreeBSD software (or ports) uses it by
default, then my vote changes to 3, which is to not enable it. If you
have to do all this work to get software to use KTLS, checking out the
ports tree and compiling custom ports, etc, then you're already far
enough along the path that recompiling the kernel isn't that big of a
stretch, and it saves the kernel code space, and the security risk
of another API...
Compiling a kernel w/ it really isn't THAT hard...
cat > sys/amd64/conf/KTLS <<EOF
include GENERIC
options KERN_TLS
EOF
make buildkernel KERNCONF=KTLS -j 4
make installkernel KERNCONF=KTLS -j 4
--
John-Mark Gurney Voice: +1 415 225 5579
"All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20210109022409.GL31099>