Date: Thu, 11 Mar 2021 15:54:48 -0700 From: Alan Somers <asomers@freebsd.org> To: John Baldwin <jhb@freebsd.org>, Rick Macklem <rmacklem@freebsd.org> Cc: FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: Getting started with ktls Message-ID: <CAOtMX2hfY%2BUAn-o_R4V_wi93uCi9YLJYK-UJTFrj_2L_HWAETA@mail.gmail.com> In-Reply-To: <24d697e1-1232-7b53-923c-5ba39c6d8d80@FreeBSD.org> References: <CAOtMX2ggNtsEQz7TinyHciqsgzUSjcdvMDb1oORKHtMBnzTELw@mail.gmail.com> <24d697e1-1232-7b53-923c-5ba39c6d8d80@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Mar 11, 2021 at 11:49 AM John Baldwin <jhb@freebsd.org> wrote: > On 3/10/21 4:18 PM, Alan Somers wrote: > > I'm trying to make ktls work with "zfs send/recv" to substantially reduce > > the CPU utilization of applications like zrepl. But I have a few > questions: > > > > * ktls(4)'s "Transmit" section says "Once TLS transmit is enabled by a > > successful set of the TCP_TXTLS_ENABLE socket option", but the "Supported > > Libraries" section says "Applications using a supported library should > > generally work with ktls without any changes". These sentences seem to > be > > contradictory. I think it means that the TCP_TXTLS_ENABLE option is > > necessary, but OpenSSL sets it automatically? > > Yes, you can do it by hand if you want but you'd have to do all the key > exchange by hand as well. > > > * When using OpenSSL, the library will automatically call setsockopt(_, > > TCP_TXTLS_ENABLE). But it swallows the error, if any. How is an > > application to tell if ktls is enabled on a particular socket or OpenSSL > > session? > > BIO_get_ktls_send() and BIO_get_ktls_recv() on the write and read BIO's of > the connection, respectively. > > > * From experiment, I can see that OpenSSL attempts to set > > TCP_TXTLS_ENABLE. But it doesn't try to set TCP_RXTLS_ENABLE. Why not? > > From reading ktls_start and ossl_statem_server_post_work, it looks like > > maybe a single socket cannot have ktls enabled for both sending and > > receiving at the same time. Is that true? > > Neither FreeBSD nor OpenSSL yet support RX offload on TLS 1.3. If you use > TLS 1.2 you will get KTLS in both directions (or if you use TLS 1.1 with > TOE offload on a Chelsio T6). > > -- > John Baldwin > Switching to TLS 1.2 turned out to be key. Once I did that, ... it just worked. I was expecting to need minor changes throughout the kernel and libzfs. However, that wasn't necessary. Here is my proof-of-concept program. So far only the recv path is implemented. I'll probably implement the send path too, but I'm not currently planning to integrate this into any open-source application. Thanks for all the help! https://github.com/asomers/freebsd-src/tree/ktls-zfs/tools/zfs-ktls -Alan
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2hfY%2BUAn-o_R4V_wi93uCi9YLJYK-UJTFrj_2L_HWAETA>