Date: Thu, 11 Mar 2021 15:42:55 +0000 From: Rick Macklem <rmacklem@uoguelph.ca> To: Alan Somers <asomers@freebsd.org>, Benjamin Kaduk <kaduk@mit.edu> Cc: FreeBSD CURRENT <freebsd-current@freebsd.org> Subject: Re: Getting started with ktls Message-ID: <YQXPR0101MB096899D3D2241D0D6D830227DD909@YQXPR0101MB0968.CANPRD01.PROD.OUTLOOK.COM> In-Reply-To: <CAOtMX2hApCJuTe8OqEJmjrj9vffLB%2BM%2Bc5qR=iPrhRnbeZf=jQ@mail.gmail.com> References: <CAOtMX2ggNtsEQz7TinyHciqsgzUSjcdvMDb1oORKHtMBnzTELw@mail.gmail.com> <20210311003136.GM56617@kduck.mit.edu> <CAOtMX2iKtBAQWRzY1K9twAFrtdH=S559J6Zd%2Bm5D-YHHPVYf7g@mail.gmail.com> <20210311031501.GP56617@kduck.mit.edu>, <CAOtMX2hApCJuTe8OqEJmjrj9vffLB%2BM%2Bc5qR=iPrhRnbeZf=jQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I'm going to cheat and top post (the discussion looks
pretty convoluted).
- The kernel must be built with "options KERN_TLS"
- OpenSSL must be built with KTLS enabled
- These two sysctls need to be set to 1
kern.ipc.tls.enable
kern.ipc.mb_use_ext_pgs
then it all happens "behind the curtain" in the
OpenSSL libraries.
To find out if it is enabled, do the following after
the handshake. (SSL_connect() or SSL_accept() or ??)
ret =3D BIO_get_ktls_send(SSL_get_wbio(ssl));
if (ret !=3D 0)
ret =3D BIO_get_ktls_recv(SSL_get_rbio(ssl));
if (ret !=3D 0)
/* KTLS enabled for both send and recv. */
The calls return non-zero if it is enabled for that direction.
You'll need it to use TLS 1.2 if you want both directions
to work at this time.
(The code is in usr.sbin/rpc.tlsclntd and usr.sbin/rpc.tlsservd.
Much easier to look at than man pages imho.)
--> Now you can sosend()/soreceive() on the socket
in the kernel. If your data is already in mbufs, then
they must be M_EXTPG mbufs. There is a function
that copies an mbuf chain into M_EXTPG mbufs,
but I can't remember what I called it.
rick
________________________________________
From: owner-freebsd-current@freebsd.org <owner-freebsd-current@freebsd.org>=
on behalf of Alan Somers <asomers@freebsd.org>
Sent: Wednesday, March 10, 2021 10:55 PM
To: Benjamin Kaduk
Cc: FreeBSD CURRENT
Subject: Re: Getting started with ktls
CAUTION: This email originated from outside of the University of Guelph. Do=
not click links or open attachments unless you recognize the sender and kn=
ow the content is safe. If in doubt, forward suspicious emails to IThelp@uo=
guelph.ca
On Wed, Mar 10, 2021 at 8:15 PM Benjamin Kaduk <kaduk@mit.edu> wrote:
> On Wed, Mar 10, 2021 at 06:17:42PM -0700, Alan Somers wrote:
> > On Wed, Mar 10, 2021 at 5:31 PM Benjamin Kaduk <kaduk@mit.edu> wrote:
> >
> > > On Wed, Mar 10, 2021 at 05:18:24PM -0700, Alan Somers wrote:
> > > > I'm trying to make ktls work with "zfs send/recv" to substantially
> reduce
> > > > the CPU utilization of applications like zrepl. But I have a few
> > > questions:
> > > >
> > > > * ktls(4)'s "Transmit" section says "Once TLS transmit is enabled b=
y
> a
> > > > successful set of the TCP_TXTLS_ENABLE socket option", but the
> "Supported
> > > > Libraries" section says "Applications using a supported library
> should
> > > > generally work with ktls without any changes". These sentences see=
m
> to
> > > be
> > > > contradictory. I think it means that the TCP_TXTLS_ENABLE option i=
s
> > > > necessary, but OpenSSL sets it automatically?
> > >
> > > Yes, OpenSSL sets it automatically for the builtin socket and
> connection
> > > BIO classes. Applications using other BIO classes will need to do
> things
> > > manually (or implement the appropriate _ctrl() parameters for their B=
IO
> > > class).
> > >
> > > > * When using OpenSSL, the library will automatically call
> setsockopt(_,
> > > > TCP_TXTLS_ENABLE). But it swallows the error, if any. How is an
> > > > application to tell if ktls is enabled on a particular socket or
> OpenSSL
> > > > session?
> > >
> > > IIRC the lack of answer for this is part of why upstream OpenSSL
> doesn't
> > > have specific KTLS tests enabled in the automated test suite.
> > >
> >
> > getsockopt(_. TCP_TXTLS_ENABLE) returns ENOPROTOOPT. Is there any reas=
on
> > why it's not implemented? That might be the easiest way to check for t=
he
> > ktls status of an individual socket.
>
> I think that's probably more of a question for jhb than me. I don't know
> of a reason why not, but I do know that there is some desire to keep the
> functionality that openssl exposes roughly compatible between linux and
> FreeBSD KTLS. I don't know whether Linux has something similar.
>
> >
> > >
> > > > * From experiment, I can see that OpenSSL attempts to set
> > > > TCP_TXTLS_ENABLE. But it doesn't try to set TCP_RXTLS_ENABLE. Why
> not?
> > > > From reading ktls_start and ossl_statem_server_post_work, it looks
> like
> > > > maybe a single socket cannot have ktls enabled for both sending and
> > > > receiving at the same time. Is that true?
> > >
> > > No. They just get enabled separately, since change_cipher_state() is
> > > called separately for read and write transitions.
> > >
> >
> > Apologies if I'm too ignorant, but what is a transition in SSL-speak?
> This
> > is my first attempt at any kind of SSL programming. What I know from
> > ktrace is that TCP_RXTLS_ENABLE never gets set.
>
> Sorry! I'm pretty conversant with this stuff (I'm the security area
> director that is responsible for the IETF TLS working group) and don't
> always target the right level. Basically, for a decent encrypting protoc=
ol
> you want different encrytion keys for the read and write direction
> (whichever peer you are), and the TLS (1.3) handshake has a multi-stage k=
ey
> hierarchy to try to encrypt as much of it as possible. So, for example,
> the client will need to update it's encryption key for reading once it
> reads the ServerHello (and before reading the Encrypted Extensions)
> message, even though the keys the client uses for writing don't change at
> that time. Internally, OpenSSL implements this "transition" of key
> material with a change_cipher_state() abstraction, that takes a flags
> argument (`which`). The flags indicate which set of keys to update, and
> which direction (read or write). So, by my read of the code, what's
> *supposed* to happen is that we call:
>
> if (BIO_set_ktls(bio, &crypto_info, which & SSL3_CC_WRITE))
>
> And if SSL3_CC_WRITE is set, that translates to calling BIO_set_ktls() wi=
th
> an `is_txt` value that evaluates to true; otherwise, `is_txt` is false,
> which corresponds to the RX case that you're failing to see happen.
>
> Just to get the boring stuff out of the way: what version of openssl are
> you testing against, and did you verify that OPENSSL_NO_KTLS_RX is not
> defined when ktls_start() is being compiled (so that the setsockopt(fd,
> IPPROTO_TCP, TCP_RXTLS_ENABLE, .) is compiled in at all)?
>
> Thanks,
>
> Ben
>
I'm using the OpenSSL that's in base in 14.0-CURRENT: 1.1.1j-freebsd . I
haven't recompiled the code to check whether OPENSSL_NO_KTLS_RX is defined,
but it sure looks like it shouldn't be, based on my reading of the source.
-Alan
_______________________________________________
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?YQXPR0101MB096899D3D2241D0D6D830227DD909>