From owner-freebsd-current@freebsd.org Thu Mar 11 18:49:13 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C053D5AA030 for ; Thu, 11 Mar 2021 18:49:13 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DxHzY54Xgz3kyM; Thu, 11 Mar 2021 18:49:13 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 50119F7FD; Thu, 11 Mar 2021 18:49:13 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Subject: Re: Getting started with ktls To: Alan Somers , FreeBSD CURRENT References: From: John Baldwin Message-ID: <24d697e1-1232-7b53-923c-5ba39c6d8d80@FreeBSD.org> Date: Thu, 11 Mar 2021 10:49:11 -0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Mar 2021 18:49:13 -0000 On 3/10/21 4:18 PM, Alan Somers wrote: > I'm trying to make ktls work with "zfs send/recv" to substantially reduce > the CPU utilization of applications like zrepl. But I have a few questions: > > * ktls(4)'s "Transmit" section says "Once TLS transmit is enabled by a > successful set of the TCP_TXTLS_ENABLE socket option", but the "Supported > Libraries" section says "Applications using a supported library should > generally work with ktls without any changes". These sentences seem to be > contradictory. I think it means that the TCP_TXTLS_ENABLE option is > necessary, but OpenSSL sets it automatically? Yes, you can do it by hand if you want but you'd have to do all the key exchange by hand as well. > * When using OpenSSL, the library will automatically call setsockopt(_, > TCP_TXTLS_ENABLE). But it swallows the error, if any. How is an > application to tell if ktls is enabled on a particular socket or OpenSSL > session? BIO_get_ktls_send() and BIO_get_ktls_recv() on the write and read BIO's of the connection, respectively. > * From experiment, I can see that OpenSSL attempts to set > TCP_TXTLS_ENABLE. But it doesn't try to set TCP_RXTLS_ENABLE. Why not? > From reading ktls_start and ossl_statem_server_post_work, it looks like > maybe a single socket cannot have ktls enabled for both sending and > receiving at the same time. Is that true? Neither FreeBSD nor OpenSSL yet support RX offload on TLS 1.3. If you use TLS 1.2 you will get KTLS in both directions (or if you use TLS 1.1 with TOE offload on a Chelsio T6). -- John Baldwin