From owner-freebsd-current@freebsd.org Mon Mar 22 17:02:48 2021 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 62A465B55A4 for ; Mon, 22 Mar 2021 17:02:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4F415h2HWTz4Sj1 for ; Mon, 22 Mar 2021 17:02:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from John-Baldwins-MacBook-Pro.local (ralph.baldwin.cx [66.234.199.215]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: jhb) by smtp.freebsd.org (Postfix) with ESMTPSA id 1249FB5F7 for ; Mon, 22 Mar 2021 17:02:47 +0000 (UTC) (envelope-from jhb@FreeBSD.org) To: freebsd-current@freebsd.org References: <20210311031501.GP56617@kduck.mit.edu> From: John Baldwin Subject: Re: Getting started with ktls Message-ID: <4ad085bd-1e90-2ab6-1232-153618fcb7e2@FreeBSD.org> Date: Mon, 22 Mar 2021 10:02:44 -0700 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Mar 2021 17:02:48 -0000 On 3/18/21 8:31 AM, tech-lists wrote: > On Wed, Mar 17, 2021 at 08:39:02PM +0000, Rick Macklem wrote: > >> Make sure you've done the following: >> ktls_ocf - is loaded >> these sysctls are set to 1 >> kern.ipc.tls.enable >> kern.ipc.mb_use_ext_pgs > > [on stable/13] > > % sysctl kern.ipc.tls.enable kern.ipc.mb_use_ext_pgs > kern.ipc.tls.enable: 1 > kern.ipc.mb_use_ext_pgs: 1 > > % kldstat | grep ktls > 7 1 0xffff000001353000 25520 ktls_ocf.ko > % > > % sysctl -a | fgrep kern.ipc.tls.stats > kern.ipc.tls.stats.ocf.retries: 0 > kern.ipc.tls.stats.ocf.separate_output: 0 > kern.ipc.tls.stats.ocf.inplace: 0 > kern.ipc.tls.stats.ocf.tls13_gcm_crypts: 0 > kern.ipc.tls.stats.ocf.tls12_gcm_crypts: 0 > kern.ipc.tls.stats.ocf.tls11_cbc_crypts: 0 > kern.ipc.tls.stats.ocf.tls10_cbc_crypts: 0 > kern.ipc.tls.stats.switch_failed: 0 > kern.ipc.tls.stats.switch_to_sw: 0 > kern.ipc.tls.stats.switch_to_ifnet: 0 > kern.ipc.tls.stats.failed_crypto: 0 > kern.ipc.tls.stats.corrupted_records: 0 > kern.ipc.tls.stats.active: 0 > kern.ipc.tls.stats.enable_calls: 535 > kern.ipc.tls.stats.offload_total: 0 > kern.ipc.tls.stats.sw_rx_inqueue: 0 > kern.ipc.tls.stats.sw_tx_inqueue: 0 > kern.ipc.tls.stats.threads: 4 > % FYI, you can do this a bit more efficiently with just 'sysctl kern.ipc.tls.stats' The 'enable_calls' means that OpenSSL is trying to offload connections, but those attempts are all failing (offload_total is a count of how many of those setsockopt() calls succeed). If you are familiar with dtrace, you can use some DTrace probes to see why 'ktls_enable_tx' and 'ktls_enable_rx' are failing, or barring that printf. For example, does ktls_create_session() fail, or does ktls_try_sw() fail? It is probably easiest to debug this using a userland application using openssl than trying NFS over TLS. -- John Baldwin