From nobody Wed Apr 24 13:53:53 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4VPgSf3Fbfz5HPk6; Wed, 24 Apr 2024 13:53:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4VPgSf14T7z4Spb; Wed, 24 Apr 2024 13:53:54 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713966834; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1mYrLTzgtlgzOxG+4BaOzDWvSnqJVoxrSDuEPUOOL8I=; b=YMYjMCFK9Src+Mh5PnOxOsUkf9fkTWEzQOiTKR7cQE1Nsm0/Fw7TLvKCKc4+sBk7E4a+Lt QUPwloD2+6hutG5TJX8WXNdjEJj4Bifjo/P5UyFH08GCiVNbMu5cGMeYjN7W8fIL7hKFN7 9c5Y8uFXd0BBOc/0SgKuM6wKxM6/9a4mGQ4bCgi6mPOocdVAaN5zTiIXCDkdbhOfo7Mrvl ou9dJJ/XacYcnXdJEWOyFtxjTKkxX62kXBsA7CZ6d6vTeuI32AiFAi6y0/32hlkq/KhnHq QLKrfzBkz6rm8s3d3sQpnzCZhMnKIPxX7YGLL6DAxoXs2mDqt0vh3TjMj48Z3w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1713966834; a=rsa-sha256; cv=none; b=sdzrXO54t8IzPWfupx6TXVmwJAvn5KhpaGCKRPardRRN4saFsF8KXU3hxpWtPXiWigU7B9 +ryPd/Ir4ZeQbVgL18f7GiXVDU/JFB72nkxeYNDkeKKXHYs1e4T3rlZ9cJ1SK+yRuMU3s7 QNjMVZTv6upZ/NutZjm7jBdJMVjUDMPgzujungPP1oXTsfoU3KdDIBzTGIP8736cxt0aqm QXQFd0ZuOX7SF0sH1h1wWEUsav/Ra/7jkRVVhllXMbcIlr4ifHlqJp+CY5Cegxx5Isx1W6 C823KxJqCktuWBEVrp9glxmsWoLDrcX8zfs+Iu5KfHKVbVDIVFU+Dyi2FJVqoQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1713966834; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1mYrLTzgtlgzOxG+4BaOzDWvSnqJVoxrSDuEPUOOL8I=; b=VuDMDfq9KIAoDopmxmla4eqHn4IaLkXeZTjb0EqDUyvHkMkHdyp9oyQI7pPbBIrh2r/XeQ 3Syr0aE2WaoduAuv8eqP8M8VhHThwakcZLb/972D0/CL2mcbf6+k0w3+rgClJf5mABC833 vYtxfOzAe6YPOCYS6gfekP7hN8V3+XOEpjg9c+MzofWFh8m7dBYzhlzFQ5PZpwttS+2Vr9 xu2r1uWurh5Wpiy2embvq2DmdZzMsrtYnhYpROYtZ/1BlLLIZfZiEs3fBDXHGeykUmi8HT /KP2KyGq9R2u0JNyZ7IVolkNzZbZghuud53rMlaPnFmXy5M1M+tYSEfK38Xx6A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4VPgSf0W4KzJ10; Wed, 24 Apr 2024 13:53:54 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 43ODrr8p020511; Wed, 24 Apr 2024 13:53:53 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 43ODrrPW020508; Wed, 24 Apr 2024 13:53:53 GMT (envelope-from git) Date: Wed, 24 Apr 2024 13:53:53 GMT Message-Id: <202404241353.43ODrrPW020508@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Somers Subject: git: 4b7949144cfb - main - geli.8: add some notes regarding performance tuning List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: asomers X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 4b7949144cfbf25b0e0bf593b135a88417304d71 Auto-Submitted: auto-generated The branch main has been updated by asomers: URL: https://cgit.FreeBSD.org/src/commit/?id=4b7949144cfbf25b0e0bf593b135a88417304d71 commit 4b7949144cfbf25b0e0bf593b135a88417304d71 Author: Alan Somers AuthorDate: 2024-04-22 21:52:07 +0000 Commit: Alan Somers CommitDate: 2024-04-24 13:53:37 +0000 geli.8: add some notes regarding performance tuning MFC after: 1 week Sponsored by: Axcient Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D44908 --- lib/geom/eli/geli.8 | 42 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 41 insertions(+), 1 deletion(-) diff --git a/lib/geom/eli/geli.8 b/lib/geom/eli/geli.8 index 501b948fde49..098ba4d0485e 100644 --- a/lib/geom/eli/geli.8 +++ b/lib/geom/eli/geli.8 @@ -22,7 +22,7 @@ .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF .\" SUCH DAMAGE. .\" -.Dd April 20, 2024 +.Dd April 24, 2024 .Dt GELI 8 .Os .Sh NAME @@ -851,6 +851,15 @@ This variable should be set in Specifies how many times the Master Key is overwritten with random values when it is destroyed. After this operation it is filled with zeros. +.It Va kern.geom.eli.use_uma_bytes +.Nm +must allocate a buffer for every write operation, used when performing +encryption. +This sysctl reports the maximum size in bytes for which geli will perform the +allocation using +.Xr UMA 9 , +as opposed to +.Xr malloc 9 . .It Va kern.geom.eli.visible_passphrase : No 0 If set to 1, the passphrase entered on boot will be visible. This alternative should be used with caution as the entered @@ -863,6 +872,9 @@ Specifies how many kernel threads should be used for doing software cryptography. Its purpose is to increase performance on SMP systems. If set to 0, a CPU-pinned thread will be started for every active CPU. +Note that this variable must be set prior to attaching +.Nm +to a disk. .It Va kern.geom.eli.batch : No 0 When set to 1, can speed-up crypto operations by using batching. Batching reduces the number of interrupts by responding to a group of @@ -891,6 +903,34 @@ This is an optimization which reduces the overhead of I/O processing. This variable is intended for debugging purposes and must be set in .Pa /boot/loader.conf . .El +.Sh PERFORMANCE CONSIDERATIONS +The default value of +.Va kern.geom.eli.threads +is usually good for a system with one SSD. +However, it may need to be lowered on systems with many disks, +so as to avoid creating too much thread-switching overhead. +On systems with more disks than CPUs, it's best to set this variable +to 1. +.Pp +.Nm +internally uses +.Xr malloc 9 +to allocate memory for operations larger than +.Va kern.geom.eli.use_uma_bytes , +but malloc is slow for allocations larger than +.Va vm.kmem_zmax . +So it's best to avoid writing more than +.Ms MAX(kern.geom.eli.use_uma_bytes, vm.kmem_zmax) +in a single write operation. +On systems that format +.Xr zfs 4 +on top of +.Nm , +the maximum write size can be controlled by +.Va vfs.zfs.vdev.aggregation_limit +and +.Va vfs.zfs.vdev.aggregation_limit_non_rotating +for HDDs and SSDs, respectively. .Sh EXIT STATUS Exit status is 0 on success, and 1 if the command fails. .Sh EXAMPLES