From owner-dev-commits-src-main@freebsd.org Wed Jan 20 01:36:12 2021
Return-Path: <owner-dev-commits-src-main@freebsd.org>
Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.nyi.freebsd.org (Postfix) with ESMTP id 70CFE4E5642;
Wed, 20 Jan 2021 01:36:12 +0000 (UTC) (envelope-from git@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4DL7Qh2qh1z3Fhv;
Wed, 20 Jan 2021 01:36:12 +0000 (UTC) (envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org (gitrepo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:5])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 540BE11076;
Wed, 20 Jan 2021 01:36:12 +0000 (UTC) (envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org ([127.0.1.44])
by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 10K1aCb2089670;
Wed, 20 Jan 2021 01:36:12 GMT (envelope-from git@gitrepo.freebsd.org)
Received: (from git@localhost)
by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 10K1aCT9089669;
Wed, 20 Jan 2021 01:36:12 GMT (envelope-from git)
Date: Wed, 20 Jan 2021 01:36:12 GMT
Message-Id: <202101200136.10K1aCT9089669@gitrepo.freebsd.org>
To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org,
dev-commits-src-main@FreeBSD.org
From: Mark Johnston <markj@FreeBSD.org>
Subject: git: 98d788c867b9 - main - opencrypto: Fix assignment of crypto
completions to worker threads
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Git-Committer: markj
X-Git-Repository: src
X-Git-Refname: refs/heads/main
X-Git-Reftype: branch
X-Git-Commit: 98d788c867b9e1d7a7e290254443b87ea77d8ab1
Auto-Submitted: auto-generated
X-BeenThere: dev-commits-src-main@freebsd.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: Commit messages for the main branch of the src repository
<dev-commits-src-main.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/dev-commits-src-main>,
<mailto:dev-commits-src-main-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/dev-commits-src-main/>
List-Post: <mailto:dev-commits-src-main@freebsd.org>
List-Help: <mailto:dev-commits-src-main-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main>,
<mailto:dev-commits-src-main-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 20 Jan 2021 01:36:12 -0000
The branch main has been updated by markj:
URL: https://cgit.FreeBSD.org/src/commit/?id=98d788c867b9e1d7a7e290254443b87ea77d8ab1
commit 98d788c867b9e1d7a7e290254443b87ea77d8ab1
Author: Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2021-01-20 01:34:35 +0000
Commit: Mark Johnston <markj@FreeBSD.org>
CommitDate: 2021-01-20 01:34:35 +0000
opencrypto: Fix assignment of crypto completions to worker threads
Since r336439 we simply take the session pointer value mod the number of
worker threads (ncpu by default). On small systems this ends up
funneling all completion work through a single thread, which becomes a
bottleneck when processing IPSec traffic using hardware crypto drivers.
(Software drivers such as aesni(4) are unaffected since they invoke
completion handlers synchonously.)
Instead, maintain an incrementing counter with a unique value per
session, and use that to distribute work to completion threads.
Reviewed by: cem, jhb
MFC after: 2 weeks
Sponsored by: Rubicon Communications, LLC ("Netgate")
Differential Revision: https://reviews.freebsd.org/D28159
---
sys/opencrypto/crypto.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/sys/opencrypto/crypto.c b/sys/opencrypto/crypto.c
index 7bc230140568..fc4169fb2365 100644
--- a/sys/opencrypto/crypto.c
+++ b/sys/opencrypto/crypto.c
@@ -135,6 +135,7 @@ static int crypto_drivers_size = 0;
struct crypto_session {
struct cryptocap *cap;
struct crypto_session_params csp;
+ uint64_t id;
};
/*
@@ -908,6 +909,7 @@ int
crypto_newsession(crypto_session_t *cses,
const struct crypto_session_params *csp, int crid)
{
+ static uint64_t sessid = 0;
crypto_session_t res;
struct cryptocap *cap;
int err;
@@ -944,6 +946,7 @@ crypto_newsession(crypto_session_t *cses,
M_WAITOK | M_ZERO);
res->cap = cap;
res->csp = *csp;
+ res->id = atomic_fetchadd_64(&sessid, 1);
/* Call the driver initialization routine. */
err = CRYPTODEV_NEWSESSION(cap->cc_dev, res, csp);
@@ -1416,7 +1419,7 @@ crypto_dispatch(struct cryptop *crp)
CRYPTOSTAT_INC(cs_ops);
- crp->crp_retw_id = ((uintptr_t)crp->crp_session) % crypto_workers_num;
+ crp->crp_retw_id = crp->crp_session->id % crypto_workers_num;
if (CRYPTOP_ASYNC(crp)) {
if (crp->crp_flags & CRYPTO_F_ASYNC_KEEPORDER) {