Date: Wed, 20 Jan 2021 01:36:12 GMT From: Mark Johnston <markj@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: 98d788c867b9 - main - opencrypto: Fix assignment of crypto completions to worker threads Message-ID: <202101200136.10K1aCT9089669@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=98d788c867b9e1d7a7e290254443b87ea77d8ab1 commit 98d788c867b9e1d7a7e290254443b87ea77d8ab1 Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2021-01-20 01:34:35 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2021-01-20 01:34:35 +0000 opencrypto: Fix assignment of crypto completions to worker threads Since r336439 we simply take the session pointer value mod the number of worker threads (ncpu by default). On small systems this ends up funneling all completion work through a single thread, which becomes a bottleneck when processing IPSec traffic using hardware crypto drivers. (Software drivers such as aesni(4) are unaffected since they invoke completion handlers synchonously.) Instead, maintain an incrementing counter with a unique value per session, and use that to distribute work to completion threads. Reviewed by: cem, jhb MFC after: 2 weeks Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D28159 --- sys/opencrypto/crypto.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sys/opencrypto/crypto.c b/sys/opencrypto/crypto.c index 7bc230140568..fc4169fb2365 100644 --- a/sys/opencrypto/crypto.c +++ b/sys/opencrypto/crypto.c @@ -135,6 +135,7 @@ static int crypto_drivers_size = 0; struct crypto_session { struct cryptocap *cap; struct crypto_session_params csp; + uint64_t id; }; /* @@ -908,6 +909,7 @@ int crypto_newsession(crypto_session_t *cses, const struct crypto_session_params *csp, int crid) { + static uint64_t sessid = 0; crypto_session_t res; struct cryptocap *cap; int err; @@ -944,6 +946,7 @@ crypto_newsession(crypto_session_t *cses, M_WAITOK | M_ZERO); res->cap = cap; res->csp = *csp; + res->id = atomic_fetchadd_64(&sessid, 1); /* Call the driver initialization routine. */ err = CRYPTODEV_NEWSESSION(cap->cc_dev, res, csp); @@ -1416,7 +1419,7 @@ crypto_dispatch(struct cryptop *crp) CRYPTOSTAT_INC(cs_ops); - crp->crp_retw_id = ((uintptr_t)crp->crp_session) % crypto_workers_num; + crp->crp_retw_id = crp->crp_session->id % crypto_workers_num; if (CRYPTOP_ASYNC(crp)) { if (crp->crp_flags & CRYPTO_F_ASYNC_KEEPORDER) {
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202101200136.10K1aCT9089669>