Date: Wed, 3 Feb 2021 14:37:25 GMT From: Mark Johnston <markj@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Subject: git: 10ed8ab4ab9b - stable/12 - opencrypto: Fix assignment of crypto completions to worker threads Message-ID: <202102031437.113EbPve001733@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch stable/12 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=10ed8ab4ab9bd0239f2913ac2f35af9b9f76221d commit 10ed8ab4ab9bd0239f2913ac2f35af9b9f76221d Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2021-01-20 01:34:35 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2021-02-03 14:37:17 +0000 opencrypto: Fix assignment of crypto completions to worker threads Since r336439 we simply take the session pointer value mod the number of worker threads (ncpu by default). On small systems this ends up funneling all completion work through a single thread, which becomes a bottleneck when processing IPSec traffic using hardware crypto drivers. (Software drivers such as aesni(4) are unaffected since they invoke completion handlers synchonously.) Instead, maintain an incrementing counter with a unique value per session, and use that to distribute work to completion threads. Reviewed by: cem, jhb Sponsored by: Rubicon Communications, LLC ("Netgate") Differential Revision: https://reviews.freebsd.org/D28159 (cherry picked from commit 98d788c867b9e1d7a7e290254443b87ea77d8ab1) --- sys/opencrypto/crypto.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/sys/opencrypto/crypto.c b/sys/opencrypto/crypto.c index dfd22662e87f..bc23056e86ad 100644 --- a/sys/opencrypto/crypto.c +++ b/sys/opencrypto/crypto.c @@ -94,6 +94,7 @@ struct crypto_session { void *softc; uint32_t hid; uint32_t capabilities; + uint64_t id; }; SDT_PROVIDER_DEFINE(opencrypto); @@ -572,6 +573,7 @@ again: int crypto_newsession(crypto_session_t *cses, struct cryptoini *cri, int crid) { + static uint64_t sessid = 0; crypto_session_t res; void *softc_mem; struct cryptocap *cap; @@ -616,6 +618,7 @@ restart: softc_mem = malloc(softc_size, M_CRYPTO_DATA, M_WAITOK | M_ZERO); res = uma_zalloc(cryptoses_zone, M_WAITOK | M_ZERO); res->softc = softc_mem; + res->id = atomic_fetchadd_64(&sessid, 1); CRYPTO_DRIVER_LOCK(); cap = crypto_checkdriver(hid); @@ -1016,7 +1019,7 @@ crypto_dispatch(struct cryptop *crp) binuptime(&crp->crp_tstamp); #endif - crp->crp_retw_id = ((uintptr_t)crp->crp_session) % crypto_workers_num; + crp->crp_retw_id = crp->crp_session->id % crypto_workers_num; if (CRYPTOP_ASYNC(crp)) { if (crp->crp_flags & CRYPTO_F_ASYNC_KEEPORDER) {
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202102031437.113EbPve001733>