From nobody Tue Aug 6 04:06:39 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WdKW34hbTz5SbW7; Tue, 06 Aug 2024 04:06:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WdKW31RF7z4kbZ; Tue, 6 Aug 2024 04:06:39 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722917199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/Ar2TTZxdhSbbhnHjnbaqWozukRpB1JC2Ln/LQ6+qiQ=; b=JFu6KID0vP0cPxY4nRM9bx/q/P1h+lZee99wxarD+uj9/u1Xlfv0up53RtPLtixwK1Ciqd 2WTtvOzjsq5EApxZiqs3TK3x/bfk8rF7bUs/6kxnOnyxIDp9xnw04zjlJ+c2gnBQNJ+Q+W ousN3GeQIQRYSAuXMDCRk4tfg6X9lbobRYA8y8TlN4sjobUdWYPdQZ1WpsuWrwzMO2k/eI raN5nQYtZe2X6aL89li4AU29dvLB4dF4q8096JmnQRXYVf0NWBA7zpoz/2iXAdhEr6zQim jjJAxncmyGEWP9igXyIQ6bG5uQTaRYUM75JDCj0zyO2Hy+c6vfQS2BZRx0qObg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1722917199; a=rsa-sha256; cv=none; b=qZDgConPJs4Tp9I1xO+/V+JAL5/SlPTksy+ryQV5YFS0W1xY2wotbLG2C6ZxdPRKLRWx+f ETBx3fDimGqYpUicvR2ALNJ5425s0XOJOyDCF7cuyhWewskVg2xgI+ttvY4Aic0310tCTo IFxRxXk2+sedVnvgJM2idJQ3Rff6WYXRXz4fXePNlCcQ/C/bSbHk7q4dPnPD3nG4SLg+VQ O1Bum5f413fLtiQ85jwchLR0BnqK6qz/hk8FGjKmGCbKzWKoiho2wnWfeLiC+AeRuLEQVq Nh2juabvag67ghu3cZPlSo48wHGJQEkpdL0FwOg/xKz51yh+eiWksbVEZELmZw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1722917199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=/Ar2TTZxdhSbbhnHjnbaqWozukRpB1JC2Ln/LQ6+qiQ=; b=Hky0euggJmKzQwSugDCaDuEyRhPqkj9JMOHaMQYj1Dhd6x/GqrGuPFU+IugSkprbIzRrzb d/S+Njxg7zf4Ao2QqmDTc0amypUWV3FzNYyT02JwpUfPG56CIbemlOHNNcSBov91flmSrE kCYEmxuXSay+A6XWHOHkX1OsanwcFr6B61COHA3KxxmALRoBJM4NUxUsDJvTn4hSjsgVQf l+42DpRZmuEms6KXrjtIgBVGIj8izzjo365noYsZYCjSTBeIqkop4nlaaiqdixE4+gs4yT GmOGN7JXuuZYCD6r9Qky5trKGD0asiQ4oZQQpYCnIc3+w9aO18m/4Lhb+H7zGQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WdKW313r8zs7T; Tue, 6 Aug 2024 04:06:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47646djS004712; Tue, 6 Aug 2024 04:06:39 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47646dvJ004709; Tue, 6 Aug 2024 04:06:39 GMT (envelope-from git) Date: Tue, 6 Aug 2024 04:06:39 GMT Message-Id: <202408060406.47646dvJ004709@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: ff1ae3b3639d - main - rangelocks: restore caching of the single rl entry in the struct thread List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ff1ae3b3639d39a6485cfc655bf565cd04b9caa6 Auto-Submitted: auto-generated The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=ff1ae3b3639d39a6485cfc655bf565cd04b9caa6 commit ff1ae3b3639d39a6485cfc655bf565cd04b9caa6 Author: Konstantin Belousov AuthorDate: 2023-08-23 23:29:50 +0000 Commit: Konstantin Belousov CommitDate: 2024-08-06 04:05:58 +0000 rangelocks: restore caching of the single rl entry in the struct thread Reviewed by: markj, Olivier Certner Tested by: pho Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D41787 --- sys/kern/kern_rangelock.c | 33 +++++++++++++++++++++++++++------ sys/kern/kern_thread.c | 1 + sys/sys/rangelock.h | 1 + 3 files changed, 29 insertions(+), 6 deletions(-) diff --git a/sys/kern/kern_rangelock.c b/sys/kern/kern_rangelock.c index 355b2125dd9b..2f16569b19aa 100644 --- a/sys/kern/kern_rangelock.c +++ b/sys/kern/kern_rangelock.c @@ -82,8 +82,15 @@ static struct rl_q_entry * rlqentry_alloc(vm_ooffset_t start, vm_ooffset_t end, int flags) { struct rl_q_entry *e; - - e = uma_zalloc_smr(rl_entry_zone, M_WAITOK); + struct thread *td; + + td = curthread; + if (td->td_rlqe != NULL) { + e = td->td_rlqe; + td->td_rlqe = NULL; + } else { + e = uma_zalloc_smr(rl_entry_zone, M_WAITOK); + } e->rl_q_next = NULL; e->rl_q_free = NULL; e->rl_q_start = start; @@ -95,6 +102,12 @@ rlqentry_alloc(vm_ooffset_t start, vm_ooffset_t end, int flags) return (e); } +void +rangelock_entry_free(struct rl_q_entry *e) +{ + uma_zfree_smr(rl_entry_zone, e); +} + void rangelock_init(struct rangelock *lock) { @@ -106,6 +119,7 @@ void rangelock_destroy(struct rangelock *lock) { struct rl_q_entry *e, *ep; + struct thread *td; MPASS(!lock->sleepers); for (e = (struct rl_q_entry *)atomic_load_ptr(&lock->head); @@ -386,8 +400,10 @@ rangelock_lock_int(struct rangelock *lock, bool trylock, vm_ooffset_t start, vm_ooffset_t end, int locktype) { struct rl_q_entry *e, *free, *x, *xp; + struct thread *td; enum RL_INSERT_RES res; + td = curthread; for (res = RL_LOCK_RETRY; res == RL_LOCK_RETRY;) { free = NULL; e = rlqentry_alloc(start, end, locktype); @@ -401,10 +417,15 @@ rangelock_lock_int(struct rangelock *lock, bool trylock, vm_ooffset_t start, e = NULL; } for (x = free; x != NULL; x = xp) { - MPASS(!rl_e_is_marked(x)); - xp = x->rl_q_free; - MPASS(!rl_e_is_marked(xp)); - uma_zfree_smr(rl_entry_zone, x); + MPASS(!rl_e_is_marked(x)); + xp = x->rl_q_free; + MPASS(!rl_e_is_marked(xp)); + if (td->td_rlqe == NULL) { + smr_synchronize(rl_smr); + td->td_rlqe = x; + } else { + uma_zfree_smr(rl_entry_zone, x); + } } } return (e); diff --git a/sys/kern/kern_thread.c b/sys/kern/kern_thread.c index c951e7297c89..9c3694feb945 100644 --- a/sys/kern/kern_thread.c +++ b/sys/kern/kern_thread.c @@ -480,6 +480,7 @@ thread_fini(void *mem, int size) EVENTHANDLER_DIRECT_INVOKE(thread_fini, td); turnstile_free(td->td_turnstile); sleepq_free(td->td_sleepqueue); + rangelock_entry_free(td->td_rlqe); umtx_thread_fini(td); MPASS(td->td_sel == NULL); } diff --git a/sys/sys/rangelock.h b/sys/sys/rangelock.h index 310371bef879..60f394b67677 100644 --- a/sys/sys/rangelock.h +++ b/sys/sys/rangelock.h @@ -65,6 +65,7 @@ void *rangelock_wlock(struct rangelock *lock, vm_ooffset_t start, vm_ooffset_t end); void *rangelock_trywlock(struct rangelock *lock, vm_ooffset_t start, vm_ooffset_t end); +void rangelock_entry_free(struct rl_q_entry *e); #if defined(INVARIANTS) || defined(INVARIANT_SUPPORT) void _rangelock_cookie_assert(void *cookie, int what, const char *file, int line);