From nobody Wed Aug 28 14:49:57 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wv6lB2SQ2z52Ppb; Wed, 28 Aug 2024 14:49:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wv6lB0r43z4NHL; Wed, 28 Aug 2024 14:49:58 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724856598; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=UQ05eBOA197Ve+Wb/r+6omAuu0zH9oXizzN/mxeDcx0=; b=F8OSSE99E9h77OHIsat9c44KneAwqXMHqzZ0/8+vtnPKFLIBJel/0/Uua8CqCrAALIpjQR e7F3my6fP9PmbDUu+cu3YWgQ8zAsomERcaO0kt0OXi6z+a7j133hesOsX58Gf/iF32JqdR Z9Y7/dKTFLgq2aDCi+Twqbeuc1AJN7uKjPh7Z6kpGINgk2Bx4SebJkUaiYJXGu79ffAQ1L r3wti51JfWUNBubrNH10jdwzn5WgeUyEQEf8zXc8xBXJaR3eQQpS/LYca7Er7Xp+rqmr2m cdpoTFn12giKB7kOE3h7U465IFCmdLOgRw+RcS8pYk/wmJpku2UHbfAC385VFg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724856598; a=rsa-sha256; cv=none; b=c5t0VovcCp6zXqP3e7w7rmYEADIszMwBfSwOocckNFZQn/0y7PvmzdQcsu8h10jT2mD8IB iehc9TGQYiy7JjtPKK9IUDCusIvqho6Q1BR3kRPw+m0X8mPE7pLTSKwaM7/8gNjef99Epj Kb/XeQessXOLGQuDa8u3aXIJtWhzdKWyfTIMj657kvLyJvav8Bvg1VVK6n0c+dyoPFKjoD t/YjaWAACyNl0gYr1jB4dAQ7bheGAb+n/FgtpfRfgyCA+m8qNzUBt4/y91EUr/ki4Dieek i09qtZKkWPHjSk4KXqIROoT+R39SD14GMqowvNZ/YHw+Y/FfX/XDgIDq1WLAWQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724856598; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=UQ05eBOA197Ve+Wb/r+6omAuu0zH9oXizzN/mxeDcx0=; b=SoIZhgbXSP1r5syjEHnOfHxT3MFrAK5UbG4Sf1ILvD06o0l4yJE9MH7+DkpgFZgsAuRq41 7RpqVPqICusrtL9RZxAqQVIMpzHZuy9uDcyFUzHrSA3o03zBQ98hh+eh2ag3z0CIXyG5Xy hZiJIw6LRTUOAGG36O/ToM7XrSjWTwKclQ4EhpCh49u70uBGb5E+kan+aGIZNrbaLIFIYf bWwvzhVx95lITavAoCMgUCX5gSqB8vxcFGM3QKA8/sigoMidUtNWjsj9gqTtYB9/hzztvt DdL7QZv/iSZtUxxwbPrwPaFVQPcGAyZc59FU1stZKIcunATlSVb+EO3VWWtkwQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wv6lB06ftzdtY; Wed, 28 Aug 2024 14:49:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47SEnvJD000120; Wed, 28 Aug 2024 14:49:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47SEnvLx000116; Wed, 28 Aug 2024 14:49:57 GMT (envelope-from git) Date: Wed, 28 Aug 2024 14:49:57 GMT Message-Id: <202408281449.47SEnvLx000116@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Konstantin Belousov Subject: git: 0b6b1c285920 - main - Add rangelock_may_recurse(9) List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kib X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 0b6b1c285920563ba0c119d3190ac25af4731d02 Auto-Submitted: auto-generated The branch main has been updated by kib: URL: https://cgit.FreeBSD.org/src/commit/?id=0b6b1c285920563ba0c119d3190ac25af4731d02 commit 0b6b1c285920563ba0c119d3190ac25af4731d02 Author: Konstantin Belousov AuthorDate: 2024-08-27 21:33:38 +0000 Commit: Konstantin Belousov CommitDate: 2024-08-28 14:33:58 +0000 Add rangelock_may_recurse(9) Reviewed by: markj Tested by: lwhsu Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D46465 --- sys/kern/kern_rangelock.c | 41 +++++++++++++++++++++++++++++++++++++++++ sys/sys/rangelock.h | 1 + 2 files changed, 42 insertions(+) diff --git a/sys/kern/kern_rangelock.c b/sys/kern/kern_rangelock.c index 0e62b91b4ee7..4d74c02302e7 100644 --- a/sys/kern/kern_rangelock.c +++ b/sys/kern/kern_rangelock.c @@ -752,6 +752,47 @@ rangelock_trywlock(struct rangelock *lock, vm_ooffset_t start, vm_ooffset_t end) return (rangelock_lock_int(lock, true, start, end, RL_LOCK_WRITE)); } +/* + * If the caller asserts that it can obtain the range locks on the + * same lock simultaneously, switch to the non-cheat mode. Cheat mode + * cannot handle it, hanging in drain or trylock retries. + */ +void +rangelock_may_recurse(struct rangelock *lock) +{ + uintptr_t v, x; + + v = atomic_load_ptr(&lock->head); + if ((v & RL_CHEAT_CHEATING) == 0) + return; + + sleepq_lock(&lock->head); + for (;;) { + if ((v & RL_CHEAT_CHEATING) == 0) { + sleepq_release(&lock->head); + return; + } + + /* Cheating and locked, drain. */ + if ((v & RL_CHEAT_WLOCKED) != 0 || + (v & ~RL_CHEAT_MASK) >= RL_CHEAT_READER) { + x = v | RL_CHEAT_DRAINING; + if (atomic_fcmpset_ptr(&lock->head, &v, x) != 0) { + rangelock_cheat_drain(lock); + return; + } + continue; + } + + /* Cheating and unlocked, clear RL_CHEAT_CHEATING. */ + x = 0; + if (atomic_fcmpset_ptr(&lock->head, &v, x) != 0) { + sleepq_release(&lock->head); + return; + } + } +} + #ifdef INVARIANT_SUPPORT void _rangelock_cookie_assert(void *cookie, int what, const char *file, int line) diff --git a/sys/sys/rangelock.h b/sys/sys/rangelock.h index 127f101ddc2e..accf33d7296b 100644 --- a/sys/sys/rangelock.h +++ b/sys/sys/rangelock.h @@ -66,6 +66,7 @@ void *rangelock_wlock(struct rangelock *lock, vm_ooffset_t start, void *rangelock_trywlock(struct rangelock *lock, vm_ooffset_t start, vm_ooffset_t end); void rangelock_entry_free(struct rl_q_entry *e); +void rangelock_may_recurse(struct rangelock *lock); #if defined(INVARIANTS) || defined(INVARIANT_SUPPORT) void _rangelock_cookie_assert(void *cookie, int what, const char *file, int line);