From owner-svn-src-all@freebsd.org Tue Nov 24 03:49:38 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B303C477485; Tue, 24 Nov 2020 03:49:38 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cg94y4gx0z3hHB; Tue, 24 Nov 2020 03:49:38 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9394818505; Tue, 24 Nov 2020 03:49:38 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0AO3ncmV094324; Tue, 24 Nov 2020 03:49:38 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0AO3nbiT094320; Tue, 24 Nov 2020 03:49:37 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <202011240349.0AO3nbiT094320@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Tue, 24 Nov 2020 03:49:37 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r367978 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 367978 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Nov 2020 03:49:38 -0000 Author: mjg Date: Tue Nov 24 03:49:37 2020 New Revision: 367978 URL: https://svnweb.freebsd.org/changeset/base/367978 Log: locks: push lock_delay_arg_init calls down Minor cleanup to skip doing them when recursing on locks and so that they can act on found lock value if need be. Modified: head/sys/kern/kern_lock.c head/sys/kern/kern_mutex.c head/sys/kern/kern_rwlock.c head/sys/kern/kern_sx.c Modified: head/sys/kern/kern_lock.c ============================================================================== --- head/sys/kern/kern_lock.c Tue Nov 24 03:48:44 2020 (r367977) +++ head/sys/kern/kern_lock.c Tue Nov 24 03:49:37 2020 (r367978) @@ -603,10 +603,10 @@ lockmgr_slock_hard(struct lock *lk, u_int flags, struc if (LK_CAN_WITNESS(flags)) WITNESS_CHECKORDER(&lk->lock_object, LOP_NEWORDER, file, line, flags & LK_INTERLOCK ? ilk : NULL); + x = lockmgr_read_value(lk); lock_delay_arg_init(&lda, &lockmgr_delay); if (!lk_adaptive) flags &= ~LK_ADAPTIVE; - x = lockmgr_read_value(lk); /* * The lock may already be locked exclusive by curthread, * avoid deadlock. Modified: head/sys/kern/kern_mutex.c ============================================================================== --- head/sys/kern/kern_mutex.c Tue Nov 24 03:48:44 2020 (r367977) +++ head/sys/kern/kern_mutex.c Tue Nov 24 03:49:37 2020 (r367978) @@ -535,12 +535,6 @@ __mtx_lock_sleep(volatile uintptr_t *c, uintptr_t v) if (SCHEDULER_STOPPED_TD(td)) return; -#if defined(ADAPTIVE_MUTEXES) - lock_delay_arg_init(&lda, &mtx_delay); -#elif defined(KDTRACE_HOOKS) - lock_delay_arg_init_noadapt(&lda); -#endif - if (__predict_false(v == MTX_UNOWNED)) v = MTX_READ_VALUE(m); @@ -562,6 +556,12 @@ __mtx_lock_sleep(volatile uintptr_t *c, uintptr_t v) opts &= ~MTX_RECURSE; #endif +#if defined(ADAPTIVE_MUTEXES) + lock_delay_arg_init(&lda, &mtx_delay); +#elif defined(KDTRACE_HOOKS) + lock_delay_arg_init_noadapt(&lda); +#endif + #ifdef HWPMC_HOOKS PMC_SOFT_CALL( , , lock, failed); #endif @@ -746,12 +746,12 @@ _mtx_lock_spin_cookie(volatile uintptr_t *c, uintptr_t if (SCHEDULER_STOPPED()) return; - lock_delay_arg_init(&lda, &mtx_spin_delay); - if (LOCK_LOG_TEST(&m->lock_object, opts)) CTR1(KTR_LOCK, "_mtx_lock_spin: %p spinning", m); KTR_STATE1(KTR_SCHED, "thread", sched_tdname((struct thread *)tid), "spinning", "lockname:\"%s\"", m->lock_object.lo_name); + + lock_delay_arg_init(&lda, &mtx_spin_delay); #ifdef HWPMC_HOOKS PMC_SOFT_CALL( , , lock, failed); Modified: head/sys/kern/kern_rwlock.c ============================================================================== --- head/sys/kern/kern_rwlock.c Tue Nov 24 03:48:44 2020 (r367977) +++ head/sys/kern/kern_rwlock.c Tue Nov 24 03:49:37 2020 (r367978) @@ -948,11 +948,6 @@ __rw_wlock_hard(volatile uintptr_t *c, uintptr_t v LOC if (SCHEDULER_STOPPED()) return; -#if defined(ADAPTIVE_RWLOCKS) - lock_delay_arg_init(&lda, &rw_delay); -#elif defined(KDTRACE_HOOKS) - lock_delay_arg_init_noadapt(&lda); -#endif if (__predict_false(v == RW_UNLOCKED)) v = RW_READ_VALUE(rw); @@ -970,6 +965,12 @@ __rw_wlock_hard(volatile uintptr_t *c, uintptr_t v LOC if (LOCK_LOG_TEST(&rw->lock_object, 0)) CTR5(KTR_LOCK, "%s: %s contested (lock=%p) at %s:%d", __func__, rw->lock_object.lo_name, (void *)rw->rw_lock, file, line); + +#if defined(ADAPTIVE_RWLOCKS) + lock_delay_arg_init(&lda, &rw_delay); +#elif defined(KDTRACE_HOOKS) + lock_delay_arg_init_noadapt(&lda); +#endif #ifdef HWPMC_HOOKS PMC_SOFT_CALL( , , lock, failed); Modified: head/sys/kern/kern_sx.c ============================================================================== --- head/sys/kern/kern_sx.c Tue Nov 24 03:48:44 2020 (r367977) +++ head/sys/kern/kern_sx.c Tue Nov 24 03:49:37 2020 (r367978) @@ -620,12 +620,6 @@ _sx_xlock_hard(struct sx *sx, uintptr_t x, int opts LO if (SCHEDULER_STOPPED()) return (0); -#if defined(ADAPTIVE_SX) - lock_delay_arg_init(&lda, &sx_delay); -#elif defined(KDTRACE_HOOKS) - lock_delay_arg_init_noadapt(&lda); -#endif - if (__predict_false(x == SX_LOCK_UNLOCKED)) x = SX_READ_VALUE(sx); @@ -644,6 +638,12 @@ _sx_xlock_hard(struct sx *sx, uintptr_t x, int opts LO if (LOCK_LOG_TEST(&sx->lock_object, 0)) CTR5(KTR_LOCK, "%s: %s contested (lock=%p) at %s:%d", __func__, sx->lock_object.lo_name, (void *)sx->sx_lock, file, line); + +#if defined(ADAPTIVE_SX) + lock_delay_arg_init(&lda, &sx_delay); +#elif defined(KDTRACE_HOOKS) + lock_delay_arg_init_noadapt(&lda); +#endif #ifdef HWPMC_HOOKS PMC_SOFT_CALL( , , lock, failed);