From owner-svn-src-all@freebsd.org Wed Nov 27 01:54:40 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C506F1C7418; Wed, 27 Nov 2019 01:54:40 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47N3jr4ms9z4Wj6; Wed, 27 Nov 2019 01:54:40 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 87E2C63AF; Wed, 27 Nov 2019 01:54:40 +0000 (UTC) (envelope-from rlibby@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xAR1seFX053649; Wed, 27 Nov 2019 01:54:40 GMT (envelope-from rlibby@FreeBSD.org) Received: (from rlibby@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xAR1sep4053646; Wed, 27 Nov 2019 01:54:40 GMT (envelope-from rlibby@FreeBSD.org) Message-Id: <201911270154.xAR1sep4053646@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rlibby set sender to rlibby@FreeBSD.org using -f From: Ryan Libby Date: Wed, 27 Nov 2019 01:54:40 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r355126 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: rlibby X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 355126 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Nov 2019 01:54:40 -0000 Author: rlibby Date: Wed Nov 27 01:54:39 2019 New Revision: 355126 URL: https://svnweb.freebsd.org/changeset/base/355126 Log: witness: sleepable rm locks are not sleepable in read mode There are two classes of rm lock, one "sleepable" and one not. But even a "sleepable" rm lock is only sleepable in write mode, and is non-sleepable when taken in read mode. Warn about sleepable rm locks in read mode as non-sleepable locks. Do this by defining a new lock operation flag, LOP_NOSLEEP, to indicate that a lock is non-sleepable despite what the LO_SLEEPABLE flag would indicate, and defining a new witness lock instance flag, LI_SLEEPABLE, to track the product of LO_SLEEPABLE and LOP_NOSLEEP on the lock instance. Reviewed by: markj Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D22527 Modified: head/sys/kern/kern_rmlock.c head/sys/kern/subr_witness.c head/sys/sys/lock.h Modified: head/sys/kern/kern_rmlock.c ============================================================================== --- head/sys/kern/kern_rmlock.c Wed Nov 27 01:21:42 2019 (r355125) +++ head/sys/kern/kern_rmlock.c Wed Nov 27 01:54:39 2019 (r355126) @@ -652,8 +652,8 @@ _rm_rlock_debug(struct rmlock *rm, struct rm_priotrack KASSERT(!rm_wowned(rm), ("rm_rlock: wlock already held for %s @ %s:%d", rm->lock_object.lo_name, file, line)); - WITNESS_CHECKORDER(&rm->lock_object, LOP_NEWORDER, file, line, - NULL); + WITNESS_CHECKORDER(&rm->lock_object, + LOP_NEWORDER | LOP_NOSLEEP, file, line, NULL); } if (_rm_rlock(rm, tracker, trylock)) { @@ -663,7 +663,7 @@ _rm_rlock_debug(struct rmlock *rm, struct rm_priotrack else LOCK_LOG_LOCK("RMRLOCK", &rm->lock_object, 0, 0, file, line); - WITNESS_LOCK(&rm->lock_object, 0, file, line); + WITNESS_LOCK(&rm->lock_object, LOP_NOSLEEP, file, line); TD_LOCKS_INC(curthread); return (1); } else if (trylock) Modified: head/sys/kern/subr_witness.c ============================================================================== --- head/sys/kern/subr_witness.c Wed Nov 27 01:21:42 2019 (r355125) +++ head/sys/kern/subr_witness.c Wed Nov 27 01:54:39 2019 (r355126) @@ -131,6 +131,7 @@ __FBSDID("$FreeBSD$"); #define LI_RECURSEMASK 0x0000ffff /* Recursion depth of lock instance. */ #define LI_EXCLUSIVE 0x00010000 /* Exclusive lock instance. */ #define LI_NORELEASE 0x00020000 /* Lock not allowed to be released. */ +#define LI_SLEEPABLE 0x00040000 /* Lock may be held while sleeping. */ #ifndef WITNESS_COUNT #define WITNESS_COUNT 1536 @@ -1302,7 +1303,7 @@ witness_checkorder(struct lock_object *lock, int flags * If we are locking Giant and this is a sleepable * lock, then skip it. */ - if ((lock1->li_lock->lo_flags & LO_SLEEPABLE) != 0 && + if ((lock1->li_flags & LI_SLEEPABLE) != 0 && lock == &Giant.lock_object) continue; @@ -1311,6 +1312,7 @@ witness_checkorder(struct lock_object *lock, int flags * is Giant, then skip it. */ if ((lock->lo_flags & LO_SLEEPABLE) != 0 && + (flags & LOP_NOSLEEP) == 0 && lock1->li_lock == &Giant.lock_object) continue; @@ -1320,15 +1322,16 @@ witness_checkorder(struct lock_object *lock, int flags * order violation to enfore a general lock order of * sleepable locks before non-sleepable locks. */ - if (((lock->lo_flags & LO_SLEEPABLE) != 0 && - (lock1->li_lock->lo_flags & LO_SLEEPABLE) == 0)) + if ((lock->lo_flags & LO_SLEEPABLE) != 0 && + (flags & LOP_NOSLEEP) == 0 && + (lock1->li_flags & LI_SLEEPABLE) == 0) goto reversal; /* * If we are locking Giant and this is a non-sleepable * lock, then treat it as a reversal. */ - if ((lock1->li_lock->lo_flags & LO_SLEEPABLE) == 0 && + if ((lock1->li_flags & LI_SLEEPABLE) == 0 && lock == &Giant.lock_object) goto reversal; @@ -1378,11 +1381,12 @@ witness_checkorder(struct lock_object *lock, int flags /* * Ok, yell about it. */ - if (((lock->lo_flags & LO_SLEEPABLE) != 0 && - (lock1->li_lock->lo_flags & LO_SLEEPABLE) == 0)) + if ((lock->lo_flags & LO_SLEEPABLE) != 0 && + (flags & LOP_NOSLEEP) == 0 && + (lock1->li_flags & LI_SLEEPABLE) == 0) witness_output( "lock order reversal: (sleepable after non-sleepable)\n"); - else if ((lock1->li_lock->lo_flags & LO_SLEEPABLE) == 0 + else if ((lock1->li_flags & LI_SLEEPABLE) == 0 && lock == &Giant.lock_object) witness_output( "lock order reversal: (Giant after non-sleepable)\n"); @@ -1440,7 +1444,8 @@ witness_checkorder(struct lock_object *lock, int flags */ if (flags & LOP_NEWORDER && !(plock->li_lock == &Giant.lock_object && - (lock->lo_flags & LO_SLEEPABLE) != 0)) { + (lock->lo_flags & LO_SLEEPABLE) != 0 && + (flags & LOP_NOSLEEP) == 0)) { CTR3(KTR_WITNESS, "%s: adding %s as a child of %s", __func__, w->w_name, plock->li_lock->lo_witness->w_name); itismychild(plock->li_lock->lo_witness, w); @@ -1500,10 +1505,11 @@ witness_lock(struct lock_object *lock, int flags, cons instance->li_lock = lock; instance->li_line = line; instance->li_file = file; + instance->li_flags = 0; if ((flags & LOP_EXCLUSIVE) != 0) - instance->li_flags = LI_EXCLUSIVE; - else - instance->li_flags = 0; + instance->li_flags |= LI_EXCLUSIVE; + if ((lock->lo_flags & LO_SLEEPABLE) != 0 && (flags & LOP_NOSLEEP) == 0) + instance->li_flags |= LI_SLEEPABLE; CTR4(KTR_WITNESS, "%s: pid %d added %s as lle[%d]", __func__, td->td_proc->p_pid, lock->lo_name, lle->ll_count - 1); } @@ -1763,7 +1769,7 @@ witness_warn(int flags, struct lock_object *lock, cons lock1->li_lock == &Giant.lock_object) continue; if (flags & WARN_SLEEPOK && - (lock1->li_lock->lo_flags & LO_SLEEPABLE) != 0) + (lock1->li_flags & LI_SLEEPABLE) != 0) continue; if (n == 0) { va_start(ap, fmt); Modified: head/sys/sys/lock.h ============================================================================== --- head/sys/sys/lock.h Wed Nov 27 01:21:42 2019 (r355125) +++ head/sys/sys/lock.h Wed Nov 27 01:54:39 2019 (r355126) @@ -107,6 +107,7 @@ struct lock_class { #define LOP_TRYLOCK 0x00000004 /* Don't check lock order. */ #define LOP_EXCLUSIVE 0x00000008 /* Exclusive lock. */ #define LOP_DUPOK 0x00000010 /* Don't check for duplicate acquires */ +#define LOP_NOSLEEP 0x00000020 /* Non-sleepable despite LO_SLEEPABLE */ /* Flags passed to witness_assert. */ #define LA_MASKASSERT 0x000000ff /* Mask for witness defined asserts. */