From owner-svn-src-head@freebsd.org Fri Nov 17 02:26:16 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3889DF12A0; Fri, 17 Nov 2017 02:26:16 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A10C17593C; Fri, 17 Nov 2017 02:26:16 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id vAH2QFTA046966; Fri, 17 Nov 2017 02:26:15 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id vAH2QFwq046965; Fri, 17 Nov 2017 02:26:15 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201711170226.vAH2QFwq046965@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Fri, 17 Nov 2017 02:26:15 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r325921 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 325921 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Nov 2017 02:26:16 -0000 Author: mjg Date: Fri Nov 17 02:26:15 2017 New Revision: 325921 URL: https://svnweb.freebsd.org/changeset/base/325921 Log: rwlock: unlock before traversing threads to wake up While here perform a minor cleanup of the unlock path. Modified: head/sys/kern/kern_rwlock.c Modified: head/sys/kern/kern_rwlock.c ============================================================================== --- head/sys/kern/kern_rwlock.c Fri Nov 17 02:25:04 2017 (r325920) +++ head/sys/kern/kern_rwlock.c Fri Nov 17 02:26:15 2017 (r325921) @@ -1081,7 +1081,7 @@ __rw_wunlock_hard(volatile uintptr_t *c, uintptr_t tid { struct rwlock *rw; struct turnstile *ts; - uintptr_t v; + uintptr_t v, setv; int queue; if (SCHEDULER_STOPPED()) @@ -1108,8 +1108,6 @@ __rw_wunlock_hard(volatile uintptr_t *c, uintptr_t tid CTR2(KTR_LOCK, "%s: %p contested", __func__, rw); turnstile_chain_lock(&rw->lock_object); - ts = turnstile_lookup(&rw->lock_object); - MPASS(ts != NULL); /* * Use the same algo as sx locks for now. Prefer waking up shared @@ -1127,19 +1125,23 @@ __rw_wunlock_hard(volatile uintptr_t *c, uintptr_t tid * there that could be worked around either by waking both queues * of waiters or doing some complicated lock handoff gymnastics. */ - v = RW_UNLOCKED; - if (rw->rw_lock & RW_LOCK_WRITE_WAITERS) { + setv = RW_UNLOCKED; + v = RW_READ_VALUE(rw); + queue = TS_SHARED_QUEUE; + if (v & RW_LOCK_WRITE_WAITERS) { queue = TS_EXCLUSIVE_QUEUE; - v |= (rw->rw_lock & RW_LOCK_READ_WAITERS); - } else - queue = TS_SHARED_QUEUE; + setv |= (v & RW_LOCK_READ_WAITERS); + } + atomic_store_rel_ptr(&rw->rw_lock, setv); /* Wake up all waiters for the specific queue. */ if (LOCK_LOG_TEST(&rw->lock_object, 0)) CTR3(KTR_LOCK, "%s: %p waking up %s waiters", __func__, rw, queue == TS_SHARED_QUEUE ? "read" : "write"); + + ts = turnstile_lookup(&rw->lock_object); + MPASS(ts != NULL); turnstile_broadcast(ts, queue); - atomic_store_rel_ptr(&rw->rw_lock, v); turnstile_unpend(ts, TS_EXCLUSIVE_LOCK); turnstile_chain_unlock(&rw->lock_object); }