From owner-svn-src-all@freebsd.org Fri Oct 20 03:32:44 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0163AE4E845; Fri, 20 Oct 2017 03:32:44 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id C521865D5D; Fri, 20 Oct 2017 03:32:43 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v9K3WgeK016472; Fri, 20 Oct 2017 03:32:42 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v9K3WgGW016471; Fri, 20 Oct 2017 03:32:42 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201710200332.v9K3WgGW016471@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Fri, 20 Oct 2017 03:32:42 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r324787 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 324787 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Oct 2017 03:32:44 -0000 Author: mjg Date: Fri Oct 20 03:32:42 2017 New Revision: 324787 URL: https://svnweb.freebsd.org/changeset/base/324787 Log: rwlock: reduce lockstat branches in the slowpath MFC after: 1 week Modified: head/sys/kern/kern_rwlock.c Modified: head/sys/kern/kern_rwlock.c ============================================================================== --- head/sys/kern/kern_rwlock.c Fri Oct 20 03:31:23 2017 (r324786) +++ head/sys/kern/kern_rwlock.c Fri Oct 20 03:32:42 2017 (r324787) @@ -847,6 +847,9 @@ __rw_wlock_hard(volatile uintptr_t *c, uintptr_t v, ui int64_t sleep_time = 0; int64_t all_time = 0; #endif +#if defined(KDTRACE_HOOKS) || defined(LOCK_PROFILING) + int doing_lockprof; +#endif if (SCHEDULER_STOPPED()) return; @@ -875,10 +878,17 @@ __rw_wlock_hard(volatile uintptr_t *c, uintptr_t v, ui CTR5(KTR_LOCK, "%s: %s contested (lock=%p) at %s:%d", __func__, rw->lock_object.lo_name, (void *)rw->rw_lock, file, line); -#ifdef KDTRACE_HOOKS - all_time -= lockstat_nsecs(&rw->lock_object); +#ifdef LOCK_PROFILING + doing_lockprof = 1; state = v; +#elif defined(KDTRACE_HOOKS) + doing_lockprof = lockstat_enabled; + if (__predict_false(doing_lockprof)) { + all_time -= lockstat_nsecs(&rw->lock_object); + state = v; + } #endif + for (;;) { if (v == RW_UNLOCKED) { if (_rw_write_lock_fetch(rw, &v, tid)) @@ -1024,6 +1034,10 @@ __rw_wlock_hard(volatile uintptr_t *c, uintptr_t v, ui #endif v = RW_READ_VALUE(rw); } +#if defined(KDTRACE_HOOKS) || defined(LOCK_PROFILING) + if (__predict_true(!doing_lockprof)) + return; +#endif #ifdef KDTRACE_HOOKS all_time += lockstat_nsecs(&rw->lock_object); if (sleep_time)