From owner-svn-src-all@freebsd.org Sat Feb 17 12:07:10 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4912BF06AFE; Sat, 17 Feb 2018 12:07:10 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E35187A313; Sat, 17 Feb 2018 12:07:09 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DE2DB1C6FD; Sat, 17 Feb 2018 12:07:09 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w1HC79Q7087360; Sat, 17 Feb 2018 12:07:09 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w1HC79C1087357; Sat, 17 Feb 2018 12:07:09 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201802171207.w1HC79C1087357@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Sat, 17 Feb 2018 12:07:09 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r329451 - in head/sys: kern sys X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: in head/sys: kern sys X-SVN-Commit-Revision: 329451 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2018 12:07:10 -0000 Author: mjg Date: Sat Feb 17 12:07:09 2018 New Revision: 329451 URL: https://svnweb.freebsd.org/changeset/base/329451 Log: Undo LOCK_PROFILING pessimisation after r313454 and r313455 With the option used to compile the kernel both sx and rw shared ops would always go to the slow path which added avoidable overhead even when the facility is disabled. Furthermore the increased time spent doing uncontested shared lock acquire would be bogusly added to total wait time, somewhat skewing the results. Restore old behaviour of going there only when profiling is enabled. This change is a no-op for kernels without LOCK_PROFILING (which is the default). Modified: head/sys/kern/kern_rwlock.c head/sys/kern/kern_sx.c head/sys/sys/lockstat.h Modified: head/sys/kern/kern_rwlock.c ============================================================================== --- head/sys/kern/kern_rwlock.c Sat Feb 17 09:58:53 2018 (r329450) +++ head/sys/kern/kern_rwlock.c Sat Feb 17 12:07:09 2018 (r329451) @@ -645,9 +645,12 @@ __rw_rlock_int(struct rwlock *rw LOCK_FILE_LINE_ARG_DE WITNESS_CHECKORDER(&rw->lock_object, LOP_NEWORDER, file, line, NULL); v = RW_READ_VALUE(rw); - if (__predict_false(LOCKSTAT_OOL_PROFILE_ENABLED(rw__acquire) || + if (__predict_false(LOCKSTAT_PROFILE_ENABLED(rw__acquire) || !__rw_rlock_try(rw, td, &v, true LOCK_FILE_LINE_ARG))) __rw_rlock_hard(rw, td, v LOCK_FILE_LINE_ARG); + else + lock_profile_obtain_lock_success(&rw->lock_object, 0, 0, + file, line); LOCK_LOG_LOCK("RLOCK", &rw->lock_object, 0, 0, file, line); WITNESS_LOCK(&rw->lock_object, 0, file, line); @@ -839,9 +842,11 @@ _rw_runlock_cookie_int(struct rwlock *rw LOCK_FILE_LIN td = curthread; v = RW_READ_VALUE(rw); - if (__predict_false(LOCKSTAT_OOL_PROFILE_ENABLED(rw__release) || + if (__predict_false(LOCKSTAT_PROFILE_ENABLED(rw__release) || !__rw_runlock_try(rw, td, &v))) __rw_runlock_hard(rw, td, v LOCK_FILE_LINE_ARG); + else + lock_profile_release_lock(&rw->lock_object); TD_LOCKS_DEC(curthread); } Modified: head/sys/kern/kern_sx.c ============================================================================== --- head/sys/kern/kern_sx.c Sat Feb 17 09:58:53 2018 (r329450) +++ head/sys/kern/kern_sx.c Sat Feb 17 12:07:09 2018 (r329451) @@ -1122,9 +1122,12 @@ _sx_slock_int(struct sx *sx, int opts LOCK_FILE_LINE_A error = 0; x = SX_READ_VALUE(sx); - if (__predict_false(LOCKSTAT_OOL_PROFILE_ENABLED(sx__acquire) || + if (__predict_false(LOCKSTAT_PROFILE_ENABLED(sx__acquire) || !__sx_slock_try(sx, &x LOCK_FILE_LINE_ARG))) error = _sx_slock_hard(sx, opts, x LOCK_FILE_LINE_ARG); + else + lock_profile_obtain_lock_success(&sx->lock_object, 0, 0, + file, line); if (error == 0) { LOCK_LOG_LOCK("SLOCK", &sx->lock_object, 0, 0, file, line); WITNESS_LOCK(&sx->lock_object, 0, file, line); @@ -1252,9 +1255,11 @@ _sx_sunlock_int(struct sx *sx LOCK_FILE_LINE_ARG_DEF) LOCK_LOG_LOCK("SUNLOCK", &sx->lock_object, 0, 0, file, line); x = SX_READ_VALUE(sx); - if (__predict_false(LOCKSTAT_OOL_PROFILE_ENABLED(sx__release) || + if (__predict_false(LOCKSTAT_PROFILE_ENABLED(sx__release) || !_sx_sunlock_try(sx, &x))) _sx_sunlock_hard(sx, x LOCK_FILE_LINE_ARG); + else + lock_profile_release_lock(&sx->lock_object); TD_LOCKS_DEC(curthread); } Modified: head/sys/sys/lockstat.h ============================================================================== --- head/sys/sys/lockstat.h Sat Feb 17 09:58:53 2018 (r329450) +++ head/sys/sys/lockstat.h Sat Feb 17 12:07:09 2018 (r329451) @@ -109,12 +109,7 @@ extern volatile int lockstat_enabled; LOCKSTAT_RECORD1(probe, lp, a); \ } while (0) -#ifndef LOCK_PROFILING #define LOCKSTAT_PROFILE_ENABLED(probe) __predict_false(lockstat_enabled) -#define LOCKSTAT_OOL_PROFILE_ENABLED(probe) LOCKSTAT_PROFILE_ENABLED(probe) -#else -#define LOCKSTAT_OOL_PROFILE_ENABLED(probe) 1 -#endif struct lock_object; uint64_t lockstat_nsecs(struct lock_object *); @@ -139,10 +134,7 @@ uint64_t lockstat_nsecs(struct lock_object *); #define LOCKSTAT_PROFILE_RELEASE_RWLOCK(probe, lp, a) \ LOCKSTAT_PROFILE_RELEASE_LOCK(probe, lp) -#ifndef LOCK_PROFILING #define LOCKSTAT_PROFILE_ENABLED(probe) 0 -#endif -#define LOCKSTAT_OOL_PROFILE_ENABLED(probe) 1 #endif /* !KDTRACE_HOOKS */