From owner-svn-src-head@freebsd.org Wed Apr 11 01:43:30 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6B059F88597; Wed, 11 Apr 2018 01:43:30 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 1B6DE7ECF2; Wed, 11 Apr 2018 01:43:30 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 1655515BD5; Wed, 11 Apr 2018 01:43:30 +0000 (UTC) (envelope-from mjg@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w3B1hTXo045077; Wed, 11 Apr 2018 01:43:29 GMT (envelope-from mjg@FreeBSD.org) Received: (from mjg@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w3B1hT5o045075; Wed, 11 Apr 2018 01:43:29 GMT (envelope-from mjg@FreeBSD.org) Message-Id: <201804110143.w3B1hT5o045075@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mjg set sender to mjg@FreeBSD.org using -f From: Mateusz Guzik Date: Wed, 11 Apr 2018 01:43:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r332398 - head/sys/kern X-SVN-Group: head X-SVN-Commit-Author: mjg X-SVN-Commit-Paths: head/sys/kern X-SVN-Commit-Revision: 332398 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Apr 2018 01:43:30 -0000 Author: mjg Date: Wed Apr 11 01:43:29 2018 New Revision: 332398 URL: https://svnweb.freebsd.org/changeset/base/332398 Log: locks: extend speculative spin waiting for readers to drain Now that 10 years have passed since the original limit of 10000 was committed, bump it a little bit. Spinning waiting for writers is semi-informed in the sense that we always know if the owner is running and base the decision to spin on that. However, no such information is provided for read-locking. In particular this means that it is possible for a write-spinner to completely waste cpu time waiting for the lock to be released, while the reader holding it was preempted and is now waiting for the spinner to go off cpu. Nonetheless, in majority of cases it is an improvement to spin instead of instantly giving up and going to sleep. The current approach is pretty simple: snatch the number of current readers and performs that many pauses before checking again. The total number of pauses to execute is limited to 10k. If the lock is still not free by that time, go to sleep. Given the previously noted problem of not knowing whether spinning makes any sense to begin with the new limit has to remain rather conservative. But at the very least it should also be related to the machine. Waiting for writers uses parameters selected based on the number of activated hardware threads. The upper limit of pause instructions to be executed in-between re-reads of the lock is typically 16384 or 32678. It was selected as the limit of total spins. The lower bound is set to already present 10000 as to not change it for smaller machines. Bumping the limit reduces system time by few % during benchmarks like buildworld, buildkernel and others. Tested on 2 and 4 socket machines (Broadwell, Skylake). Figuring out how to make a more informed decision while not pessimizing the fast path is left as an exercise for the reader. Modified: head/sys/kern/kern_rwlock.c head/sys/kern/kern_sx.c Modified: head/sys/kern/kern_rwlock.c ============================================================================== --- head/sys/kern/kern_rwlock.c Tue Apr 10 23:38:31 2018 (r332397) +++ head/sys/kern/kern_rwlock.c Wed Apr 11 01:43:29 2018 (r332398) @@ -95,8 +95,8 @@ struct lock_class lock_class_rw = { }; #ifdef ADAPTIVE_RWLOCKS -static int __read_frequently rowner_retries = 10; -static int __read_frequently rowner_loops = 10000; +static int __read_frequently rowner_retries; +static int __read_frequently rowner_loops; static SYSCTL_NODE(_debug, OID_AUTO, rwlock, CTLFLAG_RD, NULL, "rwlock debugging"); SYSCTL_INT(_debug_rwlock, OID_AUTO, retry, CTLFLAG_RW, &rowner_retries, 0, ""); @@ -109,7 +109,15 @@ SYSCTL_INT(_debug_rwlock, OID_AUTO, delay_base, CTLFLA SYSCTL_INT(_debug_rwlock, OID_AUTO, delay_max, CTLFLAG_RW, &rw_delay.max, 0, ""); -LOCK_DELAY_SYSINIT_DEFAULT(rw_delay); +static void +rw_lock_delay_init(void *arg __unused) +{ + + lock_delay_default_init(&rw_delay); + rowner_retries = 10; + rowner_loops = max(10000, rw_delay.max); +} +LOCK_DELAY_SYSINIT(rw_lock_delay_init); #endif /* Modified: head/sys/kern/kern_sx.c ============================================================================== --- head/sys/kern/kern_sx.c Tue Apr 10 23:38:31 2018 (r332397) +++ head/sys/kern/kern_sx.c Wed Apr 11 01:43:29 2018 (r332398) @@ -145,8 +145,8 @@ struct lock_class lock_class_sx = { #endif #ifdef ADAPTIVE_SX -static __read_frequently u_int asx_retries = 10; -static __read_frequently u_int asx_loops = 10000; +static __read_frequently u_int asx_retries; +static __read_frequently u_int asx_loops; static SYSCTL_NODE(_debug, OID_AUTO, sx, CTLFLAG_RD, NULL, "sxlock debugging"); SYSCTL_UINT(_debug_sx, OID_AUTO, retries, CTLFLAG_RW, &asx_retries, 0, ""); SYSCTL_UINT(_debug_sx, OID_AUTO, loops, CTLFLAG_RW, &asx_loops, 0, ""); @@ -158,7 +158,15 @@ SYSCTL_INT(_debug_sx, OID_AUTO, delay_base, CTLFLAG_RW SYSCTL_INT(_debug_sx, OID_AUTO, delay_max, CTLFLAG_RW, &sx_delay.max, 0, ""); -LOCK_DELAY_SYSINIT_DEFAULT(sx_delay); +static void +sx_lock_delay_init(void *arg __unused) +{ + + lock_delay_default_init(&sx_delay); + asx_retries = 10; + asx_loops = max(10000, sx_delay.max); +} +LOCK_DELAY_SYSINIT(sx_lock_delay_init); #endif void