From owner-svn-src-all@freebsd.org Sat May 18 11:14:45 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D6A1E15AF3EB; Sat, 18 May 2019 11:14:44 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 75CCC752C6; Sat, 18 May 2019 11:14:44 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 65E0B1ED6A; Sat, 18 May 2019 11:14:44 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x4IBEiLG036530; Sat, 18 May 2019 11:14:44 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x4IBEixh036528; Sat, 18 May 2019 11:14:44 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201905181114.x4IBEixh036528@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Sat, 18 May 2019 11:14:44 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r347952 - head/sys/powerpc/aim X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/powerpc/aim X-SVN-Commit-Revision: 347952 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 75CCC752C6 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.984,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 May 2019 11:14:45 -0000 Author: jhibbits Date: Sat May 18 11:14:43 2019 New Revision: 347952 URL: https://svnweb.freebsd.org/changeset/base/347952 Log: powerpc64/pmap: NUMA-ize the page pv lock pool to reduce contention It was found during building llvm that the page pv lock pool was seeing very high contention. Since the pmap is already NUMA aware, it was surmised that the domains were referencing similar pages in the different domains. This reduces contention to the point of noise in a lockstat(8) run (~51% down to under 5%), reducing build times by up to 20%. This doesn't do a perfect domain alignment, just a best-guess based on hardware available, that the domain is roughly specified in the upper bits of the PA. Trying to be more clever would more than likely result in reduced performance just on the work needed. MFC after: 2 weeks Modified: head/sys/powerpc/aim/mmu_oea64.c Modified: head/sys/powerpc/aim/mmu_oea64.c ============================================================================== --- head/sys/powerpc/aim/mmu_oea64.c Sat May 18 03:15:07 2019 (r347951) +++ head/sys/powerpc/aim/mmu_oea64.c Sat May 18 11:14:43 2019 (r347952) @@ -118,10 +118,18 @@ uintptr_t moea64_get_unique_vsid(void); * */ -#define PV_LOCK_COUNT PA_LOCK_COUNT*3 +#define PV_LOCK_PER_DOM PA_LOCK_COUNT*3 +#define PV_LOCK_COUNT PV_LOCK_PER_DOM*MAXMEMDOM static struct mtx_padalign pv_lock[PV_LOCK_COUNT]; -#define PV_LOCKPTR(pa) ((struct mtx *)(&pv_lock[pa_index(pa) % PV_LOCK_COUNT])) +/* + * Cheap NUMA-izing of the pv locks, to reduce contention across domains. + * NUMA domains on POWER9 appear to be indexed as sparse memory spaces, with the + * index at (N << 45). + */ +#define PV_LOCK_IDX(pa) (pa_index(pa) % PV_LOCK_PER_DOM + \ + (((pa) >> 45) % MAXMEMDOM) * PV_LOCK_PER_DOM) +#define PV_LOCKPTR(pa) ((struct mtx *)(&pv_lock[PV_LOCK_IDX(pa)])) #define PV_LOCK(pa) mtx_lock(PV_LOCKPTR(pa)) #define PV_UNLOCK(pa) mtx_unlock(PV_LOCKPTR(pa)) #define PV_LOCKASSERT(pa) mtx_assert(PV_LOCKPTR(pa), MA_OWNED)