Date: Sat, 18 May 2019 11:14:44 +0000 (UTC) From: Justin Hibbits <jhibbits@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r347952 - head/sys/powerpc/aim Message-ID: <201905181114.x4IBEixh036528@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: jhibbits Date: Sat May 18 11:14:43 2019 New Revision: 347952 URL: https://svnweb.freebsd.org/changeset/base/347952 Log: powerpc64/pmap: NUMA-ize the page pv lock pool to reduce contention It was found during building llvm that the page pv lock pool was seeing very high contention. Since the pmap is already NUMA aware, it was surmised that the domains were referencing similar pages in the different domains. This reduces contention to the point of noise in a lockstat(8) run (~51% down to under 5%), reducing build times by up to 20%. This doesn't do a perfect domain alignment, just a best-guess based on hardware available, that the domain is roughly specified in the upper bits of the PA. Trying to be more clever would more than likely result in reduced performance just on the work needed. MFC after: 2 weeks Modified: head/sys/powerpc/aim/mmu_oea64.c Modified: head/sys/powerpc/aim/mmu_oea64.c ============================================================================== --- head/sys/powerpc/aim/mmu_oea64.c Sat May 18 03:15:07 2019 (r347951) +++ head/sys/powerpc/aim/mmu_oea64.c Sat May 18 11:14:43 2019 (r347952) @@ -118,10 +118,18 @@ uintptr_t moea64_get_unique_vsid(void); * */ -#define PV_LOCK_COUNT PA_LOCK_COUNT*3 +#define PV_LOCK_PER_DOM PA_LOCK_COUNT*3 +#define PV_LOCK_COUNT PV_LOCK_PER_DOM*MAXMEMDOM static struct mtx_padalign pv_lock[PV_LOCK_COUNT]; -#define PV_LOCKPTR(pa) ((struct mtx *)(&pv_lock[pa_index(pa) % PV_LOCK_COUNT])) +/* + * Cheap NUMA-izing of the pv locks, to reduce contention across domains. + * NUMA domains on POWER9 appear to be indexed as sparse memory spaces, with the + * index at (N << 45). + */ +#define PV_LOCK_IDX(pa) (pa_index(pa) % PV_LOCK_PER_DOM + \ + (((pa) >> 45) % MAXMEMDOM) * PV_LOCK_PER_DOM) +#define PV_LOCKPTR(pa) ((struct mtx *)(&pv_lock[PV_LOCK_IDX(pa)])) #define PV_LOCK(pa) mtx_lock(PV_LOCKPTR(pa)) #define PV_UNLOCK(pa) mtx_unlock(PV_LOCKPTR(pa)) #define PV_LOCKASSERT(pa) mtx_assert(PV_LOCKPTR(pa), MA_OWNED)
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201905181114.x4IBEixh036528>