Date: Tue, 12 Jan 2021 01:10:06 GMT From: Andrew Gallatin <gallatin@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Subject: git: 7eaea04a5bb1 - main - amd64: compare TLB shootdown target to all_cpus Message-ID: <202101120110.10C1A6mg069684@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch main has been updated by gallatin: URL: https://cgit.FreeBSD.org/src/commit/?id=7eaea04a5bb1dc86c43ce046311e1c1a042994d3 commit 7eaea04a5bb1dc86c43ce046311e1c1a042994d3 Author: Andrew Gallatin <gallatin@FreeBSD.org> AuthorDate: 2021-01-12 01:03:37 +0000 Commit: Andrew Gallatin <gallatin@FreeBSD.org> CommitDate: 2021-01-12 01:09:32 +0000 amd64: compare TLB shootdown target to all_cpus On amd64, the pmap code passes all_cpus to smp_targeted_tlb_shootdown() when unmapping from the kernel pmap. This function has an optimized path to send IPIs to all but itself, which it intends to do when the target is all cpus. However, we need to compare the target cpu mask with all_cpus, rather than using CPU_ISFULLSET(). Comparing with CPU_ISFULLSET() will only work when we have MAXCPU cpus active in the system, otherwise, we'll be sending repeated IPIs, rather than a single IPI to all CPUs but ourself. Fixing this should reduce the time spent in native_lapic_ipi_wait() as we will be sending ipis in parallel, rather than one-by-one. This is confirmed by dtrace. Reviewed by: alc, jhb, kib, markj Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D28102 --- sys/amd64/amd64/mp_machdep.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c index 63777014e151..794a11bf1276 100644 --- a/sys/amd64/amd64/mp_machdep.c +++ b/sys/amd64/amd64/mp_machdep.c @@ -673,7 +673,7 @@ smp_targeted_tlb_shootdown(cpuset_t mask, pmap_t pmap, vm_offset_t addr1, /* * Check for other cpus. Return if none. */ - if (CPU_ISFULLSET(&mask)) { + if (!CPU_CMP(&mask, &all_cpus)) { if (mp_ncpus <= 1) goto local_cb; } else { @@ -719,7 +719,7 @@ smp_targeted_tlb_shootdown(cpuset_t mask, pmap_t pmap, vm_offset_t addr1, * (zeroing slot) and reading from it below (wait for * acknowledgment). */ - if (CPU_ISFULLSET(&mask)) { + if (!CPU_CMP(&mask, &all_cpus)) { ipi_all_but_self(IPI_INVLOP); other_cpus = all_cpus; CPU_CLR(PCPU_GET(cpuid), &other_cpus);
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202101120110.10C1A6mg069684>