Date: Wed, 29 Dec 2021 19:36:44 +0000 From: bugzilla-noreply@freebsd.org To: bugs@FreeBSD.org Subject: [Bug 260805] sysctl shows bogus numa domains Message-ID: <bug-260805-227@https.bugs.freebsd.org/bugzilla/>
next in thread | raw e-mail | index | archive | help
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D260805 Bug ID: 260805 Summary: sysctl shows bogus numa domains Product: Base System Version: 12.3-RELEASE Hardware: Any OS: Any Status: New Severity: Affects Only Me Priority: --- Component: kern Assignee: bugs@FreeBSD.org Reporter: pmc@citylink.dinoex.sub.org Environmental Data: ------------------- ASUS Z10PA-D8, Cluster-on-Die and NUMA enabled. CPU: Intel(R) Xeon(R) CPU E5-2660 v3 @ 2.60GHz (2594.04-MHz K8-class CPU) FreeBSD/SMP: Multiprocessor System Detected: 20 CPUs FreeBSD/SMP: 1 package(s) x 2 cache groups x 5 core(s) x 2 hardware threads kern.sched.topology_spec: <groups> <group level=3D"1" cache-level=3D"0"> <cpu count=3D"20" mask=3D"fffff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, = 11, 12, 13, 14, 15, 16, 17, 18, 19</cpu> <children> <group level=3D"2" cache-level=3D"3"> <cpu count=3D"10" mask=3D"3ff,0,0,0">0, 1, 2, 3, 4, 5, 6, 7, 8, 9</cpu> <children> <group level=3D"3" cache-level=3D"2"> <cpu count=3D"2" mask=3D"3,0,0,0">0, 1</cpu> <flags><flag name=3D"THREAD">THREAD group</flag><flag name=3D"SMT">SMT group</flag></flags> </group> <group level=3D"3" cache-level=3D"2"> <cpu count=3D"2" mask=3D"c,0,0,0">2, 3</cpu> <flags><flag name=3D"THREAD">THREAD group</flag><flag name=3D"SMT">SMT group</flag></flags> </group> <group level=3D"3" cache-level=3D"2"> <cpu count=3D"2" mask=3D"30,0,0,0">4, 5</cpu> <flags><flag name=3D"THREAD">THREAD group</flag><flag name=3D"SMT">SMT group</flag></flags> </group> [etc.etc.] This would say that cpu.4 and cpu.5 are SMT brothers on the same processor. But, when it comes to NUMA domains, sysctl says this: dev.cpu.19.%domain: 1 dev.cpu.18.%domain: 1 dev.cpu.17.%domain: 1 dev.cpu.16.%domain: 1 dev.cpu.15.%domain: 1 dev.cpu.14.%domain: 0 dev.cpu.13.%domain: 0 dev.cpu.12.%domain: 0 dev.cpu.11.%domain: 0 dev.cpu.10.%domain: 0 dev.cpu.9.%domain: 1 dev.cpu.8.%domain: 1 dev.cpu.7.%domain: 1 dev.cpu.6.%domain: 1 dev.cpu.5.%domain: 1 dev.cpu.4.%domain: 0 dev.cpu.3.%domain: 0 dev.cpu.2.%domain: 0 dev.cpu.1.%domain: 0 dev.cpu.0.%domain: 0 As it is highly unlikely for SMT brothers to share L1/L2 cache but use different memory banks, something must be wrong here. I did test with=20 cpuset -l 4 -n ft:0-1 sh -c "awk 'END {for(i=3D0; i < 1000000000; i++) { a[= i] =3D 2.53 } }' < /dev/null" cpuset -l 5 -n ft:0-1 sh -c "awk 'END {for(i=3D0; i < 1000000000; i++) { a[= i] =3D 2.53 } }' < /dev/null" and both do allocate from vm.domain.0.stats.free_count, so it seems the dev.cpu.X.%domain values are wrong, and there seems no obvious harm resulti= ng from it (at least until somebody grabs these dev.cpu.X.%domain values to ma= ke their software numa-aware).=20 As I have no other numa-capable hardware available, I cannot say if this mi= ght be just an individual flaw in this BIOS, or such. --=20 You are receiving this mail because: You are the assignee for the bug.=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-260805-227>