From owner-svn-src-all@freebsd.org Fri Jan 4 18:31:20 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0178C1434BFD; Fri, 4 Jan 2019 18:31:20 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 9BBDA75466; Fri, 4 Jan 2019 18:31:19 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 85B8149CC; Fri, 4 Jan 2019 18:31:19 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x04IVJ9k071000; Fri, 4 Jan 2019 18:31:19 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x04IVHa5070991; Fri, 4 Jan 2019 18:31:17 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201901041831.x04IVHa5070991@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: Conrad Meyer Date: Fri, 4 Jan 2019 18:31:17 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r342771 - in head: share/man/man4 sys/kern sys/powerpc/powernv sys/powerpc/powerpc sys/powerpc/ps3 sys/powerpc/pseries sys/sys sys/x86/x86 X-SVN-Group: head X-SVN-Commit-Author: cem X-SVN-Commit-Paths: in head: share/man/man4 sys/kern sys/powerpc/powernv sys/powerpc/powerpc sys/powerpc/ps3 sys/powerpc/pseries sys/sys sys/x86/x86 X-SVN-Commit-Revision: 342771 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 9BBDA75466 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.97)[-0.967,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-0.999,0] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Jan 2019 18:31:20 -0000 Author: cem Date: Fri Jan 4 18:31:17 2019 New Revision: 342771 URL: https://svnweb.freebsd.org/changeset/base/342771 Log: Expose threads-per-core and physical core count information With new sysctls (to the best of our ability do detect them). Restructured smp.4 slightly for clarity (keep relevant stuff closer to the top) while documenting. Reviewed by: markj, jhibbits (ppc parts) MFC after: 3 days Sponsored by: Dell EMC Isilon Differential Revision: https://reviews.freebsd.org/D18322 Modified: head/share/man/man4/smp.4 head/sys/kern/subr_smp.c head/sys/powerpc/powernv/platform_powernv.c head/sys/powerpc/powerpc/mp_machdep.c head/sys/powerpc/ps3/platform_ps3.c head/sys/powerpc/pseries/platform_chrp.c head/sys/sys/smp.h head/sys/x86/x86/mp_x86.c Modified: head/share/man/man4/smp.4 ============================================================================== --- head/share/man/man4/smp.4 Fri Jan 4 18:21:49 2019 (r342770) +++ head/share/man/man4/smp.4 Fri Jan 4 18:31:17 2019 (r342771) @@ -23,7 +23,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 6, 2018 +.Dd January 4, 2019 .Dt SMP 4 .Os .Sh NAME @@ -35,27 +35,6 @@ The .Nm kernel implements symmetric multi-processor support. -.Sh COMPATIBILITY -Support for multi-processor systems is present for all Tier-1 -architectures on -.Fx . -Currently, this includes amd64, i386 and sparc64. -Support is enabled using -.Cd options SMP . -It is permissible to use the SMP kernel configuration on non-SMP equipped -motherboards. -.Sh I386 NOTES -For i386 systems, the -.Nm -kernel supports motherboards that follow the Intel MP specification, -version 1.4. -In addition to -.Cd options SMP , -i386 also requires -.Cd device apic . -The -.Xr mptable 1 -command may be used to view the status of multi-processor support. .Pp .Nm support can be disabled by setting the loader tunable @@ -66,6 +45,13 @@ The number of CPUs detected by the system is available the read-only sysctl variable .Va hw.ncpu . .Pp +The number of online threads per CPU core is available in the read-only sysctl +variable +.Va kern.smp.threads_per_core . +The number of physical CPU cores detected by the system is available in the +read-only sysctl variable +.Va kern.smp.cores . +.Pp .Fx allows specific CPUs on a multi-processor system to be disabled. This can be done using the @@ -74,6 +60,12 @@ tunable, where X is the APIC ID of a CPU. Setting this tunable to 1 will result in the corresponding CPU being disabled. .Pp +.Fx +supports simultaneous multithreading on x86 and powerpc platforms. +On x86, the logical CPUs can be disabled by setting the +.Va machdep.hyperthreading_allowed +tunable to zero. +.Pp The .Xr sched_ule 4 scheduler implements CPU topology detection and adjusts the scheduling @@ -122,13 +114,26 @@ two quad-core processors is: .Pp This information is used internally by the kernel to schedule related tasks on CPUs that are closely grouped together. -.Pp -.Fx -supports hyperthreading on Intel CPU's on the i386 and AMD64 platforms. -Because using logical CPUs can cause performance penalties under certain loads, -the logical CPUs can be disabled by setting the -.Va machdep.hyperthreading_allowed -tunable to zero. +.Sh COMPATIBILITY +Support for multi-processor systems is present for all Tier-1 and Tier-2 +architectures on +.Fx . +Currently, this includes x86, powerpc, arm, and sparc64. +Support is enabled using +.Cd options SMP . +It is permissible to use the SMP kernel configuration on non-SMP hardware. +.Sh I386 NOTES +For i386 systems, the +.Nm +kernel supports motherboards that follow the Intel MP specification, +version 1.4. +In addition to +.Cd options SMP , +i386 also requires +.Cd device apic . +The +.Xr mptable 1 +command may be used to view the status of multi-processor support. .Sh SEE ALSO .Xr cpuset 1 , .Xr mptable 1 , @@ -166,3 +171,20 @@ in also introduced support for SMP on the sparc64 architecture. .Sh AUTHORS .An Steve Passe Aq Mt fsmp@FreeBSD.org +.Sh CAVEATS +The +.Va kern.smp.threads_per_core +and +.Va kern.smp.cores +sysctl variables are provided as a best-effort guess. +If an architecture or platform adds SMT and +.Fx +has not yet implemented detection, the reported values may be inaccurate. +In this case, +.Va kern.smp.threads_per_core +will report +.Dv 1 +and +.Va kern.smp.cores +will report the same value as +.Va hw.ncpu . Modified: head/sys/kern/subr_smp.c ============================================================================== --- head/sys/kern/subr_smp.c Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/kern/subr_smp.c Fri Jan 4 18:31:17 2019 (r342771) @@ -98,6 +98,14 @@ int smp_cpus = 1; /* how many cpu's running */ SYSCTL_INT(_kern_smp, OID_AUTO, cpus, CTLFLAG_RD|CTLFLAG_CAPRD, &smp_cpus, 0, "Number of CPUs online"); +int smp_threads_per_core = 1; /* how many SMT threads are running per core */ +SYSCTL_INT(_kern_smp, OID_AUTO, threads_per_core, CTLFLAG_RD|CTLFLAG_CAPRD, + &smp_threads_per_core, 0, "Number of SMT threads online per core"); + +int mp_ncores = -1; /* how many physical cores running */ +SYSCTL_INT(_kern_smp, OID_AUTO, cores, CTLFLAG_RD|CTLFLAG_CAPRD, &mp_ncores, 0, + "Number of CPUs online"); + int smp_topology = 0; /* Which topology we're using. */ SYSCTL_INT(_kern_smp, OID_AUTO, topology, CTLFLAG_RDTUN, &smp_topology, 0, "Topology override setting; 0 is default provided by hardware."); @@ -154,6 +162,7 @@ mp_start(void *dummy) /* Probe for MP hardware. */ if (smp_disabled != 0 || cpu_mp_probe() == 0) { + mp_ncores = 1; mp_ncpus = 1; CPU_SETOF(PCPU_GET(cpuid), &all_cpus); return; @@ -162,6 +171,11 @@ mp_start(void *dummy) cpu_mp_start(); printf("FreeBSD/SMP: Multiprocessor System Detected: %d CPUs\n", mp_ncpus); + + /* Provide a default for most architectures that don't have SMT/HTT. */ + if (mp_ncores < 0) + mp_ncores = mp_ncpus; + cpu_mp_announce(); } SYSINIT(cpu_mp, SI_SUB_CPU, SI_ORDER_THIRD, mp_start, NULL); @@ -823,6 +837,7 @@ static void mp_setvariables_for_up(void *dummy) { mp_ncpus = 1; + mp_ncores = 1; mp_maxid = PCPU_GET(cpuid); CPU_SETOF(mp_maxid, &all_cpus); KASSERT(PCPU_GET(cpuid) == 0, ("UP must have a CPU ID of zero")); Modified: head/sys/powerpc/powernv/platform_powernv.c ============================================================================== --- head/sys/powerpc/powernv/platform_powernv.c Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/powerpc/powernv/platform_powernv.c Fri Jan 4 18:31:17 2019 (r342771) @@ -435,12 +435,16 @@ powernv_smp_topo(platform_t plat) break; } + smp_threads_per_core = nthreads; + if (mp_ncpus % nthreads != 0) { printf("WARNING: Irregular SMP topology. Performance may be " "suboptimal (%d threads, %d on first core)\n", mp_ncpus, nthreads); return (smp_topo_none()); } + + mp_ncores = mp_ncpus / nthreads; /* Don't do anything fancier for non-threaded SMP */ if (nthreads == 1) Modified: head/sys/powerpc/powerpc/mp_machdep.c ============================================================================== --- head/sys/powerpc/powerpc/mp_machdep.c Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/powerpc/powerpc/mp_machdep.c Fri Jan 4 18:31:17 2019 (r342771) @@ -186,6 +186,11 @@ cpu_mp_start(void) next: error = platform_smp_next_cpu(&cpu); } + +#ifdef SMP + /* Probe mp_ncores and smp_threads_per_core as a side effect. */ + (void)cpu_topo(); +#endif } void Modified: head/sys/powerpc/ps3/platform_ps3.c ============================================================================== --- head/sys/powerpc/ps3/platform_ps3.c Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/powerpc/ps3/platform_ps3.c Fri Jan 4 18:31:17 2019 (r342771) @@ -246,6 +246,8 @@ ps3_smp_start_cpu(platform_t plat, struct pcpu *pc) static struct cpu_group * ps3_smp_topo(platform_t plat) { + mp_ncores = 1; + smp_threads_per_core = 2; return (smp_topo_1level(CG_SHARE_L1, 2, CG_FLAG_SMT)); } #endif Modified: head/sys/powerpc/pseries/platform_chrp.c ============================================================================== --- head/sys/powerpc/pseries/platform_chrp.c Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/powerpc/pseries/platform_chrp.c Fri Jan 4 18:31:17 2019 (r342771) @@ -517,6 +517,8 @@ chrp_smp_topo(platform_t plat) ncpus++; } + mp_ncores = ncores; + if (ncpus % ncores != 0) { printf("WARNING: Irregular SMP topology. Performance may be " "suboptimal (%d CPUS, %d cores)\n", ncpus, ncores); @@ -527,6 +529,7 @@ chrp_smp_topo(platform_t plat) if (ncpus == ncores) return (smp_topo_none()); + smp_threads_per_core = ncpus / ncores; return (smp_topo_1level(CG_SHARE_L1, ncpus / ncores, CG_FLAG_SMT)); } #endif Modified: head/sys/sys/smp.h ============================================================================== --- head/sys/sys/smp.h Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/sys/smp.h Fri Jan 4 18:31:17 2019 (r342771) @@ -167,8 +167,10 @@ extern cpuset_t logical_cpus_mask; extern u_int mp_maxid; extern int mp_maxcpus; +extern int mp_ncores; extern int mp_ncpus; extern volatile int smp_started; +extern int smp_threads_per_core; extern cpuset_t all_cpus; extern cpuset_t cpuset_domain[MAXMEMDOM]; /* CPUs in each NUMA domain. */ Modified: head/sys/x86/x86/mp_x86.c ============================================================================== --- head/sys/x86/x86/mp_x86.c Fri Jan 4 18:21:49 2019 (r342770) +++ head/sys/x86/x86/mp_x86.c Fri Jan 4 18:31:17 2019 (r342771) @@ -607,6 +607,7 @@ assign_cpu_ids(void) { struct topo_node *node; u_int smt_mask; + int nhyper; smt_mask = (1u << core_id_shift) - 1; @@ -615,6 +616,7 @@ assign_cpu_ids(void) * beyond MAXCPU. CPU 0 is always assigned to the BSP. */ mp_ncpus = 0; + nhyper = 0; TOPO_FOREACH(node, &topo_root) { if (node->type != TOPO_TYPE_PU) continue; @@ -642,6 +644,9 @@ assign_cpu_ids(void) continue; } + if (cpu_info[node->hwid].cpu_hyperthread) + nhyper++; + cpu_apic_ids[mp_ncpus] = node->hwid; apic_cpuids[node->hwid] = mp_ncpus; topo_set_pu_id(node, mp_ncpus); @@ -651,6 +656,9 @@ assign_cpu_ids(void) KASSERT(mp_maxid >= mp_ncpus - 1, ("%s: counters out of sync: max %d, count %d", __func__, mp_maxid, mp_ncpus)); + + mp_ncores = mp_ncpus - nhyper; + smp_threads_per_core = mp_ncpus / mp_ncores; } /*