From owner-svn-src-all@freebsd.org Fri Oct 25 14:51:23 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 919F1172BAC; Fri, 25 Oct 2019 14:51:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4706WH3MFbz4bZ3; Fri, 25 Oct 2019 14:51:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 569F127FD2; Fri, 25 Oct 2019 14:51:23 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x9PEpNdQ002433; Fri, 25 Oct 2019 14:51:23 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x9PEpLvA002425; Fri, 25 Oct 2019 14:51:21 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201910251451.x9PEpLvA002425@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Fri, 25 Oct 2019 14:51:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r354073 - in stable/12: share/man/man4 sys/kern sys/powerpc/powernv sys/powerpc/powerpc sys/powerpc/ps3 sys/powerpc/pseries sys/sys sys/x86/x86 X-SVN-Group: stable-12 X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in stable/12: share/man/man4 sys/kern sys/powerpc/powernv sys/powerpc/powerpc sys/powerpc/ps3 sys/powerpc/pseries sys/sys sys/x86/x86 X-SVN-Commit-Revision: 354073 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Oct 2019 14:51:23 -0000 Author: mav Date: Fri Oct 25 14:51:21 2019 New Revision: 354073 URL: https://svnweb.freebsd.org/changeset/base/354073 Log: MFC r342771 (by cem): Expose threads-per-core and physical core count information With new sysctls (to the best of our ability do detect them). Restructured smp.4 slightly for clarity (keep relevant stuff closer to the top) while documenting. Modified: stable/12/share/man/man4/smp.4 stable/12/sys/kern/subr_smp.c stable/12/sys/powerpc/powernv/platform_powernv.c stable/12/sys/powerpc/powerpc/mp_machdep.c stable/12/sys/powerpc/ps3/platform_ps3.c stable/12/sys/powerpc/pseries/platform_chrp.c stable/12/sys/sys/smp.h stable/12/sys/x86/x86/mp_x86.c Directory Properties: stable/12/ (props changed) Modified: stable/12/share/man/man4/smp.4 ============================================================================== --- stable/12/share/man/man4/smp.4 Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/share/man/man4/smp.4 Fri Oct 25 14:51:21 2019 (r354073) @@ -23,7 +23,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 6, 2018 +.Dd January 4, 2019 .Dt SMP 4 .Os .Sh NAME @@ -35,27 +35,6 @@ The .Nm kernel implements symmetric multi-processor support. -.Sh COMPATIBILITY -Support for multi-processor systems is present for all Tier-1 -architectures on -.Fx . -Currently, this includes amd64, i386 and sparc64. -Support is enabled using -.Cd options SMP . -It is permissible to use the SMP kernel configuration on non-SMP equipped -motherboards. -.Sh I386 NOTES -For i386 systems, the -.Nm -kernel supports motherboards that follow the Intel MP specification, -version 1.4. -In addition to -.Cd options SMP , -i386 also requires -.Cd device apic . -The -.Xr mptable 1 -command may be used to view the status of multi-processor support. .Pp .Nm support can be disabled by setting the loader tunable @@ -66,6 +45,13 @@ The number of CPUs detected by the system is available the read-only sysctl variable .Va hw.ncpu . .Pp +The number of online threads per CPU core is available in the read-only sysctl +variable +.Va kern.smp.threads_per_core . +The number of physical CPU cores detected by the system is available in the +read-only sysctl variable +.Va kern.smp.cores . +.Pp .Fx allows specific CPUs on a multi-processor system to be disabled. This can be done using the @@ -74,6 +60,12 @@ tunable, where X is the APIC ID of a CPU. Setting this tunable to 1 will result in the corresponding CPU being disabled. .Pp +.Fx +supports simultaneous multithreading on x86 and powerpc platforms. +On x86, the logical CPUs can be disabled by setting the +.Va machdep.hyperthreading_allowed +tunable to zero. +.Pp The .Xr sched_ule 4 scheduler implements CPU topology detection and adjusts the scheduling @@ -122,13 +114,26 @@ two quad-core processors is: .Pp This information is used internally by the kernel to schedule related tasks on CPUs that are closely grouped together. -.Pp -.Fx -supports hyperthreading on Intel CPU's on the i386 and AMD64 platforms. -Because using logical CPUs can cause performance penalties under certain loads, -the logical CPUs can be disabled by setting the -.Va machdep.hyperthreading_allowed -tunable to zero. +.Sh COMPATIBILITY +Support for multi-processor systems is present for all Tier-1 and Tier-2 +architectures on +.Fx . +Currently, this includes x86, powerpc, arm, and sparc64. +Support is enabled using +.Cd options SMP . +It is permissible to use the SMP kernel configuration on non-SMP hardware. +.Sh I386 NOTES +For i386 systems, the +.Nm +kernel supports motherboards that follow the Intel MP specification, +version 1.4. +In addition to +.Cd options SMP , +i386 also requires +.Cd device apic . +The +.Xr mptable 1 +command may be used to view the status of multi-processor support. .Sh SEE ALSO .Xr cpuset 1 , .Xr mptable 1 , @@ -166,3 +171,20 @@ in also introduced support for SMP on the sparc64 architecture. .Sh AUTHORS .An Steve Passe Aq Mt fsmp@FreeBSD.org +.Sh CAVEATS +The +.Va kern.smp.threads_per_core +and +.Va kern.smp.cores +sysctl variables are provided as a best-effort guess. +If an architecture or platform adds SMT and +.Fx +has not yet implemented detection, the reported values may be inaccurate. +In this case, +.Va kern.smp.threads_per_core +will report +.Dv 1 +and +.Va kern.smp.cores +will report the same value as +.Va hw.ncpu . Modified: stable/12/sys/kern/subr_smp.c ============================================================================== --- stable/12/sys/kern/subr_smp.c Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/kern/subr_smp.c Fri Oct 25 14:51:21 2019 (r354073) @@ -98,6 +98,14 @@ int smp_cpus = 1; /* how many cpu's running */ SYSCTL_INT(_kern_smp, OID_AUTO, cpus, CTLFLAG_RD|CTLFLAG_CAPRD, &smp_cpus, 0, "Number of CPUs online"); +int smp_threads_per_core = 1; /* how many SMT threads are running per core */ +SYSCTL_INT(_kern_smp, OID_AUTO, threads_per_core, CTLFLAG_RD|CTLFLAG_CAPRD, + &smp_threads_per_core, 0, "Number of SMT threads online per core"); + +int mp_ncores = -1; /* how many physical cores running */ +SYSCTL_INT(_kern_smp, OID_AUTO, cores, CTLFLAG_RD|CTLFLAG_CAPRD, &mp_ncores, 0, + "Number of CPUs online"); + int smp_topology = 0; /* Which topology we're using. */ SYSCTL_INT(_kern_smp, OID_AUTO, topology, CTLFLAG_RDTUN, &smp_topology, 0, "Topology override setting; 0 is default provided by hardware."); @@ -154,6 +162,7 @@ mp_start(void *dummy) /* Probe for MP hardware. */ if (smp_disabled != 0 || cpu_mp_probe() == 0) { + mp_ncores = 1; mp_ncpus = 1; CPU_SETOF(PCPU_GET(cpuid), &all_cpus); return; @@ -162,6 +171,11 @@ mp_start(void *dummy) cpu_mp_start(); printf("FreeBSD/SMP: Multiprocessor System Detected: %d CPUs\n", mp_ncpus); + + /* Provide a default for most architectures that don't have SMT/HTT. */ + if (mp_ncores < 0) + mp_ncores = mp_ncpus; + cpu_mp_announce(); } SYSINIT(cpu_mp, SI_SUB_CPU, SI_ORDER_THIRD, mp_start, NULL); @@ -823,6 +837,7 @@ static void mp_setvariables_for_up(void *dummy) { mp_ncpus = 1; + mp_ncores = 1; mp_maxid = PCPU_GET(cpuid); CPU_SETOF(mp_maxid, &all_cpus); KASSERT(PCPU_GET(cpuid) == 0, ("UP must have a CPU ID of zero")); Modified: stable/12/sys/powerpc/powernv/platform_powernv.c ============================================================================== --- stable/12/sys/powerpc/powernv/platform_powernv.c Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/powerpc/powernv/platform_powernv.c Fri Oct 25 14:51:21 2019 (r354073) @@ -435,12 +435,16 @@ powernv_smp_topo(platform_t plat) break; } + smp_threads_per_core = nthreads; + if (mp_ncpus % nthreads != 0) { printf("WARNING: Irregular SMP topology. Performance may be " "suboptimal (%d threads, %d on first core)\n", mp_ncpus, nthreads); return (smp_topo_none()); } + + mp_ncores = mp_ncpus / nthreads; /* Don't do anything fancier for non-threaded SMP */ if (nthreads == 1) Modified: stable/12/sys/powerpc/powerpc/mp_machdep.c ============================================================================== --- stable/12/sys/powerpc/powerpc/mp_machdep.c Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/powerpc/powerpc/mp_machdep.c Fri Oct 25 14:51:21 2019 (r354073) @@ -182,6 +182,11 @@ cpu_mp_start(void) next: error = platform_smp_next_cpu(&cpu); } + +#ifdef SMP + /* Probe mp_ncores and smp_threads_per_core as a side effect. */ + (void)cpu_topo(); +#endif } void Modified: stable/12/sys/powerpc/ps3/platform_ps3.c ============================================================================== --- stable/12/sys/powerpc/ps3/platform_ps3.c Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/powerpc/ps3/platform_ps3.c Fri Oct 25 14:51:21 2019 (r354073) @@ -246,6 +246,8 @@ ps3_smp_start_cpu(platform_t plat, struct pcpu *pc) static struct cpu_group * ps3_smp_topo(platform_t plat) { + mp_ncores = 1; + smp_threads_per_core = 2; return (smp_topo_1level(CG_SHARE_L1, 2, CG_FLAG_SMT)); } #endif Modified: stable/12/sys/powerpc/pseries/platform_chrp.c ============================================================================== --- stable/12/sys/powerpc/pseries/platform_chrp.c Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/powerpc/pseries/platform_chrp.c Fri Oct 25 14:51:21 2019 (r354073) @@ -517,6 +517,8 @@ chrp_smp_topo(platform_t plat) ncpus++; } + mp_ncores = ncores; + if (ncpus % ncores != 0) { printf("WARNING: Irregular SMP topology. Performance may be " "suboptimal (%d CPUS, %d cores)\n", ncpus, ncores); @@ -527,6 +529,7 @@ chrp_smp_topo(platform_t plat) if (ncpus == ncores) return (smp_topo_none()); + smp_threads_per_core = ncpus / ncores; return (smp_topo_1level(CG_SHARE_L1, ncpus / ncores, CG_FLAG_SMT)); } #endif Modified: stable/12/sys/sys/smp.h ============================================================================== --- stable/12/sys/sys/smp.h Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/sys/smp.h Fri Oct 25 14:51:21 2019 (r354073) @@ -167,8 +167,10 @@ extern cpuset_t logical_cpus_mask; extern u_int mp_maxid; extern int mp_maxcpus; +extern int mp_ncores; extern int mp_ncpus; extern volatile int smp_started; +extern int smp_threads_per_core; extern cpuset_t all_cpus; extern cpuset_t cpuset_domain[MAXMEMDOM]; /* CPUs in each NUMA domain. */ Modified: stable/12/sys/x86/x86/mp_x86.c ============================================================================== --- stable/12/sys/x86/x86/mp_x86.c Fri Oct 25 14:46:09 2019 (r354072) +++ stable/12/sys/x86/x86/mp_x86.c Fri Oct 25 14:51:21 2019 (r354073) @@ -608,6 +608,7 @@ assign_cpu_ids(void) { struct topo_node *node; u_int smt_mask; + int nhyper; smt_mask = (1u << core_id_shift) - 1; @@ -616,6 +617,7 @@ assign_cpu_ids(void) * beyond MAXCPU. CPU 0 is always assigned to the BSP. */ mp_ncpus = 0; + nhyper = 0; TOPO_FOREACH(node, &topo_root) { if (node->type != TOPO_TYPE_PU) continue; @@ -643,6 +645,9 @@ assign_cpu_ids(void) continue; } + if (cpu_info[node->hwid].cpu_hyperthread) + nhyper++; + cpu_apic_ids[mp_ncpus] = node->hwid; apic_cpuids[node->hwid] = mp_ncpus; topo_set_pu_id(node, mp_ncpus); @@ -652,6 +657,9 @@ assign_cpu_ids(void) KASSERT(mp_maxid >= mp_ncpus - 1, ("%s: counters out of sync: max %d, count %d", __func__, mp_maxid, mp_ncpus)); + + mp_ncores = mp_ncpus - nhyper; + smp_threads_per_core = mp_ncpus / mp_ncores; } /*