From nobody Thu Jan 15 19:21:20 2026 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dsXsP6GDDz6PL9L for ; Thu, 15 Jan 2026 19:21:29 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Received: from mail-oi1-x22c.google.com (mail-oi1-x22c.google.com [IPv6:2607:f8b0:4864:20::22c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4dsXsN48j6z3vp5 for ; Thu, 15 Jan 2026 19:21:28 +0000 (UTC) (envelope-from shawn.webb@hardenedbsd.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=hardenedbsd.org header.s=google header.b=AadBepIl; dmarc=pass (policy=none) header.from=hardenedbsd.org; spf=pass (mx1.freebsd.org: domain of shawn.webb@hardenedbsd.org designates 2607:f8b0:4864:20::22c as permitted sender) smtp.mailfrom=shawn.webb@hardenedbsd.org Received: by mail-oi1-x22c.google.com with SMTP id 5614622812f47-45c798c92edso752660b6e.1 for ; Thu, 15 Jan 2026 11:21:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hardenedbsd.org; s=google; t=1768504882; x=1769109682; darn=freebsd.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=7VCK+L+9EzJrr63/hrUlRSiPfQ+GRY+lRvnTV2XIFoU=; b=AadBepIlVBIOMgAWOCvTSfs1ZSkrVm5vj+4QzoED0aOTv7yZtIVtn9EnNiOlPkA0M3 eUzQOcabJyH2EBvapaXVXrNC2fnylAZ22brWVIJTuuo+nKJ5Q5WPc63/T2JkA1Kk+C9A +GMpo4KjvLa0h1qiGmOdQjZ2XYKjwfPo2oyXGtPt8iL01x6q2fDdfJTG6ddZF8ZcFdWE AOrUMjGLtCeuLeqrOBw1s7Ai1co3u2fe+v0L4eSrIJc1mueWWOfrcihOfz+xR/Z/mV5a Lm2L6SaWowj1JWjMGYDuNWy1+HGvPK3AhyS3R7BaC0f3oLXmRGSTVX7TXyE4eFb74DOf EdUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1768504882; x=1769109682; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7VCK+L+9EzJrr63/hrUlRSiPfQ+GRY+lRvnTV2XIFoU=; b=CUWNt4rtcL+F59sWoljKRBZzvHFmMK4QlfKZoVnlF7+xzw4f6zdy5hBBM8fdnUes7K DkUTViOda9RfB52Wyt9KKKHAE1NTV31weEUvImFbbSHaWH9wAC8lz3WfIZxo5qFUf2jk gp5h4HbOoqqCE+5iq3G+T6NlU9uv/zBV3iDPns3mU0tf5OTpA8Fg0dF+g3MXfjkSlFaB Dq/HxRnLAUFnwsJ6oUuvG0Wk82YR1/FBWFu9Fjpczd8apwnyGmMdpjzo4j5nXr82G+2w V4Au3Hyy2fPmdYailbkBtrz+9HQHzvOq+B/aCHCuPPwTL5U1NPFWox3kTFDVTKX3Gi2c 44IQ== X-Gm-Message-State: AOJu0Ywf17VK4Tl6JM5nyTUh8xKzqLUzzDXcOow55KpJ3JMbySvCAgbx y6X1dXmB+AhiEqdwlRZGXq/y6C61kfGFtwJ/la6HNUlSkihEfx7iopSQyTSWBM5Uog/eRmToiSm ou+On+v0= X-Gm-Gg: AY/fxX5ie1AsGGigeLHi1aRtegP4iwMH6n5XkwbFVd32p0ZxHKF+VT1XEGw9qnXiM5d 5hwRLPc69ornqT4EHF8GsI6MzjIivQ082yTR7qt2WU09a/JJXzloGb9C5yM5S6TS9v4jrlla2k3 bZNQyuLgizLiO4dNs1gdY8ANJF4peld7siWxLQrmk4o9ItBMnoGkulBoYn2IqgRMwzp6KZQrORi OYUs3f3EUNONTe5PWgP/anySsdNLA/AZru410ygGQLurtlgWTZCxlIb8TucOypLCKXqlS6V1hHf GSHyZDsQ6irzvbSS8C+5d04GJBhSU/TV3MAAD57iDohgyyHV/sxmp+KfteJXluwWSZOrCmFCxP+ OkB0VBwK2p3mIZqA81UqBl6l/lmSbq0aDENviFugWsr0oBhsBGFW5kVIqnwD05ko/L0CU X-Received: by 2002:a05:6808:191c:b0:455:7ccd:d11 with SMTP id 5614622812f47-45c9bf77e61mr277735b6e.24.1768504882142; Thu, 15 Jan 2026 11:21:22 -0800 (PST) Received: from mutt-hbsd ([2001:470:4001:1::95]) by smtp.gmail.com with ESMTPSA id 5614622812f47-45c9e08df1bsm78905b6e.22.2026.01.15.11.21.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jan 2026 11:21:21 -0800 (PST) Date: Thu, 15 Jan 2026 19:21:20 +0000 From: Shawn Webb To: Minsoo Choo Cc: freebsd-hackers Subject: Re: HMP scheduling on FreeBSD Message-ID: <5a2sxxiqkafxdolq362ivmpyfemi5wc75zknnuzqdhstezumrq@2p7o2dmf7r6n> X-Operating-System: FreeBSD mutt-hbsd 14.3-STABLE-HBSD FreeBSD 14.3-STABLE-HBSD HARDENEDBSD-14-STABLE amd64 X-PGP-Key: https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/blob/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc References: <0Ng09S3rEB0BvT9vzHqVKU7rWxoad96kjEc7U2LCwDFJKmmswXujip7qbRlo_BIhNKcI7d-2CUHdp9Dxr3-7hhafpD6uagJSFUCjtC9qRr4=@proton.me> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="qjjjsycu6227j47j" Content-Disposition: inline In-Reply-To: <0Ng09S3rEB0BvT9vzHqVKU7rWxoad96kjEc7U2LCwDFJKmmswXujip7qbRlo_BIhNKcI7d-2CUHdp9Dxr3-7hhafpD6uagJSFUCjtC9qRr4=@proton.me> X-Spamd-Bar: ----- X-Spamd-Result: default: False [-5.60 / 15.00]; SIGNED_PGP(-2.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[hardenedbsd.org,none]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[hardenedbsd.org:s=google]; TO_DN_ALL(0.00)[]; MISSING_XM_UA(0.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; RCPT_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::22c:from]; DKIM_TRACE(0.00)[hardenedbsd.org:+] X-Rspamd-Queue-Id: 4dsXsN48j6z3vp5 --qjjjsycu6227j47j Content-Type: text/plain; protected-headers=v1; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: HMP scheduling on FreeBSD MIME-Version: 1.0 On Wed, Jan 14, 2026 at 04:10:15PM +0000, Minsoo Choo wrote: > Greetings, >=20 > For the last few days, I've been working on scheduler optimization for he= terogeneous cores ("HMP scheduling" from now on) on FreeBSD. After days of = reading specs, planning structure, and writing code, I came to a model that= can be utilized for implementing HMP scheduling. I opened a first revision= on Phabricator (D54674) a few days ago but decided to introduce this model= in the mailing list and hear others' opinions. >=20 > First of all, HMP-related code (scheduling, provider, cpucap framework, e= tc) are only built and enabled when "options HMP" is specified in kernel co= nfiguration. Of course, this shouldn't be activated by default because it's= in experiment status. "options HMP" should be added to kernel configuratio= n to enable this featured and this won't be enabled by default until the ma= jority of developers believe that HMP scheduling is stabilized. >=20 > The first component of HMP scheduling is "cpucap". One issue with HMP sch= eduling is that identifying the capacity and scores of a processor (i.e. pr= oviders) is machine-dependent while the scheduler code should be machine-in= dependent, so cpucap acts as an interface between the scheduler and provide= rs. CPU capacity and scores are stored in pcpu structure while the machine'= s cpucap status (e.g. initialized, has dynamic scores, etc) is stored in gl= obal cpucap structure of type "cpucap_t". It also includes functions for sc= heduler and providers, such as accessors, setters, finding "best" cpu, etc.= The review (D54674) adds these facilities under HMP option. >=20 > What are capacity and scores? Capacity describes cpu's heterogeneity (mor= e information in sys/contrib/device-tree/Bindings/cpu/cpu-capacity.txt). Th= is is static; it is initialized on boot time (e.g. from device tree) and st= ored in pc_cap_capacity in pcpu after being normalized to 0-1024. If it is = not provided, the default value of 1024 will be used. It is static and core= specific; if a performance core has capacity of 1024 and a efficiency core= has capacity of 600, this means all performance cores have 1024 and all ef= ficiency cores have 600, and whether they are throttled or not, this inform= ation stays the same. For this nature, capacity is hint for loading balanci= ng. By dividing a core's capacity by total capacity, we can assign an equal= fraction of tasks to the core's run queue. These cpucap information, inclu= ding enablement status, has dynamic score, individual capacity and scores, = can be queried using sysctl. >=20 > On the other hand, scores reflect the real time status of a processor and= are normalized to 0-1024. Providers like Intel Thread Director give the cu= rrent status of each core and feed it to pc_cap_scores in pcpu. Scores are = used for thread selection. For example, if a performance core is experienci= ng throttling, its score could go down to 1000. In that case, the scheduler= will prefer core that has the highest score. Scores are completely optiona= l because many processors do not provide this, and when score information i= s absent, cpucap fall backs to capacity. Since scores are dynamic, they are= retrieved and set using atomic operations. >=20 > Providers feed capacity or score information to cpucap. Capacity provider= s such as device trees and ACPI CPPC run on boot time and feed capacity inf= ormation. If there is no capacity provider, (again) the default value is us= ed. Score providers such as Intel ITD are implemented as device drivers. We= had ITD implementation patch (D44453-D44459) back in May 2024 called cored= irector, but it was neglected. With some modification, this can be the firs= t cpucap provider. Providers cannot be loaded or unloaded on runtime. An ex= ception is Arm SCMI which doesn't feed information to cpucap, but instead, = the scheduler should control and request CPU's performance. >=20 > Before integrating scheduler and cpucap, I need to go through sched_ule.c= =E2=80=8B from top to bottom. After that, I'll add new functions or drop ex= isting ones from the cpucap framework then work on the integration. >=20 > If anyone is interested in this implementation, please reply to this thre= ad and/or review my cpucap patch at D54674. Hey Minsoo, Thank you very much for working on this! I've merged the patch locally on one of my systems running HardenedBSD. I can confirm compilation succeeds (and boots). I'll be paying attention to this thread for updates. I'm happy to test future patch revisions, too, so please reach out even if you want me to test a one-line change. I subscribed to patch D54674, so I should see updates there. One question I have regarding the security posture of heterogeneous cores is whether constant-time cryptographic/string/math/etc operations take between cores. Would it be possible for an attacker to do by virtue of the different types of cores? For example, when installing FreeBSD with a geli-backed root filesystem, geli determines how quickly cryptographic operations take and will adjust the number of rounds accordingly. If that operation takes place on an efficiency core, would that result in fewer encryption rounds with geli? Can some sort of timing oracle exist by virtue of the different cores? How are speculative execution vulnerabilities manifest? I'm not expecting an answer to any of these questions. I'm just writing down some thoughts. But, if anyone can answer those questions, all the better. :-) Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Signal Username: shawn_webb.74 Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --qjjjsycu6227j47j Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmlpPikACgkQ/y5nonf4 4fr9LA/7BMsMp5yNsxx4PrdFvIjVpR+3ktslPwIOvPYpib/c1iAO92e/b4LtDC2O AjAKCJDk8cTj1U3tPUSZJRa661iObTj18d9JQ3SUhLBUQ9pOIjVRrV2YHbDLgi8p 1kxMpSQ4NaDjfi97rky6CeSYc1FcMd2pO57O0YsioXMU2A7Gg922QNPysqWYXJFz OCXzEZwUTuCeiTNuN33D/D00fq/yE1io+lgrNDCZs7CyJ43um03WJgEkLek45oaO OtZhPrR5Ecr5leCGLLAusMM8Uc5bYyTANw+u+H392SlTUZcCQ0Rzi+VYMRtsUNiH HK2qVyCYSLGo9SpC3RHhESShDtFytMVlqm6HfReOINDhm2Hb3rQyjbu26l3xvrjC M5lSbboUrciPafUetBXmWkveP5ObOUlHRdXW+FvqdzQdgVJneRXP0oVVD54KxK0B aIGCFAz58pD5y8zTx+REPQUw9TFu3dAP+kDdpP/kHBDUVtC7kIpis3a+STiioHzi aVNjVuQMAZpucIRxfDMCEPLubKogpLok6iwbXoyuM6asjwsbOhs87nSTfHJ4lt7Q ArqyvtaSkpSGfexKLW88sb3lJb7BJG+Ekr47P4FjfI1SuHRqFcwiRCfrVSR6fp1k I8XzlN+1w7+HAQ2+9jo83xyUkQsg3JqrJ3KXYkJwctfD6FaTuAA= =L5hP -----END PGP SIGNATURE----- --qjjjsycu6227j47j--