Date: Thu, 15 Jan 2026 19:21:20 +0000 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Minsoo Choo <minsoochoo0122@proton.me> Cc: freebsd-hackers <freebsd-hackers@freebsd.org> Subject: Re: HMP scheduling on FreeBSD Message-ID: <5a2sxxiqkafxdolq362ivmpyfemi5wc75zknnuzqdhstezumrq@2p7o2dmf7r6n> In-Reply-To: <0Ng09S3rEB0BvT9vzHqVKU7rWxoad96kjEc7U2LCwDFJKmmswXujip7qbRlo_BIhNKcI7d-2CUHdp9Dxr3-7hhafpD6uagJSFUCjtC9qRr4=@proton.me>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
On Wed, Jan 14, 2026 at 04:10:15PM +0000, Minsoo Choo wrote:
> Greetings,
>
> For the last few days, I've been working on scheduler optimization for heterogeneous cores ("HMP scheduling" from now on) on FreeBSD. After days of reading specs, planning structure, and writing code, I came to a model that can be utilized for implementing HMP scheduling. I opened a first revision on Phabricator (D54674) a few days ago but decided to introduce this model in the mailing list and hear others' opinions.
>
> First of all, HMP-related code (scheduling, provider, cpucap framework, etc) are only built and enabled when "options HMP" is specified in kernel configuration. Of course, this shouldn't be activated by default because it's in experiment status. "options HMP" should be added to kernel configuration to enable this featured and this won't be enabled by default until the majority of developers believe that HMP scheduling is stabilized.
>
> The first component of HMP scheduling is "cpucap". One issue with HMP scheduling is that identifying the capacity and scores of a processor (i.e. providers) is machine-dependent while the scheduler code should be machine-independent, so cpucap acts as an interface between the scheduler and providers. CPU capacity and scores are stored in pcpu structure while the machine's cpucap status (e.g. initialized, has dynamic scores, etc) is stored in global cpucap structure of type "cpucap_t". It also includes functions for scheduler and providers, such as accessors, setters, finding "best" cpu, etc. The review (D54674) adds these facilities under HMP option.
>
> What are capacity and scores? Capacity describes cpu's heterogeneity (more information in sys/contrib/device-tree/Bindings/cpu/cpu-capacity.txt). This is static; it is initialized on boot time (e.g. from device tree) and stored in pc_cap_capacity in pcpu after being normalized to 0-1024. If it is not provided, the default value of 1024 will be used. It is static and core specific; if a performance core has capacity of 1024 and a efficiency core has capacity of 600, this means all performance cores have 1024 and all efficiency cores have 600, and whether they are throttled or not, this information stays the same. For this nature, capacity is hint for loading balancing. By dividing a core's capacity by total capacity, we can assign an equal fraction of tasks to the core's run queue. These cpucap information, including enablement status, has dynamic score, individual capacity and scores, can be queried using sysctl.
>
> On the other hand, scores reflect the real time status of a processor and are normalized to 0-1024. Providers like Intel Thread Director give the current status of each core and feed it to pc_cap_scores in pcpu. Scores are used for thread selection. For example, if a performance core is experiencing throttling, its score could go down to 1000. In that case, the scheduler will prefer core that has the highest score. Scores are completely optional because many processors do not provide this, and when score information is absent, cpucap fall backs to capacity. Since scores are dynamic, they are retrieved and set using atomic operations.
>
> Providers feed capacity or score information to cpucap. Capacity providers such as device trees and ACPI CPPC run on boot time and feed capacity information. If there is no capacity provider, (again) the default value is used. Score providers such as Intel ITD are implemented as device drivers. We had ITD implementation patch (D44453-D44459) back in May 2024 called coredirector, but it was neglected. With some modification, this can be the first cpucap provider. Providers cannot be loaded or unloaded on runtime. An exception is Arm SCMI which doesn't feed information to cpucap, but instead, the scheduler should control and request CPU's performance.
>
> Before integrating scheduler and cpucap, I need to go through sched_ule.c from top to bottom. After that, I'll add new functions or drop existing ones from the cpucap framework then work on the integration.
>
> If anyone is interested in this implementation, please reply to this thread and/or review my cpucap patch at D54674.
Hey Minsoo,
Thank you very much for working on this! I've merged the patch locally
on one of my systems running HardenedBSD. I can confirm compilation
succeeds (and boots).
I'll be paying attention to this thread for updates. I'm happy to test
future patch revisions, too, so please reach out even if you want me
to test a one-line change. I subscribed to patch D54674, so I should
see updates there.
One question I have regarding the security posture of heterogeneous
cores is whether constant-time cryptographic/string/math/etc
operations take between cores. Would it be possible for an attacker to
do <something> by virtue of the different types of cores?
For example, when installing FreeBSD with a geli-backed root
filesystem, geli determines how quickly cryptographic operations take
and will adjust the number of rounds accordingly. If that operation
takes place on an efficiency core, would that result in fewer
encryption rounds with geli?
Can some sort of timing oracle exist by virtue of the different cores?
How are speculative execution vulnerabilities manifest?
I'm not expecting an answer to any of these questions. I'm just
writing down some thoughts. But, if anyone can answer those questions,
all the better. :-)
Thanks,
--
Shawn Webb
Cofounder / Security Engineer
HardenedBSD
Signal Username: shawn_webb.74
Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50
https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmlpPikACgkQ/y5nonf4
4fr9LA/7BMsMp5yNsxx4PrdFvIjVpR+3ktslPwIOvPYpib/c1iAO92e/b4LtDC2O
AjAKCJDk8cTj1U3tPUSZJRa661iObTj18d9JQ3SUhLBUQ9pOIjVRrV2YHbDLgi8p
1kxMpSQ4NaDjfi97rky6CeSYc1FcMd2pO57O0YsioXMU2A7Gg922QNPysqWYXJFz
OCXzEZwUTuCeiTNuN33D/D00fq/yE1io+lgrNDCZs7CyJ43um03WJgEkLek45oaO
OtZhPrR5Ecr5leCGLLAusMM8Uc5bYyTANw+u+H392SlTUZcCQ0Rzi+VYMRtsUNiH
HK2qVyCYSLGo9SpC3RHhESShDtFytMVlqm6HfReOINDhm2Hb3rQyjbu26l3xvrjC
M5lSbboUrciPafUetBXmWkveP5ObOUlHRdXW+FvqdzQdgVJneRXP0oVVD54KxK0B
aIGCFAz58pD5y8zTx+REPQUw9TFu3dAP+kDdpP/kHBDUVtC7kIpis3a+STiioHzi
aVNjVuQMAZpucIRxfDMCEPLubKogpLok6iwbXoyuM6asjwsbOhs87nSTfHJ4lt7Q
ArqyvtaSkpSGfexKLW88sb3lJb7BJG+Ekr47P4FjfI1SuHRqFcwiRCfrVSR6fp1k
I8XzlN+1w7+HAQ2+9jo83xyUkQsg3JqrJ3KXYkJwctfD6FaTuAA=
=L5hP
-----END PGP SIGNATURE-----
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5a2sxxiqkafxdolq362ivmpyfemi5wc75zknnuzqdhstezumrq>