From nobody Wed Jan 14 22:14:52 2026 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ds0m51qqzz6P0bg for ; Wed, 14 Jan 2026 22:15:01 +0000 (UTC) (envelope-from olce@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R12" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ds0m512mqz3J7h; Wed, 14 Jan 2026 22:15:01 +0000 (UTC) (envelope-from olce@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768428901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=v0uKXjGl2Zrtxq2GYXxsI0dKMCY2D69Ma3cZenB84xk=; b=b4fG3aEoIB0mHAq3XefaE/47rkIDkvorZYQgy6gQWsllafNbfWu57SMBkvLK/IgNWDZIem BM06nuxTnX8PvCItx/WRkDyx/S2DPZgZ0kDIQDmCQVcZTiEB5ctMmLQEuk1EsGhK6tVoUG SRceASl+n1T+x0GDSotrzDeUfvCcmiHn3YNdiIg4OUHiBhwBq2h8d1QU2zrEXuPLDOB6Kz rBiYp7BXe2TKy6HgwTfPU3N9wu/xVZ/Krg4Va5nI8t7KFJb5g/6ZJz35atFs3cXNzte+p1 /ZHz7WVlCr+qXUwV/0Zi4BtLyXysNTEGlrv/EzNJE3HBJVavcpiFjzTC+mHyBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768428901; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=v0uKXjGl2Zrtxq2GYXxsI0dKMCY2D69Ma3cZenB84xk=; b=SdVZxv2ADexmX4ZEzg6e42LSGPR0/RcQTPE+7GL7Vr4fl4F6fQ8LPBfdqWdyy4wvDMS95b s7d7hHe5siyKJvSWKJSSRi8JtByW8oxRIdK3AVFYLYeWZvrIpxNNOnxVvB7E9yNYI6OWuA VInlk4THdGFbbDTzTKAYeOdKMX/HaqhGaoFEV00R+64+M38zs6iRu0JeVQjKoPsyBi44XJ Mrzhb61kgNegxQ5k3YvaPn6nbRWxGWa3v21fCkrsMkVz/1dqgJzLAoM1zf1pOc7Me6Z0e3 SMnOrXLbmo+HzHPWq0CMwsFNOALOlAyBY8uihM1u56iPVjUSWq3veyK+qLDDFw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1768428901; a=rsa-sha256; cv=none; b=W70lADf9JlGz5eLxY95gjdmaY/WgA8r7OpY/x+vs+S7HNWvPbZCg7GNFgxL+Wl74msayi7 oZkSWBNIGI1qMxXai6sd9eaxm5wU7Bq9pqV2kWIYLc2Go0Ff68TvuCBKZxLHZ6T5lekIm2 zrcmjXdoO3qqlzdDv7xBqKnPMW0qiff5wBjCiYuvTvt3uhRpEr+p9Mrwc7gMTpmC25StXL P6Nd0yM9FX0hPizWYIe+e7V+jVHw3YMV8u9x4Pk+hiuOjBmt/LPyTzsL5xELkm+xTsM/Sd l81zNqbZKzyXOfXXQlEp62Fgv1Inbgh4nQtOjC+Wz8I4+zF2JMx+W2vGRIKEVg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from ravel.localnet (aclermont-ferrand-653-1-222-123.w90-14.abo.wanadoo.fr [90.14.66.123]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: olce/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4ds0m457MQz17xL; Wed, 14 Jan 2026 22:15:00 +0000 (UTC) (envelope-from olce@freebsd.org) From: Olivier Certner To: Minsoo Choo Cc: freebsd-hackers Subject: Re: HMP scheduling on FreeBSD Date: Wed, 14 Jan 2026 23:14:52 +0100 Message-ID: <1886427.OVFmXjEfDW@ravel> In-Reply-To: <0Ng09S3rEB0BvT9vzHqVKU7rWxoad96kjEc7U2LCwDFJKmmswXujip7qbRlo_BIhNKcI7d-2CUHdp9Dxr3-7hhafpD6uagJSFUCjtC9qRr4=@proton.me> References: <0Ng09S3rEB0BvT9vzHqVKU7rWxoad96kjEc7U2LCwDFJKmmswXujip7qbRlo_BIhNKcI7d-2CUHdp9Dxr3-7hhafpD6uagJSFUCjtC9qRr4=@proton.me> List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart7040536.qJWK8QVVMX"; micalg="pgp-sha384"; protocol="application/pgp-signature" --nextPart7040536.qJWK8QVVMX Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8"; protected-headers="v1" From: Olivier Certner To: Minsoo Choo Cc: freebsd-hackers Subject: Re: HMP scheduling on FreeBSD Date: Wed, 14 Jan 2026 23:14:52 +0100 Message-ID: <1886427.OVFmXjEfDW@ravel> MIME-Version: 1.0 Hi Minsoo, > For the last few days, I've been working on scheduler optimization for he= terogeneous cores ("HMP scheduling" from now on) on FreeBSD. That's great! I've also been working on it, albeit in a slow fashion and m= ostly in the background, rather focusing on scheduler design and integratio= n on our cpusets. Giving quickly some first comments. > The first component of HMP scheduling is "cpucap". One issue with HMP sch= eduling is that identifying the capacity and scores of a processor (i.e. pr= oviders) is machine-dependent while the scheduler code should be machine-in= dependent, so cpucap acts as an interface between the scheduler and provide= rs. CPU capacity and scores are stored in pcpu structure while the machine'= s cpucap status (e.g. initialized, has dynamic scores, etc) is stored in gl= obal cpucap structure of type "cpucap_t". It also includes functions for sc= heduler and providers, such as accessors, setters, finding "best" cpu, etc.= The review (D54674) adds these facilities under HMP option. I'll review D54674, but not in the immediate. Hopefully next week. =20 > By dividing a core's capacity by total capacity, we can assign an equal f= raction of tasks to the core's run queue. > > On the other hand, scores reflect the real time status of a processor (sn= ip). For example, if a performance core is experiencing throttling, its sco= re could go down to 1000. In that case, the scheduler will prefer core that= has the highest score. These are good first observations but they can only really apply in specifi= c circumstances. Converting core's capacity in run queue length can only d= rive a loaded system, not a mostly idle one. This mechanism will also caus= e an increase in latency for threads running on performant cores. There are several theoretical considerations that should be met *together*,= such as fairness, latency, bias to performance or to energy (policy), affi= nity, cpusets (directives), etc., and... > Before integrating scheduler and cpucap, I need to go through sched_ule.c= =E2=80=8B from top to bottom. After that, I'll add new functions or drop ex= isting ones from the cpucap framework then work on the integration. =2E..there are some practical considerations too. ULE maintains per-CPU ru= n queues and does inter-CPU thread exchange relatively infrequently (throug= h the so-called "long-term long balancer") for fairness. It will not excha= nge two threads on two different cores if there are the only ones running, = which again is unfair if the two cores have different performances. A general scheduler must cater to a variety of workloads, and it can be qui= te difficult to improve some characteristics without degrading others. We = certainly don't want to rush things. I invite you to read the https://wiki.freebsd.org/Scheduler/Hybrid for a gl= impse on some of the trade-offs involved and a wider perspective, which how= ever is by no means complete and for which input from you and any other int= erested parties is welcome. Thanks and regards. =2D-=20 Olivier Certner --nextPart7040536.qJWK8QVVMX Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part. Content-Transfer-Encoding: 7Bit -----BEGIN PGP SIGNATURE----- iQIzBAABCQAdFiEEmNCxHjkosai0LYIujKEwQJceJicFAmloFV0ACgkQjKEwQJce JidsGBAAtVD3WCKqZrowiBZng4d4lgjPXKy+2gh5O9if/fNP4PZG2kGDLACXO1WE yGMdM6kR5ykCgHfMaFTmS3P9ekeo0154aHiboueOnreEcJKcE1g4FbhKgyrshlKt NUUrhancmCSAOpWtvrc/zR2x/vqaQdV/9OgM3mJyLdGIvup4j6hr9EswcgLntrit kMiJgvbKkEGu7HwU0QeFpAnI+wvbP/VdW/4e2e/OqGUVyXvI93gOdjvcuRMnJz/E fHWJ1ca11WXjClZiRlSj1ZJGbvs9omFflJF4iiyw+o577+xAQgwyd04Jt4wVXRaP 4sPn3V3z1VWfnzmtp+wlg/YyDQ3U0FiU5qtG4MyRoq7+FdXKavyN+58qqP0hP0f0 8wBCK3YLssAW/Y8tRFWPAw6boynToUibf6XSdCaktVgLrYdvaHNYpNBqV9U8w/pb Vgp1EZxvM5ypIP8l7BlasGVe9nJ6Ona+oK3dF1oWBuAM2eUza4Udn/MdDNx3W9xI JTyXfQ/OjlMoUk3Xoqs9UFDjHxtpD2DifovsylvJGiWmEKHBgdXE61Dnoa1Mbp7q Azq0ndB00vkL7mvS98frEw+eYh2dLU/mYzN031/l07uKPFqpICh3XgmURDrQ1c0u oBp24yUmZYuNqF+d2O5IGvZTQf5F0HQ7kpFfFNR9IxKGrScEXj8= =NqLo -----END PGP SIGNATURE----- --nextPart7040536.qJWK8QVVMX--