Date: Thu, 23 Mar 2023 08:57:45 +0100 From: Alexander Leidinger <Alexander@leidinger.net> To: Mateusz Guzik <mjguzik@gmail.com> Cc: sgk@troutmask.apl.washington.edu, Matthias Andree <mandree@freebsd.org>, freebsd-hackers@freebsd.org Subject: Re: Periodic rant about SCHED_ULE Message-ID: <20230323085745.Horde.eeJqWekvVqCybD-psTGfIRE@webmail.leidinger.net> In-Reply-To: <CAGudoHEj%2BkoaYhkjzDE5KX9OsCno=X5M_E3z9uwg6Pg7dtqTsA@mail.gmail.com> References: <a401e51a-250a-64a0-15cb-ff79bcefbf94@m5p.com> <8173cc7e-e934-dd5c-312a-1dfa886941aa@FreeBSD.org> <8cfdb951-9b1f-ecd3-2291-7a528e1b042c@m5p.com> <c3f5f667-ba0b-c40c-b8a6-19d1c9c63c5f@FreeBSD.org> <ZBtRJhNHluj5Nzyk@troutmask.apl.washington.edu> <CAGudoHEj%2BkoaYhkjzDE5KX9OsCno=X5M_E3z9uwg6Pg7dtqTsA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This message is in MIME format and has been PGP signed.
--=_xdph0Tnv8P9I0E3lje7qzJ1
Content-Type: text/plain; charset=utf-8; format=flowed; DelSp=Yes
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Quoting Mateusz Guzik <mjguzik@gmail.com> (from Wed, 22 Mar 2023=20=20
20:23:42=20+0100):
> sched_4bsd *round robins* all workers across all CPUs, which comes at
> a performance *hit* compared to ule if number of workers is <=3D CPU
> count -- here ule maintains affinity, avoiding cache busting. If you
> slap in $cpu_count + 1 workers, 4bsd keeps the round robin equally
> penalizing everyone, while ule mostly penalizes a select victim. By
> the end of it, you get lower total cpu time spent, but higher total
> real time. See below for an example.
>
> Two massive problems with 4bsd, apart from mandatory round robin which
> also happens to help by accident:
> 1. it has one *global* lock, meaning the scheduler itself just does
> not scale and this is visible at modest contemporary scales
> 2. it does not understand topology -- no accounting done for ht nor
> numa, but i concede the latter wont be a factor for most people
>
> That said, ULE definitely has performance bugs which need to be fixed.
> At least for the case below, 4BSD just "lucked" into sucking less
> simply because it is so primitive.
IIRC ULE also has a semantic difference compared with 4BSD in terms of=20=
=20
handling=20of the "idle" (and realtime) priority. What I remember (it's=20=
=20
been=20a while since ULE was introduced) is, that with 4BSD the=20=20
idprio(1)=20case could result in such processes getting no CPU time,=20=20
whereas=20with ULE it is quaranteed that such processes will get "some"=20=
=20
(no=20idea what a more specific description of "some" would be) CPU=20=20
time.=20BTW: someone may want to review the BUGS part of idprio(1) if=20=20
the=20part about "preempted" is still correct.
I remember there was a lengthy discussion about the handing of process=20=
=20
priority=20back then, but a quick googling didn't seems to have had the=20=
=20
right=20keywords to find it. What I got was this:
Original ULE design/scientific paper:
https://web.cs.ucdavis.edu/~roper/ecs150/ULE.pdf
Laymans-terms description:
https://people.freebsd.org/~jeff/FBSD_14_10ULE_aj.jm_cx10_09.pdf
ULE 2.0:
=20=20=20=20=20=20
https://lists.freebsd.org/pipermail/freebsd-current/2007-January/068404.htm=
l
Discussion=20about ULE/prio/sche_preempt_threads sysctl:
=20=20=20=20=20=20
https://forums.freebsd.org/threads/what-is-sysctl-kern-sched-preempt_thresh=
.85601/page-2
Bye,
Alexander.
--=20
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF
--=_xdph0Tnv8P9I0E3lje7qzJ1
Content-Type: application/pgp-signature
Content-Description: Digitale PGP-Signatur
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIzBAABCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmQcBnkACgkQEg2wmwP4
2IZO1BAAjzrfS3XZNUxEOWSBQuU8UvYUr3IYPHU4fqzXYprydAe/c05Fu85DUA+M
8iLGMDg073yTjOx+CzD9j56stBffr9UWiIuw0DOvkV8XtIRveiyZQw2DNK+ppnSZ
2C0HI0EbARipsrQd+g0eCHVc0i+9u8Ntr8EIMlzdiXu6BEtw4Pc1dzf4qyCVq7Tc
gLEg0812v3ErvEIQcP4bKlN0uWQQlSjjADXh9bk2IPRwW1a8fIWiM1hujK7qrFrL
gK6PcjbPCqUxwFUR8ogX4wkwLn8l8FtTlwjlq5ZRabuJeaJPG6cHbTyStZ3qgDMt
GjzqEmsh3mHJrE5OtBn2sonOJvMaSJqigK9Tf+1Y021EvySmkaeiytFcWKnWAyHE
NRQjJJWzrf5caQC4hTX/BhSE4TpWAuZ4J6hsdxNr3uhOKMc8gNhpOqUA6RljXLfJ
5L9k3SQvihlysK1JW+fxu75MiCYCF5YOlH43XqRULwMwQ03i47KKyDbeEkvZTsHX
siowP7jDeq0TtW2USADSGRqlhG7QIgRWgNjnGtGBg7qOVghuQNQKcKdHFEyj4cuT
o1WV+7wIMPIC1riOmKEIxWlDlp6yvURfTulcQM+6xNFkmMeoHEUI4CjxI97WDlsE
xU2b2QY09DfJFoE9Hn3KNcAUIbNLBBP28SmYd0z7r7OpOgxjHZQ=
=gkYZ
-----END PGP SIGNATURE-----
--=_xdph0Tnv8P9I0E3lje7qzJ1--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20230323085745.Horde.eeJqWekvVqCybD-psTGfIRE>