Date: Thu, 23 Mar 2023 08:57:45 +0100 From: Alexander Leidinger <Alexander@leidinger.net> To: Mateusz Guzik <mjguzik@gmail.com> Cc: sgk@troutmask.apl.washington.edu, Matthias Andree <mandree@freebsd.org>, freebsd-hackers@freebsd.org Subject: Re: Periodic rant about SCHED_ULE Message-ID: <20230323085745.Horde.eeJqWekvVqCybD-psTGfIRE@webmail.leidinger.net> In-Reply-To: <CAGudoHEj%2BkoaYhkjzDE5KX9OsCno=X5M_E3z9uwg6Pg7dtqTsA@mail.gmail.com> References: <a401e51a-250a-64a0-15cb-ff79bcefbf94@m5p.com> <8173cc7e-e934-dd5c-312a-1dfa886941aa@FreeBSD.org> <8cfdb951-9b1f-ecd3-2291-7a528e1b042c@m5p.com> <c3f5f667-ba0b-c40c-b8a6-19d1c9c63c5f@FreeBSD.org> <ZBtRJhNHluj5Nzyk@troutmask.apl.washington.edu> <CAGudoHEj%2BkoaYhkjzDE5KX9OsCno=X5M_E3z9uwg6Pg7dtqTsA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Quoting Mateusz Guzik <mjguzik@gmail.com> (from Wed, 22 Mar 2023
20:23:42 +0100):
> sched_4bsd *round robins* all workers across all CPUs, which comes at
> a performance *hit* compared to ule if number of workers is <= CPU
> count -- here ule maintains affinity, avoiding cache busting. If you
> slap in $cpu_count + 1 workers, 4bsd keeps the round robin equally
> penalizing everyone, while ule mostly penalizes a select victim. By
> the end of it, you get lower total cpu time spent, but higher total
> real time. See below for an example.
>
> Two massive problems with 4bsd, apart from mandatory round robin which
> also happens to help by accident:
> 1. it has one *global* lock, meaning the scheduler itself just does
> not scale and this is visible at modest contemporary scales
> 2. it does not understand topology -- no accounting done for ht nor
> numa, but i concede the latter wont be a factor for most people
>
> That said, ULE definitely has performance bugs which need to be fixed.
> At least for the case below, 4BSD just "lucked" into sucking less
> simply because it is so primitive.
IIRC ULE also has a semantic difference compared with 4BSD in terms of
handling of the "idle" (and realtime) priority. What I remember (it's
been a while since ULE was introduced) is, that with 4BSD the
idprio(1) case could result in such processes getting no CPU time,
whereas with ULE it is quaranteed that such processes will get "some"
(no idea what a more specific description of "some" would be) CPU
time. BTW: someone may want to review the BUGS part of idprio(1) if
the part about "preempted" is still correct.
I remember there was a lengthy discussion about the handing of process
priority back then, but a quick googling didn't seems to have had the
right keywords to find it. What I got was this:
Original ULE design/scientific paper:
https://web.cs.ucdavis.edu/~roper/ecs150/ULE.pdf
Laymans-terms description:
https://people.freebsd.org/~jeff/FBSD_14_10ULE_aj.jm_cx10_09.pdf
ULE 2.0:
https://lists.freebsd.org/pipermail/freebsd-current/2007-January/068404.html
Discussion about ULE/prio/sche_preempt_threads sysctl:
https://forums.freebsd.org/threads/what-is-sysctl-kern-sched-preempt_thresh.85601/page-2
Bye,
Alexander.
--
http://www.Leidinger.net Alexander@Leidinger.net: PGP 0x8F31830F9F2772BF
http://www.FreeBSD.org netchild@FreeBSD.org : PGP 0x8F31830F9F2772BF
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2
iQIzBAABCAAdFiEER9UlYXp1PSd08nWXEg2wmwP42IYFAmQcBnkACgkQEg2wmwP4
2IZO1BAAjzrfS3XZNUxEOWSBQuU8UvYUr3IYPHU4fqzXYprydAe/c05Fu85DUA+M
8iLGMDg073yTjOx+CzD9j56stBffr9UWiIuw0DOvkV8XtIRveiyZQw2DNK+ppnSZ
2C0HI0EbARipsrQd+g0eCHVc0i+9u8Ntr8EIMlzdiXu6BEtw4Pc1dzf4qyCVq7Tc
gLEg0812v3ErvEIQcP4bKlN0uWQQlSjjADXh9bk2IPRwW1a8fIWiM1hujK7qrFrL
gK6PcjbPCqUxwFUR8ogX4wkwLn8l8FtTlwjlq5ZRabuJeaJPG6cHbTyStZ3qgDMt
GjzqEmsh3mHJrE5OtBn2sonOJvMaSJqigK9Tf+1Y021EvySmkaeiytFcWKnWAyHE
NRQjJJWzrf5caQC4hTX/BhSE4TpWAuZ4J6hsdxNr3uhOKMc8gNhpOqUA6RljXLfJ
5L9k3SQvihlysK1JW+fxu75MiCYCF5YOlH43XqRULwMwQ03i47KKyDbeEkvZTsHX
siowP7jDeq0TtW2USADSGRqlhG7QIgRWgNjnGtGBg7qOVghuQNQKcKdHFEyj4cuT
o1WV+7wIMPIC1riOmKEIxWlDlp6yvURfTulcQM+6xNFkmMeoHEUI4CjxI97WDlsE
xU2b2QY09DfJFoE9Hn3KNcAUIbNLBBP28SmYd0z7r7OpOgxjHZQ=
=gkYZ
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20230323085745.Horde.eeJqWekvVqCybD-psTGfIRE>