Date: Thu, 23 Mar 2023 15:06:14 -0700 From: Mark Millard <marklmi@yahoo.com> To: Warner Losh <imp@bsdimp.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Periodic rant about SCHED_ULE Message-ID: <907C8A16-D271-4C32-8DF2-A719CCC76FE0@yahoo.com> In-Reply-To: <CANCZdfpe9tPk=0=7R6WF1XDUr=9W-sFWnY5tKmKVbt=iKXCDGQ@mail.gmail.com> References: <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39.ref@yahoo.com> <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39@yahoo.com> <CANCZdfpe9tPk=0=7R6WF1XDUr=9W-sFWnY5tKmKVbt=iKXCDGQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mar 23, 2023, at 13:53, Warner Losh <imp@bsdimp.com> wrote: >=20 >> On Thu, Mar 23, 2023, 9:46 PM Mark Millard <marklmi@yahoo.com> wrote: >> Warner Losh <imp_at_bsdimp.com> wrote on >> Date: Wed, 22 Mar 2023 22:57:08 UTC : >>=20 >> > On Wed, Mar 22, 2023 at 1:41=E2=80=AFPM George Mitchell = <george+freebsd@m5p.com> >> > wrote: >> >=20 >> > > service dnetc start >> > > I am literally running "make buildworld" with no additional = options. >> > > >> > > >> > So what are the results for make buildworld -j $(sysctl -n hw.ncpu = )? >>=20 >>=20 >> Note: My experiments have been in this -j $(sysctl -n hw.ncpu ) >> realm. >>=20 >> > ULE scales much better, but when there's too little to do it can = make poor >> > choices. >> >=20 >> > ULE is better locked and don't fall over on high core count systems = like >> > BSD does at moderate load. >>=20 >> (I'm presuming the above is not about the specifics >> of the effectively different interpretations of the >> likes of having extra "nice 20" activity by the two >> schedulers for the examples related to the original >> "rant", other than the -jN issue.) >>=20 >> Any idea on what scale "high core count systems" need to be >> for what sort of "moderate load" to end up with signficant >> differences? >=20 > Sched_bsd is basically unusable on my 64 core 128 thread machine with = make -j 150 (nice or no). So, well beyond what I've access to, even if 32 core 64 thread would also show such issues. > With ULE I don't notice. That's not to say ule can't be better (me not = noticing is hardly scientific), but I tried sched bsd when I got the = thread ripper and found the machine too unresponsive when I was doing = large builds...=20 So the classification is based on responsiveness. Good to know. (I assume you had avoided having interactive processes end up with their kernel stacks swapped out. That is a separate sort of issue.) > But I wasn't playing video on this box... so maybe I hit a local = optimal point... No X11 or such invovled in my context. Mostly ssh over EtherNet. I'm rarely at the (video) console (no serial console). I did not see responsiveness issues in the ssh sessions during my activities with the 2 schedulers. (So I'd not explicitly commented about such at the time.) Thanks for reporting the example. > Warner >=20 >> What sort of context(s) show ULE scaling much >> better? On the 16 core ThreadRipper 1950X (32 hardware >> threads) I've really only demonstrated the "nice 20" >> distinction as significant between the schedulers so far. >> (I do not have acccess to anything with more hardware threads.) >>=20 >> Note: I've not (yet?) been looking at having just a little >> more than the number of hardware threads active (no nice >> involvement). >=20 Side note: In order to be sure to avoid interactive processes ending up with their kernel threads swapped out, I use in /etc/sysctl.conf (which prevents far more from ending up that way): # # Together this pair avoids swapping out the process kernel stacks. # This avoids processes for interacting with the system from being # hung-up by such. vm.swap_enabled=3D0 vm.swap_idle_enabled=3D0 While I have such everywhere, only some machines and a rare type of use on them is actually likely need to above in my context. After adding the above, I've never had the loss of access problem again. But nothing I've done would indicate much about X11 use reasonableness, for example. =3D=3D=3D Mark Millard marklmi at yahoo.com
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?907C8A16-D271-4C32-8DF2-A719CCC76FE0>