Date: Thu, 23 Mar 2023 21:53:25 +0100 From: Warner Losh <imp@bsdimp.com> To: Mark Millard <marklmi@yahoo.com> Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: Periodic rant about SCHED_ULE Message-ID: <CANCZdfpe9tPk=0=7R6WF1XDUr=9W-sFWnY5tKmKVbt=iKXCDGQ@mail.gmail.com> In-Reply-To: <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39@yahoo.com> References: <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39.ref@yahoo.com> <24F6D88F-3F15-48FB-AA5A-AFD4B77A1D39@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000006da3d505f79778ca Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Mar 23, 2023, 9:46 PM Mark Millard <marklmi@yahoo.com> wrote: > Warner Losh <imp_at_bsdimp.com> wrote on > Date: Wed, 22 Mar 2023 22:57:08 UTC : > > > On Wed, Mar 22, 2023 at 1:41=E2=80=AFPM George Mitchell <george+freebsd= @m5p.com> > > wrote: > > > > > service dnetc start > > > I am literally running "make buildworld" with no additional options. > > > > > > > > So what are the results for make buildworld -j $(sysctl -n hw.ncpu )? > > > Note: My experiments have been in this -j $(sysctl -n hw.ncpu ) > realm. > > > ULE scales much better, but when there's too little to do it can make > poor > > choices. > > > > ULE is better locked and don't fall over on high core count systems lik= e > > BSD does at moderate load. > > (I'm presuming the above is not about the specifics > of the effectively different interpretations of the > likes of having extra "nice 20" activity by the two > schedulers for the examples related to the original > "rant", other than the -jN issue.) > > Any idea on what scale "high core count systems" need to be > for what sort of "moderate load" to end up with signficant > differences? Sched_bsd is basically unusable on my 64 core 128 thread machine with make -j 150 (nice or no). With ULE I don't notice. That's not to say ule can't be better (me not noticing is hardly scientific), but I tried sched bsd when I got the thread ripper and found the machine too unresponsive when I was doing large builds... But I wasn't playing video on this box... so maybe I hit a local optimal point... Warner What sort of context(s) show ULE scaling much > better? On the 16 core ThreadRipper 1950X (32 hardware > threads) I've really only demonstrated the "nice 20" > distinction as significant between the schedulers so far. > (I do not have acccess to anything with more hardware threads.) > > Note: I've not (yet?) been looking at having just a little > more than the number of hardware threads active (no nice > involvement). > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > --0000000000006da3d505f79778ca Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote"><div dir=3D"ltr" = class=3D"gmail_attr">On Thu, Mar 23, 2023, 9:46 PM Mark Millard <<a href= =3D"mailto:marklmi@yahoo.com">marklmi@yahoo.com</a>> wrote:<br></div><bl= ockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #= ccc solid;padding-left:1ex">Warner Losh <<a href=3D"http://imp_at_bsdimp= .com" rel=3D"noreferrer noreferrer" target=3D"_blank">imp_at_bsdimp.com</a>= > wrote on<br> Date: Wed, 22 Mar 2023 22:57:08 UTC :<br> <br> > On Wed, Mar 22, 2023 at 1:41=E2=80=AFPM George Mitchell <<a href=3D= "mailto:george%2Bfreebsd@m5p.com" target=3D"_blank" rel=3D"noreferrer">geor= ge+freebsd@m5p.com</a>><br> > wrote:<br> > <br> > > service dnetc start<br> > > I am literally running "make buildworld" with no additi= onal options.<br> > ><br> > ><br> > So what are the results for make buildworld -j $(sysctl -n hw.ncpu )?<= br> <br> <br> Note: My experiments have been in this -j $(sysctl -n hw.ncpu )<br> realm.<br> <br> > ULE scales much better, but when there's too little to do it can m= ake poor<br> > choices.<br> > <br> > ULE is better locked and don't fall over on high core count system= s like<br> > BSD does at moderate load.<br> <br> (I'm presuming the above is not about the specifics<br> of the effectively different interpretations of the<br> likes of having extra "nice 20" activity by the two<br> schedulers for the examples related to the original<br> "rant", other than the -jN issue.)<br> <br> Any idea on what scale "high core count systems" need to be<br> for what sort of "moderate load" to end up with signficant<br> differences?</blockquote></div></div><div dir=3D"auto"><br></div><div dir= =3D"auto">Sched_bsd is basically unusable on my 64 core 128 thread machine = with make -j 150 (nice or no). With ULE I don't notice. That's not = to say ule can't be better (me not noticing is hardly scientific), but = I tried sched bsd when I got the thread ripper and found the machine too un= responsive when I was doing large builds...=C2=A0</div><div dir=3D"auto"><b= r></div><div dir=3D"auto">But I wasn't playing video on this box... so = maybe I hit a local optimal point...</div><div dir=3D"auto"><br></div><div = dir=3D"auto">Warner</div><div dir=3D"auto"><br></div><div dir=3D"auto"><div= class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 = 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> What sort of context= (s) show ULE scaling much<br> better? On the 16 core ThreadRipper 1950X (32 hardware<br> threads) I've really only demonstrated the "nice 20"<br> distinction as significant between the schedulers so far.<br> (I do not have acccess to anything with more hardware threads.)<br> <br> Note: I've not (yet?) been looking at having just a little<br> more than the number of hardware threads active (no nice<br> involvement).<br> <br> =3D=3D=3D<br> Mark Millard<br> marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer noreferrer" targe= t=3D"_blank">yahoo.com</a><br> <br> </blockquote></div></div></div> --0000000000006da3d505f79778ca--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfpe9tPk=0=7R6WF1XDUr=9W-sFWnY5tKmKVbt=iKXCDGQ>