Date: Thu, 22 Sep 2022 02:55:31 -0400 From: Paul Procacci <pprocacci@gmail.com> To: Kevin Oberman <rkoberman@gmail.com> Cc: "freebsd-questions@freebsd.org" <questions@freebsd.org> Subject: Re: Disabling CPUs Message-ID: <CAFbbPuivdu-YVzoU1Y=dQ3imsV1Oj1PdEG8hgLurdDaaR_QPrA@mail.gmail.com> In-Reply-To: <CAN6yY1sqwFQ1EkaV5PBNU7uOMuZoi90=FBvLjVaOuXpmwM00ow@mail.gmail.com> References: <CAN6yY1s=ds_=yWsEWtVfoXchD5XXqU5uPw6bdFNW%2BdaU6t491A@mail.gmail.com> <CAFbbPuijZ9YmRkS=0JdHNNj34Zv8Qoh78nbnP2JbHCyotD-fiA@mail.gmail.com> <CAN6yY1sqwFQ1EkaV5PBNU7uOMuZoi90=FBvLjVaOuXpmwM00ow@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000ccc43d05e93e8c91
Content-Type: text/plain; charset="UTF-8"
On Thu, Sep 22, 2022 at 1:15 AM Kevin Oberman <rkoberman@gmail.com> wrote:
> On Wed, Sep 21, 2022 at 7:26 PM Paul Procacci <pprocacci@gmail.com> wrote:
>
>> (Resending with list included)
>>
>> Hey Keith,
>>
>> Taken from smp(4):
>>
>> FreeBSD allows specific CPUs on a multi-processor system to be
>> disabled.
>> This can be done using the hint.lapic.X.disabled tunable, where X is
>> the
>> APIC ID of a CPU. Setting this tunable to 1 will result in the
>> corresponding CPU being disabled.
>>
>> smp(4) further describes how to detect CPU topologies and whatnot.
>> Specifically the following sysctl seems useful: kern.sched.topology_spec
>>
>> Though I'm not entirely sure, I'd personally start with the above.
>>
>> ~Paul
>>
>> On Wed, Sep 21, 2022 at 10:06 PM Kevin Oberman <rkoberman@gmail.com>
>> wrote:
>>
>>> I am looking to disable all 8 E-cores on my Alder Lake system to prevent
>>> repeated crashes. The man page has an example of this:
>>> Modify the cpuset all threads are in by default to contain only the
>>> first 4 CPUs, leaving the rest idle:
>>> cpuset -l 0-3 -s 1
>>> I did this, but in subsequent port build, all 12 "CPUs" were running at
>>> 100%. Am I missing something? Maybe use -p 1" instead of "-s 1".
>>>
>>> I also found suggestions to use "hint.lapic.N.disabled", but teh lines
>>> that were supposed to be in dmesg and the messages log were not present. I
>>> am baffled, but really need to do something to stop the crashes currently
>>> impacting Alder Lake systems.
>>> --
>>> Kevin Oberman, Part time kid herder and retired Network Engineer
>>> E-mail: rkoberman@gmail.com
>>> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
>>>
>>
>>
>> --
>> __________________
>>
>> :(){ :|:& };:
>>
>
>
>
> Nothing looks useful in kern.sched. I am wondering if SCHED_4BSD is
> impacting this. My knowledge of the schedulers is minimal, but 4BSD
> provided much better responsiveness than ULE.I 've been running 4BSD on my
> systems (excluding servers) for over a decade... Probably much longer. I
> may try building a kernel with ULE and see what effect that has.
>
> # sysctl kern.sched
> kern.sched.runq_fuzz: 1
> kern.sched.ipiwakeup.useloop: 0
> kern.sched.ipiwakeup.usemask: 1
> kern.sched.ipiwakeup.delivered: 26954287
> kern.sched.ipiwakeup.requested: 26899799
> kern.sched.ipiwakeup.enabled: 1
> kern.sched.slice: 12
> kern.sched.quantum: 94488
> kern.sched.name: 4BSD
> kern.sched.preemption: 1
> kern.sched.cpusetsize: 32
> -
> Kevin Oberman, Part time kid herder and retired Network Engineer
> E-mail: rkoberman@gmail.com
> PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683
>
Really wished you mentioned that part about 4BSD from the get-go. ;)
I switched to ULE shortly after it was released in ~05' and haven't looked
back.
The topology_spec sysctl is a ULE scheduler thing only from what I can tell.
Sorry!
~Paul
--
__________________
:(){ :|:& };:
--000000000000ccc43d05e93e8c91
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div><div><div><div><div dir=3D"ltr"><br></div><br><div cl=
ass=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Sep 22, 2=
022 at 1:15 AM Kevin Oberman <<a href=3D"mailto:rkoberman@gmail.com">rko=
berman@gmail.com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" =
style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);pa=
dding-left:1ex"><div dir=3D"ltr"><div dir=3D"ltr"><div style=3D"font-family=
:tahoma,sans-serif;font-size:small">On Wed, Sep 21, 2022 at 7:26 PM Paul Pr=
ocacci <<a href=3D"mailto:pprocacci@gmail.com" target=3D"_blank">pprocac=
ci@gmail.com</a>> wrote:<br></div></div><div class=3D"gmail_quote"><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:=
1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr">
<div>(Resending with list included)<br><br>Hey Keith,</div><div><br></div><=
div>Taken from smp(4):<br><br></div><div>=C2=A0 =C2=A0=C2=A0 FreeBSD allows=
specific CPUs on a multi-processor system to be disabled.<br>=C2=A0 =C2=A0=
=C2=A0This can be done using the hint.lapic.X.disabled tunable, where X is=
the<br>=C2=A0 =C2=A0 =C2=A0APIC ID of a CPU.=C2=A0 Setting this tunable to=
1 will result in the<br>=C2=A0 =C2=A0 =C2=A0corresponding CPU being disabl=
ed.<br><br></div><div>smp(4) further describes how to detect CPU topologies=
and whatnot.<br></div><div>Specifically the following sysctl seems useful:=
kern.sched.topology_spec<br></div><div><br></div><div>Though I'm not e=
ntirely sure, I'd personally start with the above.<font color=3D"#88888=
8"><br><br></font></div><font color=3D"#888888"><div>~Paul</div></font>
</div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">=
On Wed, Sep 21, 2022 at 10:06 PM Kevin Oberman <<a href=3D"mailto:rkober=
man@gmail.com" target=3D"_blank">rkoberman@gmail.com</a>> wrote:<br></di=
v><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;borde=
r-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div s=
tyle=3D"font-family:tahoma,sans-serif;font-size:small">I am looking to disa=
ble all 8 E-cores on my Alder Lake system to prevent repeated crashes. The =
man page has an example of this:</div><div style=3D"font-family:tahoma,sans=
-serif;font-size:small">Modify the cpuset all threads are in by default to =
contain only the first 4 CPUs, leaving the rest idle:<br>=C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0cpuset -l 0-3 -s 1</div><div style=3D"font-family:t=
ahoma,sans-serif;font-size:small">I did this, but in subsequent port build,=
all 12 "CPUs" were running at 100%. Am I missing something? Mayb=
e use -p 1" instead of "-s 1".</div><div style=3D"font-famil=
y:tahoma,sans-serif;font-size:small"><br></div><div style=3D"font-family:ta=
homa,sans-serif;font-size:small">I also found suggestions to use "hint=
.lapic.N.disabled", but teh lines that were supposed to be in dmesg an=
d the messages log were not present. I am baffled, but really need to do so=
mething to stop the crashes currently impacting Alder Lake systems.<br></di=
v>-- <br><div dir=3D"ltr"><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div =
dir=3D"ltr"><div><div dir=3D"ltr">Kevin Oberman, Part time kid herder and r=
etired Network Engineer<br>E-mail: <a href=3D"mailto:rkoberman@gmail.com" t=
arget=3D"_blank">rkoberman@gmail.com</a><br></div><div>PGP Fingerprint: D03=
FB98AFA78E3B78C1694B318AB39EF1B055683</div></div></div></div></div></div></=
div></div></div>
</blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr">__________=
________<br><br>:(){ :|:& };:</div>
</blockquote></div><br clear=3D"all"><br><div><br></div><div><span class=3D=
"gmail_default" style=3D"font-family:tahoma,sans-serif;font-size:small">Not=
hing looks useful in kern.sched. I am wondering if SCHED_4BSD is impacting =
this. My knowledge of the schedulers is minimal, but 4BSD provided much bet=
ter responsiveness than ULE.I 've been running 4BSD on my systems (excl=
uding servers) for over a decade... Probably much longer. I may try buildin=
g a kernel with ULE and see what effect that has.<br></span></div><div><spa=
n class=3D"gmail_default" style=3D"font-family:tahoma,sans-serif;font-size:=
small"><br></span></div><div><span class=3D"gmail_default" style=3D"font-fa=
mily:tahoma,sans-serif;font-size:small"></span># sysctl kern.sched</div>ker=
n.sched.runq_fuzz: 1<br>kern.sched.ipiwakeup.useloop: 0<br>kern.sched.ipiwa=
keup.usemask: 1<br>kern.sched.ipiwakeup.delivered: 26954287<br>kern.sched.i=
piwakeup.requested: 26899799<br>kern.sched.ipiwakeup.enabled: 1<br>kern.sch=
ed.slice: 12<br>kern.sched.quantum: 94488<br><a href=3D"http://kern.sched.n=
ame" target=3D"_blank">kern.sched.name</a>: 4BSD<br>kern.sched.preemption: =
1<br>kern.sched.cpusetsize: 32<br>- <br><div dir=3D"ltr"><div dir=3D"ltr"><=
div><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=3D"ltr">Kevin Ober=
man, Part time kid herder and retired Network Engineer<br>E-mail: <a href=
=3D"mailto:rkoberman@gmail.com" target=3D"_blank">rkoberman@gmail.com</a><b=
r></div><div>PGP Fingerprint: D03FB98AFA78E3B78C1694B318AB39EF1B055683</div=
></div></div></div></div></div></div></div></div>
</blockquote></div><br clear=3D"all"><br></div>Really wished you mentioned =
that part about 4BSD from the get-go.=C2=A0 ;)<br></div>I switched to ULE s=
hortly after it was released in ~05' and haven't looked back.<br></=
div>The topology_spec sysctl is a ULE scheduler thing only from what I can =
tell.<br><br></div><div>Sorry!<br><br></div>~Paul<br><div><div><div><div>--=
<br><div dir=3D"ltr" class=3D"gmail_signature">__________________<br><br>:=
(){ :|:& };:</div></div></div></div></div></div>
--000000000000ccc43d05e93e8c91--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFbbPuivdu-YVzoU1Y=dQ3imsV1Oj1PdEG8hgLurdDaaR_QPrA>