Date: Thu, 6 Mar 2025 06:49:03 -0800 From: Warner Losh <imp@bsdimp.com> To: Mateusz Guzik <mjguzik@gmail.com> Cc: Zhenlei Huang <zlei@freebsd.org>, Mateusz Guzik <mjg@freebsd.org>, src-committers <src-committers@freebsd.org>, "<dev-commits-src-all@freebsd.org>" <dev-commits-src-all@freebsd.org>, Warner Losh <imp@freebsd.org>, "<dev-commits-src-main@freebsd.org>" <dev-commits-src-main@freebsd.org> Subject: Re: git: 234683726708 - main - devclass: make devclass_alloc_unit use M_NOWAIT Message-ID: <CANCZdfqmgOmpAO4UFNV4RdTC6MrjRb58kvPdj0BmwYRA7hqYXA@mail.gmail.com> In-Reply-To: <CAGudoHF=eRaHCcjRrvd4sG4-OBu0GrmVRpiHEeU1ayG=M9oXrg@mail.gmail.com> References: <202503061103.526B32Id022652@gitrepo.freebsd.org> <F1B3652E-0D0C-402A-8509-D510992DAC15@FreeBSD.org> <CAGudoHF=eRaHCcjRrvd4sG4-OBu0GrmVRpiHEeU1ayG=M9oXrg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000014a6a1062fad9ca8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Thu, Mar 6, 2025, 3:35=E2=80=AFAM Mateusz Guzik <mjguzik@gmail.com> wrot= e: > On Thu, Mar 6, 2025 at 12:32=E2=80=AFPM Zhenlei Huang <zlei@freebsd.org> = wrote: > > > > > > > > On Mar 6, 2025, at 7:03 PM, Mateusz Guzik <mjg@FreeBSD.org> wrote: > > > > The branch main has been updated by mjg: > > > > URL: > https://cgit.FreeBSD.org/src/commit/?id=3D234683726708cf5212d672d676d3005= 6d4133859 > > > > commit 234683726708cf5212d672d676d30056d4133859 > > Author: Mateusz Guzik <mjg@FreeBSD.org> > > AuthorDate: 2025-03-06 11:01:49 +0000 > > Commit: Mateusz Guzik <mjg@FreeBSD.org> > > CommitDate: 2025-03-06 11:01:49 +0000 > > > > devclass: make devclass_alloc_unit use M_NOWAIT > > > > The only caller already does this. > > > > The routine can be called with a mutex held making M_WAITOK illegal. > > > > Sponsored by: Rubicon Communications, LLC ("Netgate") > > --- > > sys/kern/subr_bus.c | 8 ++++++-- > > 1 file changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/sys/kern/subr_bus.c b/sys/kern/subr_bus.c > > index 9506e471705c..0422352bba51 100644 > > --- a/sys/kern/subr_bus.c > > +++ b/sys/kern/subr_bus.c > > @@ -1208,6 +1208,7 @@ devclass_get_sysctl_tree(devclass_t dc) > > static int > > devclass_alloc_unit(devclass_t dc, device_t dev, int *unitp) > > { > > + device_t *devices; > > const char *s; > > int unit =3D *unitp; > > > > @@ -1264,8 +1265,11 @@ devclass_alloc_unit(devclass_t dc, device_t dev, > int *unitp) > > int newsize; > > > > newsize =3D unit + 1; > > - dc->devices =3D reallocf(dc->devices, > > - newsize * sizeof(*dc->devices), M_BUS, M_WAITOK); > > + devices =3D reallocf(dc->devices, > > + newsize * sizeof(*dc->devices), M_BUS, M_NOWAIT); > > > > > > I'd recommend against this. From the commit message of f3d3c63442ff, > Warner said, > > > In addition, transition to M_WAITOK since this is a sleepable context > > So, the M_WAITOK is intentional. > > > > Rather than reverting this, the caller devclass_add_device() should use > M_WAITOK. > > > > Per my commit message this is callable from a *NOT* sleepable context. > > Here is a splat we got at Netgate: > > uma_zalloc_debug: zone "malloc-16" with the following non-sleepable locks > held: > exclusive sleep mutex SD slot mtx (sdhci) r =3D 0 (0xd8dec028) locked @ > > /var/jenkins/workspace/pfSense-Plus-snapshots-25_03-main/sources/FreeBSD-= src-plus-RELENG_25_03/sys/dev/sdhci/sdhci.c:688 > stack backtrace: > #0 0xc0330ebc at witness_debugger+0x78 > #1 0xc033217c at witness_warn+0x428 > #2 0xc05b0a58 at uma_zalloc_debug+0x34 > #3 0xc05b067c at uma_zalloc_arg+0x30 > #4 0xc0291760 at malloc+0x8c > #5 0xc02920ec at reallocf+0x14 > #6 0xc02f8894 at devclass_add_device+0x1e8 > #7 0xc02f6c78 at make_device+0xe0 > #8 0xc02f6abc at device_add_child_ordered+0x30 > #9 0xc0156e0c at sdhci_card_task+0x238 > #10 0xc0324090 at taskqueue_run_locked+0x1b4 > #11 0xc0323ea0 at taskqueue_run+0x50 > #12 0xc0275f88 at ithread_loop+0x264 > #13 0xc0271f28 at fork_exit+0xa0 > #14 0xc05f82d4 at swi_exit+0 > > It may be some callers are sleepable. Perhaps a different variant > accepting flags would be prudent, but I have no interest in looking > into that. > This is a big in sdhci_card_task. Newbus in general isn't callable from a sleepable context. Warner > ``` > > - dev->nameunit =3D malloc(buflen, M_BUS, M_NOWAIT|M_ZERO); > > - if (!dev->nameunit) > > - return (ENOMEM); > > + dev->nameunit =3D malloc(buflen, M_BUS, M_WAITOK | M_ZERO); > > ``` > > > > Best regards, > > Zhenlei > > > > + if (devices =3D=3D NULL) > > + return (ENOMEM); > > + dc->devices =3D devices; > > memset(dc->devices + dc->maxunit, 0, > > sizeof(device_t) * (newsize - dc->maxunit)); > > dc->maxunit =3D newsize; > > > > > > > > > > > -- > Mateusz Guzik <mjguzik gmail.com> > --00000000000014a6a1062fad9ca8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto"><div><br><br><div class=3D"gmail_quote gmail_quote_contai= ner"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Mar 6, 2025, 3:35=E2=80= =AFAM Mateusz Guzik <<a href=3D"mailto:mjguzik@gmail.com">mjguzik@gmail.= com</a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"marg= in:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Mar 6, 2= 025 at 12:32=E2=80=AFPM Zhenlei Huang <<a href=3D"mailto:zlei@freebsd.or= g" target=3D"_blank" rel=3D"noreferrer">zlei@freebsd.org</a>> wrote:<br> ><br> ><br> ><br> > On Mar 6, 2025, at 7:03 PM, Mateusz Guzik <mjg@FreeBSD.org> wrot= e:<br> ><br> > The branch main has been updated by mjg:<br> ><br> > URL: <a href=3D"https://cgit.FreeBSD.org/src/commit/?id=3D234683726708= cf5212d672d676d30056d4133859" rel=3D"noreferrer noreferrer" target=3D"_blan= k">https://cgit.FreeBSD.org/src/commit/?id=3D234683726708cf5212d672d676d300= 56d4133859</a><br> ><br> > commit 234683726708cf5212d672d676d30056d4133859<br> > Author:=C2=A0 =C2=A0 =C2=A0Mateusz Guzik <mjg@FreeBSD.org><br> > AuthorDate: 2025-03-06 11:01:49 +0000<br> > Commit:=C2=A0 =C2=A0 =C2=A0Mateusz Guzik <mjg@FreeBSD.org><br> > CommitDate: 2025-03-06 11:01:49 +0000<br> ><br> >=C2=A0 =C2=A0 devclass: make devclass_alloc_unit use M_NOWAIT<br> ><br> >=C2=A0 =C2=A0 The only caller already does this.<br> ><br> >=C2=A0 =C2=A0 The routine can be called with a mutex held making M_WAIT= OK illegal.<br> ><br> >=C2=A0 =C2=A0 Sponsored by:=C2=A0 =C2=A0Rubicon Communications, LLC (&q= uot;Netgate")<br> > ---<br> > sys/kern/subr_bus.c | 8 ++++++--<br> > 1 file changed, 6 insertions(+), 2 deletions(-)<br> ><br> > diff --git a/sys/kern/subr_bus.c b/sys/kern/subr_bus.c<br> > index 9506e471705c..0422352bba51 100644<br> > --- a/sys/kern/subr_bus.c<br> > +++ b/sys/kern/subr_bus.c<br> > @@ -1208,6 +1208,7 @@ devclass_get_sysctl_tree(devclass_t dc)<br> > static int<br> > devclass_alloc_unit(devclass_t dc, device_t dev, int *unitp)<br> > {<br> > + device_t *devices;<br> > const char *s;<br> > int unit =3D *unitp;<br> ><br> > @@ -1264,8 +1265,11 @@ devclass_alloc_unit(devclass_t dc, device_t dev= , int *unitp)<br> > int newsize;<br> ><br> > newsize =3D unit + 1;<br> > - dc->devices =3D reallocf(dc->devices,<br> > -=C2=A0 =C2=A0 newsize * sizeof(*dc->devices), M_BUS, M_WAITOK);<br= > > + devices =3D reallocf(dc->devices,<br> > +=C2=A0 =C2=A0 newsize * sizeof(*dc->devices), M_BUS, M_NOWAIT);<br= > ><br> ><br> > I'd recommend against this. From the commit message of f3d3c63442f= f, Warner said,<br> > > In addition, transition to M_WAITOK since this is a sleepable con= text<br> > So, the M_WAITOK is intentional.<br> ><br> > Rather than reverting this, the caller devclass_add_device() should us= e M_WAITOK.<br> ><br> <br> Per my commit message this is callable from a *NOT* sleepable context.<br> <br> Here is a splat we got at Netgate:<br> <br> uma_zalloc_debug: zone "malloc-16" with the following non-sleepab= le locks held:<br> exclusive sleep mutex SD slot mtx (sdhci) r =3D 0 (0xd8dec028) locked @<br> /var/jenkins/workspace/pfSense-Plus-snapshots-25_03-main/sources/FreeBSD-sr= c-plus-RELENG_25_03/sys/dev/sdhci/sdhci.c:688<br> stack backtrace:<br> #0 0xc0330ebc at witness_debugger+0x78<br> #1 0xc033217c at witness_warn+0x428<br> #2 0xc05b0a58 at uma_zalloc_debug+0x34<br> #3 0xc05b067c at uma_zalloc_arg+0x30<br> #4 0xc0291760 at malloc+0x8c<br> #5 0xc02920ec at reallocf+0x14<br> #6 0xc02f8894 at devclass_add_device+0x1e8<br> #7 0xc02f6c78 at make_device+0xe0<br> #8 0xc02f6abc at device_add_child_ordered+0x30<br> #9 0xc0156e0c at sdhci_card_task+0x238<br> #10 0xc0324090 at taskqueue_run_locked+0x1b4<br> #11 0xc0323ea0 at taskqueue_run+0x50<br> #12 0xc0275f88 at ithread_loop+0x264<br> #13 0xc0271f28 at fork_exit+0xa0<br> #14 0xc05f82d4 at swi_exit+0<br> <br> It may be some callers are sleepable. Perhaps a different variant<br> accepting flags would be prudent, but I have no interest in looking<br> into that.<br></blockquote></div></div><div dir=3D"auto"><br></div><div dir= =3D"auto">This is a big in sdhci_card_task. Newbus in general isn't cal= lable from a sleepable context.</div><div dir=3D"auto"><br></div><div dir= =3D"auto"><br></div><div dir=3D"auto">Warner</div><div dir=3D"auto"><br></d= iv><div dir=3D"auto"><div class=3D"gmail_quote gmail_quote_container"><bloc= kquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #cc= c solid;padding-left:1ex"> > ```<br> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->nameunit =3D malloc(buflen, M_BUS,= M_NOWAIT|M_ZERO);<br> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0if (!dev->nameunit)<br> > -=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0return (ENOMEM= );<br> > +=C2=A0 =C2=A0 =C2=A0 =C2=A0dev->nameunit =3D malloc(buflen, M_BUS,= M_WAITOK | M_ZERO);<br> > ```<br> ><br> > Best regards,<br> > Zhenlei<br> ><br> > + if (devices =3D=3D NULL)<br> > + return (ENOMEM);<br> > + dc->devices =3D devices;<br> > memset(dc->devices + dc->maxunit, 0,<br> >=C2=A0 =C2=A0 sizeof(device_t) * (newsize - dc->maxunit));<br> > dc->maxunit =3D newsize;<br> ><br> ><br> ><br> ><br> <br> <br> -- <br> Mateusz Guzik <mjguzik <a href=3D"http://gmail.com" rel=3D"noreferrer no= referrer" target=3D"_blank">gmail.com</a>><br> </blockquote></div></div></div> --00000000000014a6a1062fad9ca8--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqmgOmpAO4UFNV4RdTC6MrjRb58kvPdj0BmwYRA7hqYXA>