Date: Thu, 6 Mar 2025 06:49:03 -0800 From: Warner Losh <imp@bsdimp.com> To: Mateusz Guzik <mjguzik@gmail.com> Cc: Zhenlei Huang <zlei@freebsd.org>, Mateusz Guzik <mjg@freebsd.org>, src-committers <src-committers@freebsd.org>, "<dev-commits-src-all@freebsd.org>" <dev-commits-src-all@freebsd.org>, Warner Losh <imp@freebsd.org>, "<dev-commits-src-main@freebsd.org>" <dev-commits-src-main@freebsd.org> Subject: Re: git: 234683726708 - main - devclass: make devclass_alloc_unit use M_NOWAIT Message-ID: <CANCZdfqmgOmpAO4UFNV4RdTC6MrjRb58kvPdj0BmwYRA7hqYXA@mail.gmail.com> In-Reply-To: <CAGudoHF=eRaHCcjRrvd4sG4-OBu0GrmVRpiHEeU1ayG=M9oXrg@mail.gmail.com> References: <202503061103.526B32Id022652@gitrepo.freebsd.org> <F1B3652E-0D0C-402A-8509-D510992DAC15@FreeBSD.org> <CAGudoHF=eRaHCcjRrvd4sG4-OBu0GrmVRpiHEeU1ayG=M9oXrg@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Thu, Mar 6, 2025, 3:35 AM Mateusz Guzik <mjguzik@gmail.com> wrote: > On Thu, Mar 6, 2025 at 12:32 PM Zhenlei Huang <zlei@freebsd.org> wrote: > > > > > > > > On Mar 6, 2025, at 7:03 PM, Mateusz Guzik <mjg@FreeBSD.org> wrote: > > > > The branch main has been updated by mjg: > > > > URL: > https://cgit.FreeBSD.org/src/commit/?id=234683726708cf5212d672d676d30056d4133859 > > > > commit 234683726708cf5212d672d676d30056d4133859 > > Author: Mateusz Guzik <mjg@FreeBSD.org> > > AuthorDate: 2025-03-06 11:01:49 +0000 > > Commit: Mateusz Guzik <mjg@FreeBSD.org> > > CommitDate: 2025-03-06 11:01:49 +0000 > > > > devclass: make devclass_alloc_unit use M_NOWAIT > > > > The only caller already does this. > > > > The routine can be called with a mutex held making M_WAITOK illegal. > > > > Sponsored by: Rubicon Communications, LLC ("Netgate") > > --- > > sys/kern/subr_bus.c | 8 ++++++-- > > 1 file changed, 6 insertions(+), 2 deletions(-) > > > > diff --git a/sys/kern/subr_bus.c b/sys/kern/subr_bus.c > > index 9506e471705c..0422352bba51 100644 > > --- a/sys/kern/subr_bus.c > > +++ b/sys/kern/subr_bus.c > > @@ -1208,6 +1208,7 @@ devclass_get_sysctl_tree(devclass_t dc) > > static int > > devclass_alloc_unit(devclass_t dc, device_t dev, int *unitp) > > { > > + device_t *devices; > > const char *s; > > int unit = *unitp; > > > > @@ -1264,8 +1265,11 @@ devclass_alloc_unit(devclass_t dc, device_t dev, > int *unitp) > > int newsize; > > > > newsize = unit + 1; > > - dc->devices = reallocf(dc->devices, > > - newsize * sizeof(*dc->devices), M_BUS, M_WAITOK); > > + devices = reallocf(dc->devices, > > + newsize * sizeof(*dc->devices), M_BUS, M_NOWAIT); > > > > > > I'd recommend against this. From the commit message of f3d3c63442ff, > Warner said, > > > In addition, transition to M_WAITOK since this is a sleepable context > > So, the M_WAITOK is intentional. > > > > Rather than reverting this, the caller devclass_add_device() should use > M_WAITOK. > > > > Per my commit message this is callable from a *NOT* sleepable context. > > Here is a splat we got at Netgate: > > uma_zalloc_debug: zone "malloc-16" with the following non-sleepable locks > held: > exclusive sleep mutex SD slot mtx (sdhci) r = 0 (0xd8dec028) locked @ > > /var/jenkins/workspace/pfSense-Plus-snapshots-25_03-main/sources/FreeBSD-src-plus-RELENG_25_03/sys/dev/sdhci/sdhci.c:688 > stack backtrace: > #0 0xc0330ebc at witness_debugger+0x78 > #1 0xc033217c at witness_warn+0x428 > #2 0xc05b0a58 at uma_zalloc_debug+0x34 > #3 0xc05b067c at uma_zalloc_arg+0x30 > #4 0xc0291760 at malloc+0x8c > #5 0xc02920ec at reallocf+0x14 > #6 0xc02f8894 at devclass_add_device+0x1e8 > #7 0xc02f6c78 at make_device+0xe0 > #8 0xc02f6abc at device_add_child_ordered+0x30 > #9 0xc0156e0c at sdhci_card_task+0x238 > #10 0xc0324090 at taskqueue_run_locked+0x1b4 > #11 0xc0323ea0 at taskqueue_run+0x50 > #12 0xc0275f88 at ithread_loop+0x264 > #13 0xc0271f28 at fork_exit+0xa0 > #14 0xc05f82d4 at swi_exit+0 > > It may be some callers are sleepable. Perhaps a different variant > accepting flags would be prudent, but I have no interest in looking > into that. > This is a big in sdhci_card_task. Newbus in general isn't callable from a sleepable context. Warner > ``` > > - dev->nameunit = malloc(buflen, M_BUS, M_NOWAIT|M_ZERO); > > - if (!dev->nameunit) > > - return (ENOMEM); > > + dev->nameunit = malloc(buflen, M_BUS, M_WAITOK | M_ZERO); > > ``` > > > > Best regards, > > Zhenlei > > > > + if (devices == NULL) > > + return (ENOMEM); > > + dc->devices = devices; > > memset(dc->devices + dc->maxunit, 0, > > sizeof(device_t) * (newsize - dc->maxunit)); > > dc->maxunit = newsize; > > > > > > > > > > > -- > Mateusz Guzik <mjguzik gmail.com> > [-- Attachment #2 --] <div dir="auto"><div><br><br><div class="gmail_quote gmail_quote_container"><div dir="ltr" class="gmail_attr">On Thu, Mar 6, 2025, 3:35 AM Mateusz Guzik <<a href="mailto:mjguzik@gmail.com">mjguzik@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">On Thu, Mar 6, 2025 at 12:32 PM Zhenlei Huang <<a href="mailto:zlei@freebsd.org" target="_blank" rel="noreferrer">zlei@freebsd.org</a>> wrote:<br> ><br> ><br> ><br> > On Mar 6, 2025, at 7:03 PM, Mateusz Guzik <mjg@FreeBSD.org> wrote:<br> ><br> > The branch main has been updated by mjg:<br> ><br> > URL: <a href="https://cgit.FreeBSD.org/src/commit/?id=234683726708cf5212d672d676d30056d4133859" rel="noreferrer noreferrer" target="_blank">https://cgit.FreeBSD.org/src/commit/?id=234683726708cf5212d672d676d30056d4133859</a><br>; ><br> > commit 234683726708cf5212d672d676d30056d4133859<br> > Author: Mateusz Guzik <mjg@FreeBSD.org><br> > AuthorDate: 2025-03-06 11:01:49 +0000<br> > Commit: Mateusz Guzik <mjg@FreeBSD.org><br> > CommitDate: 2025-03-06 11:01:49 +0000<br> ><br> > devclass: make devclass_alloc_unit use M_NOWAIT<br> ><br> > The only caller already does this.<br> ><br> > The routine can be called with a mutex held making M_WAITOK illegal.<br> ><br> > Sponsored by: Rubicon Communications, LLC ("Netgate")<br> > ---<br> > sys/kern/subr_bus.c | 8 ++++++--<br> > 1 file changed, 6 insertions(+), 2 deletions(-)<br> ><br> > diff --git a/sys/kern/subr_bus.c b/sys/kern/subr_bus.c<br> > index 9506e471705c..0422352bba51 100644<br> > --- a/sys/kern/subr_bus.c<br> > +++ b/sys/kern/subr_bus.c<br> > @@ -1208,6 +1208,7 @@ devclass_get_sysctl_tree(devclass_t dc)<br> > static int<br> > devclass_alloc_unit(devclass_t dc, device_t dev, int *unitp)<br> > {<br> > + device_t *devices;<br> > const char *s;<br> > int unit = *unitp;<br> ><br> > @@ -1264,8 +1265,11 @@ devclass_alloc_unit(devclass_t dc, device_t dev, int *unitp)<br> > int newsize;<br> ><br> > newsize = unit + 1;<br> > - dc->devices = reallocf(dc->devices,<br> > - newsize * sizeof(*dc->devices), M_BUS, M_WAITOK);<br> > + devices = reallocf(dc->devices,<br> > + newsize * sizeof(*dc->devices), M_BUS, M_NOWAIT);<br> ><br> ><br> > I'd recommend against this. From the commit message of f3d3c63442ff, Warner said,<br> > > In addition, transition to M_WAITOK since this is a sleepable context<br> > So, the M_WAITOK is intentional.<br> ><br> > Rather than reverting this, the caller devclass_add_device() should use M_WAITOK.<br> ><br> <br> Per my commit message this is callable from a *NOT* sleepable context.<br> <br> Here is a splat we got at Netgate:<br> <br> uma_zalloc_debug: zone "malloc-16" with the following non-sleepable locks held:<br> exclusive sleep mutex SD slot mtx (sdhci) r = 0 (0xd8dec028) locked @<br> /var/jenkins/workspace/pfSense-Plus-snapshots-25_03-main/sources/FreeBSD-src-plus-RELENG_25_03/sys/dev/sdhci/sdhci.c:688<br> stack backtrace:<br> #0 0xc0330ebc at witness_debugger+0x78<br> #1 0xc033217c at witness_warn+0x428<br> #2 0xc05b0a58 at uma_zalloc_debug+0x34<br> #3 0xc05b067c at uma_zalloc_arg+0x30<br> #4 0xc0291760 at malloc+0x8c<br> #5 0xc02920ec at reallocf+0x14<br> #6 0xc02f8894 at devclass_add_device+0x1e8<br> #7 0xc02f6c78 at make_device+0xe0<br> #8 0xc02f6abc at device_add_child_ordered+0x30<br> #9 0xc0156e0c at sdhci_card_task+0x238<br> #10 0xc0324090 at taskqueue_run_locked+0x1b4<br> #11 0xc0323ea0 at taskqueue_run+0x50<br> #12 0xc0275f88 at ithread_loop+0x264<br> #13 0xc0271f28 at fork_exit+0xa0<br> #14 0xc05f82d4 at swi_exit+0<br> <br> It may be some callers are sleepable. Perhaps a different variant<br> accepting flags would be prudent, but I have no interest in looking<br> into that.<br></blockquote></div></div><div dir="auto"><br></div><div dir="auto">This is a big in sdhci_card_task. Newbus in general isn't callable from a sleepable context.</div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto">Warner</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_quote gmail_quote_container"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> > ```<br> > - dev->nameunit = malloc(buflen, M_BUS, M_NOWAIT|M_ZERO);<br> > - if (!dev->nameunit)<br> > - return (ENOMEM);<br> > + dev->nameunit = malloc(buflen, M_BUS, M_WAITOK | M_ZERO);<br> > ```<br> ><br> > Best regards,<br> > Zhenlei<br> ><br> > + if (devices == NULL)<br> > + return (ENOMEM);<br> > + dc->devices = devices;<br> > memset(dc->devices + dc->maxunit, 0,<br> > sizeof(device_t) * (newsize - dc->maxunit));<br> > dc->maxunit = newsize;<br> ><br> ><br> ><br> ><br> <br> <br> -- <br> Mateusz Guzik <mjguzik <a href="http://gmail.com" rel="noreferrer noreferrer" target="_blank">gmail.com</a>><br> </blockquote></div></div></div>help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqmgOmpAO4UFNV4RdTC6MrjRb58kvPdj0BmwYRA7hqYXA>