Date: Fri, 12 Jan 2024 17:57:30 +0000 From: Doug Rabson <dfr@rabson.org> To: Mark Millard <marklmi@yahoo.com> Cc: Current FreeBSD <freebsd-current@freebsd.org>, FreeBSD-STABLE Mailing List <freebsd-stable@freebsd.org> Subject: Re: 15 & 14: ram_attach vs. its using regions_to_avail vs. "bus_alloc_resource" can lead to: panic("ram_attach: resource %d failed to attach", rid) Message-ID: <CACA0VUjwCcXTq5m=S8_Mj6pTpiyZ7=v7um=NxsR3Uj1c-_vuYQ@mail.gmail.com> In-Reply-To: <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com> References: <3CB904C2-D983-4EF7-84D3-6BDED0700B08.ref@yahoo.com> <3CB904C2-D983-4EF7-84D3-6BDED0700B08@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--0000000000006b1e9f060ec36675 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Sat, 30 Sept 2023 at 08:47, Mark Millard <marklmi@yahoo.com> wrote: > ram_attach is based on regions_to_avail but that is a problem for > its later bus_alloc_resource use --and that can lead to: > > panic("ram_attach: resource %d failed to attach", rid); > > Unfortunately, the known example is use of EDK2 on RPi4B > class systems, not what is considered the supported way. > The panic happens for main [so: 15] and will happen once > the cortex-a72 handling in 14.0-* is in a build fixed by: > > =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata workaro= unds that > depend on smccc Andrew Turner > > The lack of the fix leads to an earlier panic as stands. > > > sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring > phys_avail and using only hwregions and exregions. In other words, > in part: > > * Initially dump_avail and phys_avail are identical. Boot time memory > * allocations remove extents from phys_avail that may still be included > * in dumps. > > This means that early, dedicated memory allocations are treated > as available for general use by regions_to_avail . The distinction > is visible in the boot -v output in that: > > real memory =3D 3138154496 (2992 MB) > Physical memory chunk(s): > 0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages) > 0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages) > 0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages) > 0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages) > 0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages) > 0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages) > avail memory =3D 3027378176 (2887 MB) > > does not list the wider: > > 0x00000040000000 - 0x000000bfffffff > > because of phys_avail . But the earlier dump based on hwregions and > exregions shows: > > Physical memory chunk(s): > 0x001d0000 - 0x001effff, 0 MB ( 32 pages) > 0x00200000 - 0x338c6fff, 822 MB ( 210631 pages) > 0x33920000 - 0x3b2fffff, 121 MB ( 31200 pages) > 0x40000000 - 0xbfffffff, 2048 MB ( 524288 pages) > Excluded memory regions: > 0x001d0000 - 0x001effff, 0 MB ( 32 pages) NoAlloc > 0x2b800000 - 0x2ce39fff, 22 MB ( 5690 pages) NoAlloc > 0x33860000 - 0x338bffff, 0 MB ( 96 pages) NoAlloc > 0x33920000 - 0x33a2ffff, 1 MB ( 272 pages) NoAlloc > 0x36f00000 - 0x372dffff, 3 MB ( 992 pages) NoAlloc > > which indicates: > > 0x40000000 - 0xbfffffff > > is available as far as it is concerned. > > (Note some code works/displays in terms of: 0x40000000 - 0xc0000000 > instead.) > > For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource > that is used as bus_alloc_resource . It ends up rejecting the > RPi4B boot via using the result of the call in ram_attach: > > if (bus_alloc_resource(dev, SYS_RES_MEMORY, &rid, start, > end, > end - start, 0) =3D=3D NULL) > panic("ram_attach: resource %d failed to attach", > rid); > > as shown by the just-prior start/end pair sequence messages: > > ram0: reserving memory region: 200000-2b800000 > ram0: reserving memory region: 2ce3a000-33860000 > ram0: reserving memory region: 338c0000-338c7000 > ram0: reserving memory region: 33a30000-36f00000 > ram0: reserving memory region: 372e0000-3b300000 > ram0: reserving memory region: 40000000-c0000000 > panic: ram_attach: resource 5 failed to attach > > I do not see anything about this that looks inherently RPi* > specific for possibly ending up with an analogous panic. So > I expect the example is sufficient context to identify a > problem is present, despite EDK2 use not being normal for > RPi4B's and the like as far as FreeBSD is concerned. > I'm not quite clear why phys_avail changes and why that is triggered by the 906bcc44641d commit. I'm wondering if it makes sense to arrange for ram_attach to happen before acpi, e.g. using BUS_PASS_ORDER_FIRST? Doug. --0000000000006b1e9f060ec36675 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Sat, 30 Sept 2023 at 08:47, Mark M= illard <<a href=3D"mailto:marklmi@yahoo.com">marklmi@yahoo.com</a>> w= rote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0p= x 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb= (204,204,204);padding-left:1ex">ram_attach is based on regions_to_avail but= that is a problem for<br> its later bus_alloc_resource use --and that can lead to:<br> <br> panic("ram_attach: resource %d failed to attach", rid);<br> <br> Unfortunately, the known example is use of EDK2 on RPi4B<br> class systems, not what is considered the supported way.<br> The panic happens for main [so: 15] and will happen once<br> the cortex-a72 handling in 14.0-* is in a build fixed by:<br> <br> =C2=A0 =C2=A0 =E2=80=A2 git: 906bcc44641d - releng/14.0 - arm64: Fix errata= workarounds that depend on smccc Andrew Turner<br> <br> The lack of the fix leads to an earlier panic as stands.<br> <br> <br> sys/kern/subr_physmem.c 's regions_to_avail is based on ignoring<br> phys_avail and using only hwregions and exregions. In other words,<br> in part:<br> <br> =C2=A0* Initially dump_avail and phys_avail are identical.=C2=A0 Boot time = memory<br> =C2=A0* allocations remove extents from phys_avail that may still be includ= ed<br> =C2=A0* in dumps.<br> <br> This means that early, dedicated memory allocations are treated<br> as available for general use by regions_to_avail . The distinction<br> is visible in the=C2=A0 boot -v output in that:<br> <br> real memory=C2=A0 =3D 3138154496 (2992 MB)<br> Physical memory chunk(s):<br> 0x00000000200000 - 0x0000002b7fffff, 727711744 bytes (177664 pages)<br> 0x0000002ce3a000 - 0x0000003385ffff, 111304704 bytes (27174 pages)<br> 0x000000338c0000 - 0x000000338c6fff, 28672 bytes (7 pages)<br> 0x00000033a30000 - 0x00000036efffff, 55377920 bytes (13520 pages)<br> 0x000000372e0000 - 0x0000003b2fffff, 67239936 bytes (16416 pages)<br> 0x00000040000000 - 0x000000bb3dcfff, 2067648512 bytes (504797 pages)<br> avail memory =3D 3027378176 (2887 MB)<br> <br> does not list the wider:<br> <br> 0x00000040000000 - 0x000000bfffffff<br> <br> because of phys_avail . But the earlier dump based on hwregions and<br> exregions shows:<br> <br> Physical memory chunk(s):<br> =C2=A0 0x001d0000 - 0x001effff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2= =A032 pages)<br> =C2=A0 0x00200000 - 0x338c6fff,=C2=A0 =C2=A0822 MB ( 210631 pages)<br> =C2=A0 0x33920000 - 0x3b2fffff,=C2=A0 =C2=A0121 MB (=C2=A0 31200 pages)<br> =C2=A0 0x40000000 - 0xbfffffff,=C2=A0 2048 MB ( 524288 pages)<br> Excluded memory regions:<br> =C2=A0 0x001d0000 - 0x001effff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2= =A032 pages) NoAlloc <br> =C2=A0 0x2b800000 - 0x2ce39fff,=C2=A0 =C2=A0 22 MB (=C2=A0 =C2=A05690 pages= ) NoAlloc <br> =C2=A0 0x33860000 - 0x338bffff,=C2=A0 =C2=A0 =C2=A00 MB (=C2=A0 =C2=A0 =C2= =A096 pages) NoAlloc <br> =C2=A0 0x33920000 - 0x33a2ffff,=C2=A0 =C2=A0 =C2=A01 MB (=C2=A0 =C2=A0 272 = pages) NoAlloc <br> =C2=A0 0x36f00000 - 0x372dffff,=C2=A0 =C2=A0 =C2=A03 MB (=C2=A0 =C2=A0 992 = pages) NoAlloc <br> <br> which indicates:<br> <br> =C2=A0 0x40000000 - 0xbfffffff<br> <br> is available as far as it is concerned.<br> <br> (Note some code works/displays in terms of: 0x40000000 - 0xc0000000<br> instead.)<br> <br> For aarch64 , sys/arm64/arm64/nexus.c has a nexus_alloc_resource<br> that is used as bus_alloc_resource . It ends up rejecting the<br> RPi4B boot via using the result of the call in ram_attach:<br> <br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (bus_alloc_resou= rce(dev, SYS_RES_MEMORY, &rid, start, end,<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 end -= start, 0) =3D=3D NULL)<br> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 panic("ram_attach: resource %d failed to attach", rid)= ;<br> <br> as shown by the just-prior start/end pair sequence messages:<br> <br> ram0: reserving memory region:=C2=A0 =C2=A0200000-2b800000<br> ram0: reserving memory region:=C2=A0 =C2=A02ce3a000-33860000<br> ram0: reserving memory region:=C2=A0 =C2=A0338c0000-338c7000<br> ram0: reserving memory region:=C2=A0 =C2=A033a30000-36f00000<br> ram0: reserving memory region:=C2=A0 =C2=A0372e0000-3b300000<br> ram0: reserving memory region:=C2=A0 =C2=A040000000-c0000000<br> panic: ram_attach: resource 5 failed to attach<br> <br> I do not see anything about this that looks inherently RPi*<br> specific for possibly ending up with an analogous panic. So<br> I expect the example is sufficient context to identify a<br> problem is present, despite EDK2 use not being normal for<br> RPi4B's and the like as far as FreeBSD is concerned.<br></blockquote><d= iv><br></div><div>I'm not quite clear why phys_avail changes and why th= at is triggered by the 906bcc44641d commit. I'm wondering if it makes s= ense to arrange for ram_attach to happen before acpi, e.g. using BUS_PASS_O= RDER_FIRST?</div><div><br></div><div>Doug.</div><div>=C2=A0</div></div></di= v> --0000000000006b1e9f060ec36675--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CACA0VUjwCcXTq5m=S8_Mj6pTpiyZ7=v7um=NxsR3Uj1c-_vuYQ>