Date: Tue, 18 Apr 2023 17:09:03 -0600 From: Warner Losh <imp@bsdimp.com> To: Mark Millard <marklmi@yahoo.com> Cc: Current FreeBSD <freebsd-current@freebsd.org>, freebsd-arm <freebsd-arm@freebsd.org>, Mateusz Guzik <mjg@freebsd.org>, Pawel Jakub Dawidek <pjd@freebsd.org>, Kyle Evans <kevans@freebsd.org> Subject: Re: The import of openzfs vs. armv7: boot crashs Message-ID: <CANCZdfoBXwrX2OXPmDUJvttsh9XqaVtYAQ5tqqPj=oMgc7JjLg@mail.gmail.com> In-Reply-To: <E00FC372-EC41-4008-8769-830660062D0F@yahoo.com> References: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F.ref@yahoo.com> <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> <CANCZdfrrWget9PY8bDbUuPTvtVsvuyWzn9qX_NdCqfxFdrLo4g@mail.gmail.com> <E00FC372-EC41-4008-8769-830660062D0F@yahoo.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--00000000000068890105f9a465db Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Apr 18, 2023 at 5:04=E2=80=AFPM Mark Millard <marklmi@yahoo.com> wr= ote: > On Apr 18, 2023, at 15:46, Warner Losh <imp@bsdimp.com> wrote: > > > Fun... > > > > I'm also fighting aarch64 issues... > > Of what kind? I've been able to use things as committed > in FreeBSD (block_cloning never having been enabled but > jumping from before the import to, effectively, after > the FreeBSD adjustments). But I have not tried anything > that is different as committed in openzfs. > > (I'm one of those that tested poudriere bulk activity > via separate media from my normal aarch64 context. Those > tests had no problems once the full set up adjustments > was present in my context.) > All boot loader issues for special environments. Not in the kernel, so maybe unrelated and I shouldn't have said anything. I'm guessing that upstream needs a generic way to disable all acceleration, even if that has bad performance. I'm looking for a good way to do that (once I get done with the bugs I was fixing when I noticed this issue). Warner > > Warner > > > > On Tue, Apr 18, 2023, 4:45 PM Mark Millard <marklmi@yahoo.com> wrote: > > > https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c741= 3aae9 > > > > does not seem to cover armv7, just aarch64. (FreeBSD disabled > > floating point for both armv7 and aarch64 but that is a > > different change than above.) > > I probably should have explicitly noted that the fpu disabling > was from after the snapshot being tested here. > > The point of the snapshot test (the most recent available) was > to find out if armv7 crashed before the fpu-use disabling commit. > > > I used: > > > > > FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img= .xz > > That is from after the import and after: > > =E2=80=A2 git: eb1feadc201a - main - zfs: fix null ap->a_fsizetd NULL= pointer > derefernce Martin Matuska > > but with no other zfs changes. It does not contain: > > =E2=80=A2 git: d6e24901349d - main - zfs: disable kernel fpu usage on= arm and > aarc64 Mateusz Guzik > > (But the openzfs changes are different.) > > > booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the > > "undefined floating point instruction" notice): > > > > # zpool import > > ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow pani= c! > > Please consider adding 'options KSTACK_PAGES=3D4' to your kernel config > > panic: undefined floating point instruction in supervisor mode > > cpuid =3D 2 > > time =3D 1680784610 > > KDB: stack backtrace: > > db_trace_self() at db_trace_self > > pc =3D 0xc05eb154 lr =3D 0xc007a688 (db_trace_self_wrapper+0x= 30) > > sp =3D 0xdd25c480 fp =3D 0xdd25c598 > > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > > pc =3D 0xc007a688 lr =3D 0xc02eb1b4 (vpanic+0x140) > > sp =3D 0xdd25c5a0 fp =3D 0xdd25c5c0 > > r4 =3D 0x00000100 r5 =3D 0x00000000 > > r6 =3D 0xc0736bfc r7 =3D 0xc0b1aea8 > > vpanic() at vpanic+0x140 > > pc =3D 0xc02eb1b4 lr =3D 0xc02eaf94 (doadump) > > sp =3D 0xdd25c5c8 fp =3D 0xdd25c5cc > > r4 =3D 0xc0b92210 r5 =3D 0x00000000 > > r6 =3D 0xc0610ca0 r7 =3D 0xf4210a0d > > r8 =3D 0xddf32e4c r9 =3D 0x00000013 > > r10 =3D 0xdd25c6c0 > > doadump() at doadump > > pc =3D 0xc02eaf94 lr =3D 0xc0610eb0 (vfp_new_thread) > > sp =3D 0xdd25c5d4 fp =3D 0xdd25c638 > > r4 =3D 0xdd25c6c0 r5 =3D 0xdd25c5cc > > r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4 > > vfp_new_thread() at vfp_new_thread > > pc =3D 0xc0610eb0 lr =3D 0xc060ff84 (undefinedinstruction+0x1= 78) > > sp =3D 0xdd25c640 fp =3D 0xdd25c6b8 > > undefinedinstruction() at undefinedinstruction+0x178 > > pc =3D 0xc060ff84 lr =3D 0xc05edaa8 (exception_exit) > > sp =3D 0xdd25c6c0 fp =3D 0xdd25c750 > > r4 =3D 0x20000013 r5 =3D 0xde45e000 > > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > > r8 =3D 0x00000000 r9 =3D 0x00000000 > > r10 =3D 0xdd25c8c0 > > exception_exit() at exception_exit > > pc =3D 0xc05edaa8 lr =3D 0xddf31f20 (K256) > > sp =3D 0xdd25c750 fp =3D 0xdd25c750 > > r0 =3D 0xdd25c890 r1 =3D 0xde45e000 > > r2 =3D 0xde45e400 r3 =3D 0xddf309fc > > r4 =3D 0x00000400 r5 =3D 0xde45e000 > > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > > r8 =3D 0x00000000 r9 =3D 0x00000000 > > r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0 > > zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c > > pc =3D 0xddf32e4c lr =3D 0xc0946e8c (pcpup) > > sp =3D 0xdd25c758 fp =3D 0xc0b0aeec > > r4 =3D 0xc0919610 r5 =3D 0xc0919630 > > r6 =3D 0xc0919618 r7 =3D 0x642ebce2 > > r8 =3D 0xc0b1b0ec r9 =3D 0xc0915e88 > > r10 =3D 0xc0b1b0dc > > Fatal kernel mode data abort: 'Translation Fault (L1)' on read > > trapframe: 0xdd25c330 > > FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3 > > r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555 > > r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c > > r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0 > > r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308 > > > > panic: Fatal abort > > . . . (repeats over and over) . . . > > > > > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > --00000000000068890105f9a465db Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"ltr"><div dir=3D"ltr"><br></div><br><div class=3D"gmail_quote">= <div dir=3D"ltr" class=3D"gmail_attr">On Tue, Apr 18, 2023 at 5:04=E2=80=AF= PM Mark Millard <<a href=3D"mailto:marklmi@yahoo.com">marklmi@yahoo.com<= /a>> wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0= px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">O= n Apr 18, 2023, at 15:46, Warner Losh <<a href=3D"mailto:imp@bsdimp.com"= target=3D"_blank">imp@bsdimp.com</a>> wrote:<br> <br> > Fun...<br> > <br> > I'm also fighting aarch64 issues...<br> <br> Of what kind? I've been able to use things as committed<br> in FreeBSD (block_cloning never having been enabled but<br> jumping from before the import to, effectively, after<br> the FreeBSD adjustments). But I have not tried anything<br> that is different as committed in openzfs.<br> <br> (I'm one of those that tested poudriere bulk activity<br> via separate media from my normal aarch64 context. Those<br> tests had no problems once the full set up adjustments<br> was present in my context.)<br></blockquote><div><br></div><div>All boot lo= ader issues for special environments. Not in the kernel,</div><div>so maybe= unrelated and I shouldn't have said anything.</div><div><br></div><div= >I'm guessing that upstream needs a generic way to disable all</div><di= v>acceleration, even if that has bad performance. I'm looking for a</di= v><div>good way to do that (once I get done with the bugs I was fixing</div= ><div>when I noticed this issue).</div><div><br></div><div>Warner</div><div= >=C2=A0</div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px = 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> > Warner<br> > <br> > On Tue, Apr 18, 2023, 4:45 PM Mark Millard <<a href=3D"mailto:markl= mi@yahoo.com" target=3D"_blank">marklmi@yahoo.com</a>> wrote:<br> > <a href=3D"https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e6= 79256c71e0c7413aae9" rel=3D"noreferrer" target=3D"_blank">https://github.co= m/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c7413aae9</a><br> > <br> > does not seem to cover armv7, just aarch64. (FreeBSD disabled<br> > floating point for both armv7 and aarch64 but that is a<br> > different change than above.)<br> <br> I probably should have explicitly noted that the fpu disabling<br> was from after the snapshot being tested here.<br> <br> The point of the snapshot test (the most recent available) was<br> to find out if armv7 crashed before the fpu-use disabling commit.<br> <br> > I used:<br> > <br> > FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.= img.xz<br> <br> That is from after the import and after:<br> <br> =C2=A0 =C2=A0 =E2=80=A2 git: eb1feadc201a - main - zfs: fix null ap->a_f= sizetd NULL pointer derefernce Martin Matuska<br> <br> but with no other zfs changes. It does not contain:<br> <br> =C2=A0 =C2=A0 =E2=80=A2 git: d6e24901349d - main - zfs: disable kernel fpu = usage on arm and aarc64 Mateusz Guzik<br> <br> (But the openzfs changes are different.)<br> <br> > booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the<b= r> > "undefined floating point instruction" notice):<br> > <br> > # zpool import<br> > ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow pan= ic!<br> > Please consider adding 'options KSTACK_PAGES=3D4' to your kern= el config<br> > panic: undefined floating point instruction in supervisor mode<br> > cpuid =3D 2<br> > time =3D 1680784610<br> > KDB: stack backtrace:<br> > db_trace_self() at db_trace_self<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc05eb154=C2=A0 lr =3D 0xc00= 7a688 (db_trace_self_wrapper+0x30)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c480=C2=A0 fp =3D 0xdd2= 5c598<br> > db_trace_self_wrapper() at db_trace_self_wrapper+0x30<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc007a688=C2=A0 lr =3D 0xc02= eb1b4 (vpanic+0x140)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c5a0=C2=A0 fp =3D 0xdd2= 5c5c0<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0x00000100=C2=A0 r5 =3D 0x000= 00000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc0736bfc=C2=A0 r7 =3D 0xc0b= 1aea8<br> > vpanic() at vpanic+0x140<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc02eb1b4=C2=A0 lr =3D 0xc02= eaf94 (doadump)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c5c8=C2=A0 fp =3D 0xdd2= 5c5cc<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0xc0b92210=C2=A0 r5 =3D 0x000= 00000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc0610ca0=C2=A0 r7 =3D 0xf42= 10a0d<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0xddf32e4c=C2=A0 r9 =3D 0x000= 00013<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xdd25c6c0<br> > doadump() at doadump<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc02eaf94=C2=A0 lr =3D 0xc06= 10eb0 (vfp_new_thread)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c5d4=C2=A0 fp =3D 0xdd2= 5c638<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0xdd25c6c0=C2=A0 r5 =3D 0xdd2= 5c5cc<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4= <br> > vfp_new_thread() at vfp_new_thread<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc0610eb0=C2=A0 lr =3D 0xc06= 0ff84 (undefinedinstruction+0x178)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c640=C2=A0 fp =3D 0xdd2= 5c6b8<br> > undefinedinstruction() at undefinedinstruction+0x178<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc060ff84=C2=A0 lr =3D 0xc05= edaa8 (exception_exit)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c6c0=C2=A0 fp =3D 0xdd2= 5c750<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0x20000013=C2=A0 r5 =3D 0xde4= 5e000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xdd25c890=C2=A0 r7 =3D 0xdd2= 5c8b0<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0x00000000=C2=A0 r9 =3D 0x000= 00000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xdd25c8c0<br> > exception_exit() at exception_exit<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc05edaa8=C2=A0 lr =3D 0xddf= 31f20 (K256)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c750=C2=A0 fp =3D 0xdd2= 5c750<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r0 =3D 0xdd25c890=C2=A0 r1 =3D 0xde4= 5e000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r2 =3D 0xde45e400=C2=A0 r3 =3D 0xddf= 309fc<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0x00000400=C2=A0 r5 =3D 0xde4= 5e000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xdd25c890=C2=A0 r7 =3D 0xdd2= 5c8b0<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0x00000000=C2=A0 r9 =3D 0x000= 00000<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0= <br> > zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xddf32e4c=C2=A0 lr =3D 0xc09= 46e8c (pcpup)<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c758=C2=A0 fp =3D 0xc0b= 0aeec<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0xc0919610=C2=A0 r5 =3D 0xc09= 19630<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc0919618=C2=A0 r7 =3D 0x642= ebce2<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0xc0b1b0ec=C2=A0 r9 =3D 0xc09= 15e88<br> >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xc0b1b0dc<br> > Fatal kernel mode data abort: 'Translation Fault (L1)' on read= <br> > trapframe: 0xdd25c330<br> > FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3<br> > r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555<br> > r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c<br> > r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0<br> > r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308<br> > <br> > panic: Fatal abort<br> > . . . (repeats over and over) . . .<br> > <br> <br> <br> <br> <br> =3D=3D=3D<br> Mark Millard<br> marklmi at <a href=3D"http://yahoo.com" rel=3D"noreferrer" target=3D"_blank= ">yahoo.com</a><br> <br> </blockquote></div></div> --00000000000068890105f9a465db--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfoBXwrX2OXPmDUJvttsh9XqaVtYAQ5tqqPj=oMgc7JjLg>