Date: Sun, 16 Nov 2025 10:51:01 -0700 From: Warner Losh <imp@bsdimp.com> To: bob prohaska <fbsd@www.zefox.net> Cc: "Herbert J. Skuhra" <herbert@gojira.at>, "freebsd-arm@freebsd.org" <freebsd-arm@freebsd.org>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Still seeing Failed assertion: "p[i] == 0" on armv7 buildworld Message-ID: <CANCZdfqTZ311DGEQmH0FLKrh8csN-P=qwUt=RXGSTrehUfZi3g@mail.gmail.com> In-Reply-To: <aRnxOtZ8W2g6ZkVF@www.zefox.net> References: <aOu7s9roGCofdCOw@www.zefox.net> <aOvTG-20QRJtJJwf@int21h> <CANCZdfrJ8rph_rkT3Mk-sNYKNspoV15SvHWLsahzS0HnULi4ww@mail.gmail.com> <aO068RrAehdiHOoZ@www.zefox.net> <aRUJPryA4Vmu8dDD@www.zefox.net> <4957be52-e57f-4f5f-9626-d0f706480fe1@FreeBSD.org> <aRalt0YwsjV_mvMq@www.zefox.net> <87ldk9f4tt.wl-herbert@gojira.at> <CANCZdfrSaJ7snshhiV2r%2BEX_sazhJ-HFAK0e=q%2B-MOmP=uLKqg@mail.gmail.com> <aRnxOtZ8W2g6ZkVF@www.zefox.net>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
Maybe try main with the following patch. Adrian noticed the TLS mismatch. I
don't think it will matter, but TLS thread model stuff always gives me a
big headache. If the following fails to apply, just copy the
JEMALLOC_TLS_MODEL line from i386 to arm. The default changed elsewhere,
but this wasn't updated here.
Warner
diff --git
a/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h
b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h
index 34c86271ab2e..bd6850ebd95f 100644
--- a/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h
+++ b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h
@@ -26,10 +26,11 @@
#undef LG_SIZEOF_LONG
#undef LG_SIZEOF_INTMAX_T
+#define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))
+
#ifdef __i386__
# define LG_VADDR 32
# define LG_SIZEOF_PTR 2
-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))
#endif
#ifdef __ia64__
# define LG_VADDR 64
@@ -38,7 +39,6 @@
#ifdef __sparc64__
# define LG_VADDR 64
# define LG_SIZEOF_PTR 3
-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))
#endif
#ifdef __amd64__
#ifdef _USE_LG_VADDR_WIDE
@@ -47,7 +47,6 @@
# define LG_VADDR 48
#endif
# define LG_SIZEOF_PTR 3
-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))
#endif
#ifdef __arm__
# define LG_VADDR 32
@@ -69,11 +68,9 @@
#ifdef __powerpc64__
# define LG_VADDR 64
# define LG_SIZEOF_PTR 3
-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))
#elif defined(__powerpc__)
# define LG_VADDR 32
# define LG_SIZEOF_PTR 2
-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))
#endif
#ifdef __riscv
# define LG_VADDR 48
On Sun, Nov 16, 2025, 8:42 AM bob prohaska <fbsd@www.zefox.net> wrote:
> On Sat, Nov 15, 2025 at 04:08:50PM -0700, Warner Losh wrote:
> > On Fri, Nov 14, 2025 at 12:31 AM Herbert J. Skuhra <herbert@gojira.at>
> > wrote:
> >
> > > On Fri, 14 Nov 2025 04:44:55 +0100, bob prohaska wrote:
> > > >
> > > > On Thu, Nov 13, 2025 at 09:11:51AM +0100, Ronald Klop wrote:
> > > > > Op 12-11-2025 om 23:25 schreef bob prohaska:
> > > > > > For lack of any better ideas I've collected some of the assertion
> > > failure
> > > > > > /tmp files by host at
> > > > > > http://www.zefox.net/~fbsd/assertion_failure/
> > > > > >
> > > > > > Thanks for reading,
> > > > > >
> > > > > > bob prohaska
> > > > > >
> > > > > >
> > > > >
> > > > > A really uneducated guess, but might the update of jemalloc [1]
> have
> > > introduced some subtle issues on armv7?
> > > > > You can try reverting:
> > >
> https://cgit.freebsd.org/src/commit/?id=c43cad87172039ccf38172129c79755ea79e6102
> > > .
> > > > >
> > > > What is the required syntax? Trying a simple
> > > >
> > > > root@generic:/usr/src # git revert -m 1
> > > c43cad87172039ccf38172129c79755ea79e
> > > > generated a torrent of conflict reports
> > > > The source tree is expendable with no valued customizations.
> > >
> > > Try:
> > >
> > > git revert -n -m 1 8ebb3de0c9dfb1a15bf24dcb0ca65cc91e7ad0e8
> > > git revert -n -m 1 edf9a2fae94a4b0ffa11d40ce52a48a609da9353
> > > git revert -n -m 1 c43cad87172039ccf38172129c79755ea79e6102
> > >
> >
> > Do you have to do all three? I'd have thought only the first one would be
> > needed.
>
> Necessary or not, it seems to have worked. No errors were reported and
> buildworld/kernel has run successfully on two of three Pi2's. One of
> them has rebooted and run an incremental buildworld using its new kernel
> and world.
>
> Next step is to run make cleandir twice on that machine and try a
> clean-start buildworld/kernel. Those results will take a few days.
>
> In the meantime, is there any explicit test to see if the reversion
> worked as expected? Searching for null (no error) results is slow.
>
> For example, Peter Holm's stress2 suite found lots of mischief in
> years gone by, might it be applied to this sort of problem?
>
> Thanks to you and everybody else for your help!
>
> bob prohaska
>
>
[-- Attachment #2 --]
<div dir="ltr">Maybe try main with the following patch. Adrian noticed the TLS mismatch. I don't think it will matter, but TLS thread model stuff always gives me a big headache. If the following fails to apply, just copy the JEMALLOC_TLS_MODEL line from i386 to arm. The default changed elsewhere, but this wasn't updated here.<div><br></div><div>Warner<br><div><br></div><div>diff --git a/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h<br>index 34c86271ab2e..bd6850ebd95f 100644<br>--- a/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h<br>+++ b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h<br>@@ -26,10 +26,11 @@<br> #undef LG_SIZEOF_LONG<br> #undef LG_SIZEOF_INTMAX_T<br><br>+#define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))<br>+<br> #ifdef __i386__<br> # define LG_VADDR 32<br> # define LG_SIZEOF_PTR 2<br>-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))<br> #endif<br> #ifdef __ia64__<br> # define LG_VADDR 64<br>@@ -38,7 +39,6 @@<br> #ifdef __sparc64__<br> # define LG_VADDR 64<br> # define LG_SIZEOF_PTR 3<br>-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))<br> #endif<br> #ifdef __amd64__<br> #ifdef _USE_LG_VADDR_WIDE<br>@@ -47,7 +47,6 @@<br> # define LG_VADDR 48<br> #endif<br> # define LG_SIZEOF_PTR 3<br>-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))<br> #endif<br> #ifdef __arm__<br> # define LG_VADDR 32<br>@@ -69,11 +68,9 @@<br> #ifdef __powerpc64__<br> # define LG_VADDR 64<br> # define LG_SIZEOF_PTR 3<br>-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))<br> #elif defined(__powerpc__)<br> # define LG_VADDR 32<br> # define LG_SIZEOF_PTR 2<br>-# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec")))<br> #endif<br> #ifdef __riscv<br> # define LG_VADDR 48</div><div><br></div></div></div><div dir="auto"><div><br><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Nov 16, 2025, 8:42 AM bob prohaska <<a href="mailto:fbsd@www.zefox.net" target="_blank">fbsd@www.zefox.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">On Sat, Nov 15, 2025 at 04:08:50PM -0700, Warner Losh wrote:<br>
> On Fri, Nov 14, 2025 at 12:31 AM Herbert J. Skuhra <<a href="mailto:herbert@gojira.at" rel="noreferrer" target="_blank">herbert@gojira.at</a>><br>
> wrote:<br>
> <br>
> > On Fri, 14 Nov 2025 04:44:55 +0100, bob prohaska wrote:<br>
> > ><br>
> > > On Thu, Nov 13, 2025 at 09:11:51AM +0100, Ronald Klop wrote:<br>
> > > > Op 12-11-2025 om 23:25 schreef bob prohaska:<br>
> > > > > For lack of any better ideas I've collected some of the assertion<br>
> > failure<br>
> > > > > /tmp files by host at<br>
> > > > > <a href="http://www.zefox.net/~fbsd/assertion_failure/" rel="noreferrer noreferrer" target="_blank">http://www.zefox.net/~fbsd/assertion_failure/</a><br>;
> > > > ><br>
> > > > > Thanks for reading,<br>
> > > > ><br>
> > > > > bob prohaska<br>
> > > > ><br>
> > > > ><br>
> > > ><br>
> > > > A really uneducated guess, but might the update of jemalloc [1] have<br>
> > introduced some subtle issues on armv7?<br>
> > > > You can try reverting:<br>
> > <a href="https://cgit.freebsd.org/src/commit/?id=c43cad87172039ccf38172129c79755ea79e6102" rel="noreferrer noreferrer" target="_blank">https://cgit.freebsd.org/src/commit/?id=c43cad87172039ccf38172129c79755ea79e6102</a><br>;
> > .<br>
> > > ><br>
> > > What is the required syntax? Trying a simple<br>
> > ><br>
> > > root@generic:/usr/src # git revert -m 1<br>
> > c43cad87172039ccf38172129c79755ea79e<br>
> > > generated a torrent of conflict reports<br>
> > > The source tree is expendable with no valued customizations.<br>
> ><br>
> > Try:<br>
> ><br>
> > git revert -n -m 1 8ebb3de0c9dfb1a15bf24dcb0ca65cc91e7ad0e8<br>
> > git revert -n -m 1 edf9a2fae94a4b0ffa11d40ce52a48a609da9353<br>
> > git revert -n -m 1 c43cad87172039ccf38172129c79755ea79e6102<br>
> ><br>
> <br>
> Do you have to do all three? I'd have thought only the first one would be<br>
> needed.<br>
<br>
Necessary or not, it seems to have worked. No errors were reported and<br>
buildworld/kernel has run successfully on two of three Pi2's. One of<br>
them has rebooted and run an incremental buildworld using its new kernel<br>
and world.<br>
<br>
Next step is to run make cleandir twice on that machine and try a <br>
clean-start buildworld/kernel. Those results will take a few days.<br>
<br>
In the meantime, is there any explicit test to see if the reversion<br>
worked as expected? Searching for null (no error) results is slow.<br>
<br>
For example, Peter Holm's stress2 suite found lots of mischief in<br>
years gone by, might it be applied to this sort of problem?<br>
<br>
Thanks to you and everybody else for your help!<br>
<br>
bob prohaska<br>
<br>
</blockquote></div></div></div>
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfqTZ311DGEQmH0FLKrh8csN-P=qwUt=RXGSTrehUfZi3g>