From nobody Sun Nov 16 17:51:01 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d8dj44D7sz6Gpw5 for ; Sun, 16 Nov 2025 17:51:20 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-pj1-x1036.google.com (mail-pj1-x1036.google.com [IPv6:2607:f8b0:4864:20::1036]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4d8dj36DJdz3v69 for ; Sun, 16 Nov 2025 17:51:19 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-343684a06b2so3114265a91.1 for ; Sun, 16 Nov 2025 09:51:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20230601.gappssmtp.com; s=20230601; t=1763315473; x=1763920273; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=T7JYXy+RBg4lEmb5zG0L1qqSL3AEzof1js0rO4fVvr0=; b=C3MM0Sc1HHNgebGbi11m5GddHjIzs6NcS7Rty2Q8izPQ5Xh2LHGhjefJ79x7DewmV6 Rq+9x4gIt49LhGMeBdkRh+TqWL9qK+wMrCXQeW/dtdSUK4B4X2QoyAH/JNBCyZoZc+6R c57sqKdXeIKjWDZCdNLRHqLiz+I8wRweugf8JxoyjnJWNQqvCAKLs4bSSvpfCdNEnGil puM5KG77Wgg3+pge1mtJ2ott/KuzWWXwF1eygnNjdx0LtM0V2lkD7zfI0VGwhuMCuNpC QMUQQvGR09Ro94LArQ5X2LyexnLV2F/70g28vwb2O5w0VOGGEj69eoMYL5PBqEz13B9e Z9cw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1763315473; x=1763920273; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-gg:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=T7JYXy+RBg4lEmb5zG0L1qqSL3AEzof1js0rO4fVvr0=; b=g8YxZM8KJROqHoMph+wJJ6lQfI4QeYdYSta+jq6cyAc0sN2k5QRisjrXdNljp8pRlp 6T03fI9YKlyIzRrYyHSHfHg9VXJvMd3LUh2BnL9btLA0dMiA6Gw0zDDNKWV2MBnDQVcj F6eil3AQ5wI5qmOPLp4iEtm4JbYt0i5NAGxTysEnNr6J21pz05HOXhcGIJbS9iET5hnh t9pXfOOHPD89tZu7ibiZOVxTBSKDxP4s/S7PUbzjfFs8uO4hF4kaC5P0NUS5TBEF/tmB gpoTrTYDnc3D79xBdPYW7mukhXEIbLEkDJ7amZlOMGz9x1x1jxLVrRmPSQm3yf7l0HR6 +cBg== X-Forwarded-Encrypted: i=1; AJvYcCVVWma2HzfdVDOUhX+aQFMMpc18klY64jX2UvVNBokt8mmLsBlOJQD/j3sxGJRBXbN0VavN3qXQKCSzwOupjHE=@freebsd.org X-Gm-Message-State: AOJu0YzijRR9gJyM0GgMFYhsSkHT4++xbFaUqcX9ADmOC8wLWx91qLd+ pmY/gxQj0vXKJookIYENnfU2J8vj1oX0p43kCLfQd6xKHBbB6eItmI7yOukHltx6YDSTlAeWCIw C6kT4yg9bqDYGElAai7LqgV4LwtJ//2dOZbztsox/NQ== X-Gm-Gg: ASbGncualoXczk/gaGyyXmxByvVVQ+FO23JJD7i85+9Hs6QerYqmACEudIopya3bqMj uTmIwMRkIKbks1drgtRO4rg+WP2Vh4u8Xd5dYOdJNTVwX61t4vUrNXI15Hn3SKfodYlIK9Gq0Tl h6UyoczuxhYeH7Fpisxb96ZJafsBAtaUjttL6yuJTS31IdZX1Nq0iBIMXfCfN+4U/t5cYNuyw6S 3dP4EUJa26OUWFwSa5XaWzQRVzuTFYZtNaECMdXLTP6XpLYn39TOTra3yH2E1ONqrWxi4U= X-Google-Smtp-Source: AGHT+IEFc3Z1kK9Yef1ns68w80tv0nL5flpDN/+gY5qTy8A82FyR4rMmbDqfATsZrVkbU8NBZS+UkEoQeC3S1hweTZ0= X-Received: by 2002:a17:90b:5544:b0:340:99fd:9676 with SMTP id 98e67ed59e1d1-343f9eaf809mr9530543a91.10.1763315472978; Sun, 16 Nov 2025 09:51:12 -0800 (PST) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 References: <4957be52-e57f-4f5f-9626-d0f706480fe1@FreeBSD.org> <87ldk9f4tt.wl-herbert@gojira.at> In-Reply-To: From: Warner Losh Date: Sun, 16 Nov 2025 10:51:01 -0700 X-Gm-Features: AWmQ_bn7aVurLZjYow5OkxhG0K-YNzqo1Dq3aznur4jywbtnkIcql6_EbIpWfdY Message-ID: Subject: Re: Still seeing Failed assertion: "p[i] == 0" on armv7 buildworld To: bob prohaska Cc: "Herbert J. Skuhra" , "freebsd-arm@freebsd.org" , FreeBSD Current Content-Type: multipart/alternative; boundary="00000000000064e2d50643b9e032" X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4d8dj36DJdz3v69 --00000000000064e2d50643b9e032 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Maybe try main with the following patch. Adrian noticed the TLS mismatch. I don't think it will matter, but TLS thread model stuff always gives me a big headache. If the following fails to apply, just copy the JEMALLOC_TLS_MODEL line from i386 to arm. The default changed elsewhere, but this wasn't updated here. Warner diff --git a/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h index 34c86271ab2e..bd6850ebd95f 100644 --- a/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h +++ b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h @@ -26,10 +26,11 @@ #undef LG_SIZEOF_LONG #undef LG_SIZEOF_INTMAX_T +#define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec"))) + #ifdef __i386__ # define LG_VADDR 32 # define LG_SIZEOF_PTR 2 -# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec"))) #endif #ifdef __ia64__ # define LG_VADDR 64 @@ -38,7 +39,6 @@ #ifdef __sparc64__ # define LG_VADDR 64 # define LG_SIZEOF_PTR 3 -# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec"))) #endif #ifdef __amd64__ #ifdef _USE_LG_VADDR_WIDE @@ -47,7 +47,6 @@ # define LG_VADDR 48 #endif # define LG_SIZEOF_PTR 3 -# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec"))) #endif #ifdef __arm__ # define LG_VADDR 32 @@ -69,11 +68,9 @@ #ifdef __powerpc64__ # define LG_VADDR 64 # define LG_SIZEOF_PTR 3 -# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec"))) #elif defined(__powerpc__) # define LG_VADDR 32 # define LG_SIZEOF_PTR 2 -# define JEMALLOC_TLS_MODEL __attribute__((tls_model("initial-exec"))) #endif #ifdef __riscv # define LG_VADDR 48 On Sun, Nov 16, 2025, 8:42=E2=80=AFAM bob prohaska wro= te: > On Sat, Nov 15, 2025 at 04:08:50PM -0700, Warner Losh wrote: > > On Fri, Nov 14, 2025 at 12:31=E2=80=AFAM Herbert J. Skuhra > > wrote: > > > > > On Fri, 14 Nov 2025 04:44:55 +0100, bob prohaska wrote: > > > > > > > > On Thu, Nov 13, 2025 at 09:11:51AM +0100, Ronald Klop wrote: > > > > > Op 12-11-2025 om 23:25 schreef bob prohaska: > > > > > > For lack of any better ideas I've collected some of the asserti= on > > > failure > > > > > > /tmp files by host at > > > > > > http://www.zefox.net/~fbsd/assertion_failure/ > > > > > > > > > > > > Thanks for reading, > > > > > > > > > > > > bob prohaska > > > > > > > > > > > > > > > > > > > > > > A really uneducated guess, but might the update of jemalloc [1] > have > > > introduced some subtle issues on armv7? > > > > > You can try reverting: > > > > https://cgit.freebsd.org/src/commit/?id=3Dc43cad87172039ccf38172129c79755= ea79e6102 > > > . > > > > > > > > > What is the required syntax? Trying a simple > > > > > > > > root@generic:/usr/src # git revert -m 1 > > > c43cad87172039ccf38172129c79755ea79e > > > > generated a torrent of conflict reports > > > > The source tree is expendable with no valued customizations. > > > > > > Try: > > > > > > git revert -n -m 1 8ebb3de0c9dfb1a15bf24dcb0ca65cc91e7ad0e8 > > > git revert -n -m 1 edf9a2fae94a4b0ffa11d40ce52a48a609da9353 > > > git revert -n -m 1 c43cad87172039ccf38172129c79755ea79e6102 > > > > > > > Do you have to do all three? I'd have thought only the first one would = be > > needed. > > Necessary or not, it seems to have worked. No errors were reported and > buildworld/kernel has run successfully on two of three Pi2's. One of > them has rebooted and run an incremental buildworld using its new kernel > and world. > > Next step is to run make cleandir twice on that machine and try a > clean-start buildworld/kernel. Those results will take a few days. > > In the meantime, is there any explicit test to see if the reversion > worked as expected? Searching for null (no error) results is slow. > > For example, Peter Holm's stress2 suite found lots of mischief in > years gone by, might it be applied to this sort of problem? > > Thanks to you and everybody else for your help! > > bob prohaska > > --00000000000064e2d50643b9e032 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Maybe try main with the following patch. Adrian noticed th= e TLS mismatch. I don't think it will matter, but TLS thread model stuf= f always gives me a big headache. If the following=C2=A0fails to apply, jus= t copy the JEMALLOC_TLS_MODEL line from i386 to arm. The default changed el= sewhere, but this wasn't updated here.

Warner

diff --git a/lib/libc/stdlib/malloc/jemalloc/include/jemal= loc/jemalloc_FreeBSD.h b/lib/libc/stdlib/malloc/jemalloc/include/jemalloc/j= emalloc_FreeBSD.h
index 34c86271ab2e..bd6850ebd95f 100644
--- a/lib/l= ibc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h
+++ b/lib= /libc/stdlib/malloc/jemalloc/include/jemalloc/jemalloc_FreeBSD.h
@@ -26,= 10 +26,11 @@
=C2=A0#undef LG_SIZEOF_LONG
=C2=A0#undef LG_SIZEOF_INTMA= X_T

+#define JEMALLOC_TLS_MODEL =C2=A0 =C2=A0 __attribute__((tls_mod= el("initial-exec")))
+
=C2=A0#ifdef __i386__
=C2=A0# =C2= =A0define LG_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 32
=C2=A0# = =C2=A0define LG_SIZEOF_PTR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A02
-# =C2=A0define JEMALLOC_TLS_MODEL =C2=A0 __attribute__((tls_mo= del("initial-exec")))
=C2=A0#endif
=C2=A0#ifdef __ia64__=C2=A0# =C2=A0define LG_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 64=
@@ -38,7 +39,6 @@
=C2=A0#ifdef __sparc64__
=C2=A0# =C2=A0define L= G_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 64
=C2=A0# =C2=A0defin= e LG_SIZEOF_PTR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03-# =C2=A0define JEMALLOC_TLS_MODEL =C2=A0 __attribute__((tls_model("i= nitial-exec")))
=C2=A0#endif
=C2=A0#ifdef __amd64__
=C2=A0#if= def _USE_LG_VADDR_WIDE
@@ -47,7 +47,6 @@
=C2=A0# =C2=A0define LG_VADD= R =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 48
=C2=A0#endif
=C2=A0# = =C2=A0define LG_SIZEOF_PTR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A03
-# =C2=A0define JEMALLOC_TLS_MODEL =C2=A0 __attribute__((tls_mo= del("initial-exec")))
=C2=A0#endif
=C2=A0#ifdef __arm__
= =C2=A0# =C2=A0define LG_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 32<= br>@@ -69,11 +68,9 @@
=C2=A0#ifdef __powerpc64__
=C2=A0# =C2=A0define= LG_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 64
=C2=A0# =C2=A0def= ine LG_SIZEOF_PTR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A03<= br>-# =C2=A0define JEMALLOC_TLS_MODEL =C2=A0 __attribute__((tls_model("= ;initial-exec")))
=C2=A0#elif defined(__powerpc__)
=C2=A0# =C2= =A0define LG_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 32
=C2=A0# = =C2=A0define LG_SIZEOF_PTR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0= =C2=A02
-# =C2=A0define JEMALLOC_TLS_MODEL =C2=A0 __attribute__((tls_mo= del("initial-exec")))
=C2=A0#endif
=C2=A0#ifdef __riscv
= =C2=A0# =C2=A0define LG_VADDR =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 48<= /div>



On Sun, Nov 16, 2025= , 8:42=E2=80=AFAM bob prohaska <fbsd@www.zefox.net> wrote:
On Sat, Nov 15, 2025 at 04:08:50PM -0700,= Warner Losh wrote:
> On Fri, Nov 14, 2025 at 12:31=E2=80=AFAM Herbert J. Skuhra <herbert@= gojira.at>
> wrote:
>
> > On Fri, 14 Nov 2025 04:44:55 +0100, bob prohaska wrote:
> > >
> > > On Thu, Nov 13, 2025 at 09:11:51AM +0100, Ronald Klop wrote:=
> > > > Op 12-11-2025 om 23:25 schreef bob prohaska:
> > > > > For lack of any better ideas I've collected so= me of the assertion
> > failure
> > > > > /tmp files by host at
> > > > > http://www.zefox.ne= t/~fbsd/assertion_failure/
> > > > >
> > > > > Thanks for reading,
> > > > >
> > > > > bob prohaska
> > > > >
> > > > >
> > > >
> > > > A really uneducated guess, but might the update of jema= lloc [1] have
> > introduced some subtle issues on armv7?
> > > > You can try reverting:
> > https://cgit.freebsd.org/src/commit/?id=3Dc43cad87172039ccf38172129c7975= 5ea79e6102
> > .
> > > >
> > > What is the required syntax? Trying a simple
> > >
> > > root@generic:/usr/src # git revert -m 1
> > c43cad87172039ccf38172129c79755ea79e
> > > generated a torrent of conflict reports
> > > The source tree is expendable with no valued customizations.=
> >
> > Try:
> >
> > git revert -n -m 1 8ebb3de0c9dfb1a15bf24dcb0ca65cc91e7ad0e8
> > git revert -n -m 1 edf9a2fae94a4b0ffa11d40ce52a48a609da9353
> > git revert -n -m 1 c43cad87172039ccf38172129c79755ea79e6102
> >
>
> Do you have to do all three? I'd have thought only the first one w= ould be
> needed.

Necessary or not, it seems to have worked. No errors were reported and
buildworld/kernel has run successfully on two of three Pi2's. One of them has rebooted and run an incremental buildworld using its new kernel and world.

Next step is to run make cleandir twice on that machine and try a
clean-start buildworld/kernel. Those results will take a few days.

In the meantime, is there any explicit test to see if the reversion
worked as expected? Searching for null (no error) results is slow.

For example, Peter Holm's stress2 suite found lots of mischief in
years gone by, might it be applied to this sort of problem?

Thanks to you and everybody else for your help!

bob prohaska

--00000000000064e2d50643b9e032--