From nobody Mon Nov 17 08:24:16 2025 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4d914h4N0bz6GqgN for ; Mon, 17 Nov 2025 08:24:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic315-8.consmr.mail.gq1.yahoo.com (sonic315-8.consmr.mail.gq1.yahoo.com [98.137.65.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4d914h2rk1z3fHH for ; Mon, 17 Nov 2025 08:24:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1763367870; bh=c7SiVbMBbMYN/a/KM7vJ5QBh2obZBCsrNRlB4LVUyLs=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=b/rQ6czKcM1QcZ9VHuZ8p58MFASMEJqfwEw/+Drk3vR0yQcUebgJ4S23r1+PuoTrGrUflVT8/TK6CqEW5bzVbwn7d4oWS4p4DyU0WGcwV2I4aSV/EyBB90w+/tQFgjsDhSwrjniHXQKBnKv6DkIr5XuZjVeFaxVjypPpUKq9sVWgvczR8Rsu1Db/jPIArVZI0/Jo0xx9SmAnJLFsnaVHFVWif747N1IeNIz2sQUhlV58LevzYKh95mHmc3bGQXpkMjzjRCZAt0/jommB+wwWLmJgxK00/TrObOF83E3Bk9hTsAFDZIBXtgFXYWvn5ynCI2myWEIxTHtgoCbxqM7yxg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1763367870; bh=FVExrfxJ8KEQH4Junpi8cCSUyY2Lw7gH03sqqoLbqPb=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=JrMY1ylCsJ71MX1vhWXb6wwUpckjeduWYyJE1mwOIQYwyesna9WGCI3oK0B8n7v0vw+Yg1BVSzVTJgmLgvDrihwxkgV72322SbM9f3aUvVxNzY5U98ZSaMVLPy2py0Cnbk5BEV4KCU5EbF0Du8RSWwqo9fJDfgq/ENiCzQzU1hBves5FKNtMrNahxm05pG4UzK7ANJ/t2hr9ZNGd1DLVS9bvcEC+76dBDVIPlXtD3FycFJoD/bXqa0yOXxkW0QoYD+wStfO3uCqJ1+cIG/TA16Q3/08UgQBB6jwIP8/Xh+eSjKyUHmFHkHM/ezXo3hW0IPbZM+xWPpr8v0+Qoh/a2g== X-YMail-OSG: w5hrrxoVM1mGkq_d.Ft6crl0SHRWM86JeS5Tz51RIX9igC5nBmoloGMuj3yW_Rb GeUoIjagz9LC_0ywhFmINCs7RkwS6FX5RcBr.YKSikhlM1e5wY_M36_3b5DmtIATXnH2Wc05xP56 kzLQA_IIM7ZJ1kQ0EJqwW4nbw1vL3c4NXeQCjYX3WeX136Q4NarZUD07_0EMptRD9QWk9dRQTM6Z 6UpHbm0L7UaLXrQlSwGBIMQNweeNGTp4TzYheuoqNVbGK_131QwXxsir5xUSJDIWB5ZDa15OEC6h DrYllIzLEe.Cj2PMSil_7VxUvJdvHV3ZLWbA3C6d_wNP0fesSZyc9.REd4yAXqB4hM28V0blNDD0 d_HdWV.YnMqoe84XalVo_SLgDcDt0265XxGaZ9VoQigYH5RVsWb7V.oPue4GohNVKz48upmQ4SyO y0rKZCzq16gqaftUf_GkcOmyLWe5c.edpCDNOgU8wnZNY30GJXbWZZqiOEO1kr8XhxU.BSSH86tD 7GGW2RgcCgKYBaM_5QnzQbY3.D8IJwTtzu.8fw3nYCPyUuy7h_W79G3Ono9ytRCbfsMBwkx3_UP4 Zm4ggTu1vhRx1hIbvsmzlrgiUbF1DKono5a8890AKoHSt7etYfWGwxisSHYwRL2g3L56aoREkkky kFFXD3A4cyM4V0ErfLWb5mVXv4HXTa25uDrW18AFeHTByFEfAmv1rqMyz4eD7iilwcKzrCoZhcjn T6qTXckhKdqqwDLZXDbJpu4vCadjZiWAmw0z_F2cuMrNsEXA1V5dW2_IPez5kjD_sFVygSE35jS3 _9pNTBorj.hLo3PnXV5GT3v1CnYAHfzMsLm0jac_BS0agnwD0nOeTcleWisywuiPunByck1c39k4 L9Jxcxs5GIq7c4YLfejEHo24ocf3SFtmVO24aZR5rwFwGVFrtGhHmhpw7PQm5oQoA5ii.56g4Rde VN8MwEAuzBay2okvie54TppL6vtlQQuow2QRWWrSMzSTE8vZlcT2DpZF91NY3y.bxykxHWdirnKA Yno1k4LdQGGw.7767fpUTZn.moNYeFdxgV3uQEoPJyL9_5.SEjHhIZG2p0PUk.w96Rl6Iq10F4TY bFmhasZvLnY0A.rTmouV9AwgRNeI0WJyp8bOVUF3v0oORhwV8rQ4GdXy8fuLIic8hV2qxlsYRmV7 fmpvmoJa_cR4sxtoIZy1BOMkvP_oTVychdOcFIWv4SiAzmJM89tIm0GTuHtSccEcNPdm8Jx4Xope pJTbM3VWf6Y.u24OuOvIzxbdolmPIwwuLblW4AkJ8j2Dnd5wtOeLbKT9243H9Im.QbpqOt4ZLEcZ byhgmvigQO.0KFQl5pQwcgyI15j51yTlK0.Cpso6x1O3.oEZS89NMYiLx2m0QtDbTBi3a03UBIcE Q20ljbuD1VMGvptv7LAo4E84974mc14.8KE0uboKeW2guZ4UCXMKhD7iLMSxfJ6SYq_HNdz4U_nr iLUeFL2zRupBWcMnNQ7oq4KPspEVifrPOyKwcvRzLIN7vhXFD0SkxkG4Nt0xKWFo5HtRDIJBzGrh 02qw4tvPPv0xP9HjUcElFg4.9XqAeRrQohrtGu8csKWdpJOzBEkmDmIZL1bv3MmXUHQZ9tbn__DR RU1LpYBi_HDRd4JGqBAZQ4_.Fh5CnaPXJDL1cdNET73Q9SUBIXCXHIYRJRsdj6JJuLkusCrAWH7i yjd.ZW2iy4ePC2fu9D7LwYnnDmUzcpKCBAQhbpt63_.XOwGOA6JI.S6c.n3TgvjRanHeUf6Gp6kH aSZ_KRw4p1H_6.il7XaxA4tf4wfxs1vy2fhwl4Fe.K6LZtVQOVSVOXQ9.FdNiBaKSjQojGuvH85m voLO0k3M54Korm6WGuufRs4V6FuJf6nK.LZOsMtUvt17zC8NcpbT87LLi9mKWamMjjJ0DlLU_xpC O31xPlo3EzhD7LvPeZ9OTEufc4nbLOT8wyY2B6EfqYs8eUKkrYHyXM_43VQbKV.5CslBRgRlgROO Ic_es0ych3SAl0_8fISute3hrog0xiHaGjZzkLa8MYk3XXup3xjFMS8V6hIGNlrVbRyzA1zxXgvn x6mdiHgCnZUZP1HfX3_t.mksLb5Bu3Jv7RIso6O5kRnGy9.P51MThALxH7vRvgqT5zZS6AJ8y3wC PcEl3YdhsjrdrGeI7Z_CfSk2xL7YZiDCZoUiF5iG2afFfrcCXIRsnM5Il95tZlKXY00pb.l.EkWM .C9lQOLbV5yzwGZDIeD24DQNVipWS0IJPnAy9XItF_sT2J96pHjph22Qm8rYlk_2r4nqe4a2aVvd bJpzpXYwiHOiCpT8- X-Sonic-MF: X-Sonic-ID: 8c7a6069-93ad-4c0f-9248-219ab76086e9 Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.gq1.yahoo.com with HTTP; Mon, 17 Nov 2025 08:24:30 +0000 Received: by hermes--production-gq1-76c986f798-mdv7f (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2d7425a1f9eaa190a7cd2694305d740e; Mon, 17 Nov 2025 08:24:27 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: Still seeing Failed assertion: "p[i] == 0" on armv7 buildworld From: Mark Millard In-Reply-To: <0a93ab5f-3fdf-45a0-8b32-2df5ef4ad60a@FreeBSD.org> Date: Mon, 17 Nov 2025 00:24:16 -0800 Cc: Warner Losh , bob prohaska , "Herbert J. Skuhra" , "freebsd-arm@freebsd.org" , FreeBSD Current Content-Transfer-Encoding: quoted-printable Message-Id: <0157ECE7-759F-4C00-9656-CB2ECA65E19E@yahoo.com> References: <4957be52-e57f-4f5f-9626-d0f706480fe1@FreeBSD.org> <87ldk9f4tt.wl-herbert@gojira.at> <0a93ab5f-3fdf-45a0-8b32-2df5ef4ad60a@FreeBSD.org> To: "mmel@freebsd.org" X-Mailer: Apple Mail (2.3826.700.81) X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4d914h2rk1z3fHH On Nov 16, 2025, at 13:11, Michal Meloun wrote: > On 16.11.2025 18:51, Warner Losh wrote: >> Maybe try main with the following patch. Adrian noticed the TLS = mismatch. I don't think it will matter, but TLS thread model stuff = always gives me a big headache. If the following fails to apply, just = copy the JEMALLOC_TLS_MODEL line from i386 to arm. The default changed = elsewhere, but this wasn't updated here. >> Warner >=20 > Unfortunately, that doesn't help. I'm out of ideas on how to debug = this, all of my attempts have failed. >=20 > The problem only occurs when Clang compiles a larger project and is = intermediate. Attempt to compile the clang generated reproducer is = always successful. > It's clear that the parallelism introduced by make plays a significant = role. But the system never reached an OOM condition before failure. >=20 > I would be grateful for any help and ideas on what to do next. > Michal [Note: The context is an official pkgbase distribution context and so the /usr/src/ is not tied to git. /usr/src-investigation/ is a copy of /usr/src/ that was then modified. Also, this is via a armv7 chroot on the aarch64 Windows Dev Kit 2023, not via armv7-only hardware.] The crude hack reported later below has shown the first failure indicated as happening during base_alloc_edata by reporting: p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x11u as the failure message. # diff -u /usr/src/contrib/jemalloc/include/jemalloc/internal/ehooks.h = /usr/src-investigation/contrib/jemalloc/include/jemalloc/ --- /usr/src/contrib/jemalloc/include/jemalloc/internal/ehooks.h = 2025-11-12 02:24:28.000000000 -0800 +++ = /usr/src-investigation/contrib/jemalloc/include/jemalloc/internal/ehooks.h= 2025-11-16 23:47:10.965711000 -0800 @@ -1,6 +1,7 @@ #ifndef JEMALLOC_INTERNAL_EHOOKS_H #define JEMALLOC_INTERNAL_EHOOKS_H =20 +#include #include "jemalloc/internal/atomic.h" #include "jemalloc/internal/extent_mmap.h" =20 @@ -158,6 +159,7 @@ * This isn't really ehooks-specific (i.e. anyone can check for zeroed = memory). * But incorrect zero information indicates an ehook bug. */ +__attribute__ ((visibility ("internal"))) extern volatile sig_atomic_t = which_base_extent_context; // HACK FOR DEBUGGING USE static inline void ehooks_debug_zero_check(void *addr, size_t size) { assert(((uintptr_t)addr & PAGE_MASK) =3D=3D 0); @@ -167,7 +169,45 @@ /* Check the whole first page. */ size_t *p =3D (size_t *)addr; for (size_t i =3D 0; i < PAGE / sizeof(size_t); i++) { - assert(p[i] =3D=3D 0); +switch (which_base_extent_context) +{ +case 0x10u: // base_alloc + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x10u); + which_base_extent_context=3D 0x0u; + break; +case 0x11u: // base_alloc_edata + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x11u); + which_base_extent_context=3D 0x0u; + break; +case 0x12u: // base_new + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x12u); + which_base_extent_context=3D 0x0u; + break; +case 0x13u: // base_boot + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x13u); + which_base_extent_context=3D 0x0u; + break; +case 0x20u: // extent_commit_wrapper + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x20u); + which_base_extent_context=3D 0x0u; + break; +case 0x21u: // extent_commit_zero + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x21u); + which_base_extent_context=3D 0x0u; + break; +case 0x22u: // ecache_alloc_grow + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x22u); + which_base_extent_context=3D 0x0u; + break; +case 0x00u: // None known + assert(p[i] =3D=3D 0 && which_base_extent_context =3D=3D 0x00u); + which_base_extent_context=3D 0x0u; + break; +default: // Some other context + assert(p[i] =3D=3D 0 && which_base_extent_context !=3D 0x00u); + which_base_extent_context=3D 0x0u; +} + //assert(p[i] =3D=3D 0); } /* * And 4 spots within. There's a tradeoff here; the = larger # diff -u /usr/src/contrib/jemalloc/src/base.c = /usr/src-investigation/contrib/jemalloc/src/base.c --- /usr/src/contrib/jemalloc/src/base.c 2025-11-12 = 02:24:28.000000000 -0800 +++ /usr/src-investigation/contrib/jemalloc/src/base.c 2025-11-16 = 23:50:14.396483000 -0800 @@ -1,3 +1,4 @@ +#include #include "jemalloc/internal/jemalloc_preamble.h" #include "jemalloc/internal/jemalloc_internal_includes.h" =20 @@ -340,12 +341,15 @@ b0get(void) { return b0; } + +__attribute__ ((visibility ("internal"))) volatile sig_atomic_t = which_base_extent_context=3D0x0u; // HACK FOR DEBUGGING USE =20 base_t * base_new(tsdn_t *tsdn, unsigned ind, const extent_hooks_t = *extent_hooks, bool metadata_use_hooks) { pszind_t pind_last =3D 0; size_t extent_sn_next =3D 0; +which_base_extent_context=3D 0x12u; =20 /* * The base will contain the ehooks eventually, but it itself is @@ -476,12 +480,14 @@ */ void * base_alloc(tsdn_t *tsdn, base_t *base, size_t size, size_t alignment) { +which_base_extent_context=3D 0x10u; return base_alloc_impl(tsdn, base, size, alignment, NULL); } =20 edata_t * base_alloc_edata(tsdn_t *tsdn, base_t *base) { size_t esn; +which_base_extent_context=3D 0x11u; edata_t *edata =3D base_alloc_impl(tsdn, base, sizeof(edata_t), EDATA_ALIGNMENT, &esn); if (edata =3D=3D NULL) { @@ -523,6 +529,7 @@ =20 bool base_boot(tsdn_t *tsdn) { +which_base_extent_context=3D 0x13u; b0 =3D base_new(tsdn, 0, (extent_hooks_t = *)&ehooks_default_extent_hooks, /* metadata_use_hooks */ true); return (b0 =3D=3D NULL); # diff -u /usr/src/contrib/jemalloc/src/extent.c = /usr/src-investigation/contrib/jemalloc/src/extent.c --- /usr/src/contrib/jemalloc/src/extent.c 2025-11-12 = 02:24:28.000000000 -0800 +++ /usr/src-investigation/contrib/jemalloc/src/extent.c = 2025-11-16 23:49:55.820658000 -0800 @@ -1,3 +1,4 @@ +#include #include "jemalloc/internal/jemalloc_preamble.h" #include "jemalloc/internal/jemalloc_internal_includes.h" =20 @@ -90,11 +91,14 @@ assert(edata =3D=3D NULL || edata_guarded_get(edata) =3D=3D = guarded); return edata; } + +__attribute__ ((visibility ("internal"))) extern volatile sig_atomic_t = which_base_extent_context; // HACK FOR DEBUGGING USE =20 edata_t * ecache_alloc_grow(tsdn_t *tsdn, pac_t *pac, ehooks_t *ehooks, ecache_t = *ecache, edata_t *expand_edata, size_t size, size_t alignment, bool zero, bool guarded) { +which_base_extent_context=3D 0x22u; assert(size !=3D 0); assert(alignment !=3D 0); witness_assert_depth_to_rank(tsdn_witness_tsdp_get(tsdn), @@ -1114,6 +1118,7 @@ bool extent_commit_wrapper(tsdn_t *tsdn, ehooks_t *ehooks, edata_t *edata, size_t offset, size_t length) { +which_base_extent_context=3D 0x20u; return extent_commit_impl(tsdn, ehooks, edata, offset, length, /* growing_retained */ false); } @@ -1297,6 +1302,7 @@ bool extent_commit_zero(tsdn_t *tsdn, ehooks_t *ehooks, edata_t *edata, bool commit, bool zero, bool growing_retained) { +which_base_extent_context=3D 0x21u; witness_assert_depth_to_rank(tsdn_witness_tsdp_get(tsdn), WITNESS_RANK_CORE, growing_retained ? 1 : 0); =20 =3D=3D=3D Mark Millard marklmi at yahoo.com