From nobody Tue Nov 26 13:21:31 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XyNWw1gH8z5fSKQ for ; Tue, 26 Nov 2024 13:21:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-20.consmr.mail.gq1.yahoo.com (sonic306-20.consmr.mail.gq1.yahoo.com [98.137.68.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4XyNWv5vRJz4Hyh for ; Tue, 26 Nov 2024 13:21:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732627306; bh=CQcaDV1rCASwiOGxrw4JwGMYjI2wJFN6viGlqigTp7E=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=KMo9Os87mXgoIB4l6PYk8M6QxXnWy+LGe7waePoGb0Ru+5i0ZmUk4tFaNAYYn1ULLENsTvRoryQp368qc8uM3IaYEsbpxOP2vKc8d0XjZqvWY4NAPyoBoLz7f2nRat7iZ5DuSjuBBKVcJx1eIumP2+QufOPqhNiTitEM4gn0pR8igMDYKO955vREMKZyyIvRp1apdwz0b19gNbkRXjCbXARPH1CaD5KjGuHnNVKvhHvzN+FrbjRBps4EohOe84n8V5tl6xQHjQvo9NWL8Dcffrxre5ZpDUIlg7mX6l6K1VJoXGnsiFB9EpYz4PPXNN0li0L1mHBjzz9Y4vkGzzs2QQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1732627306; bh=01Jxo3cFYVuAoCFx6R1vU7Q5pwNh9eQQ3kMifFjgStA=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=srcqweq3w7ismN6vcv0X/p9mz3MJOCPZ5jgXw5lWr3vhh+fE83EePzvfMsnyUIWhGkMVibij1t/1zHQgCsBXhgP8zzDkqEDOkXHsQmp8+GAySTNUUW2q3Zd1i7zQviuPP+TDZqg79HMO3t0LTGFY/x4Aw5JV0GCs3PMymHssSqsvAMUHd0+rkKTrHOaaIIXzVQLOFVbNIMufp7853D96tFCVPKhTYV9UD0XUa5r0EdzcXNB3D2y4Cvokn0pVCpNGT+Rs6LB8TkGLNu2BLWnjYJNPgQ85JkNnYAaB5gzx8G9QnLZ4NEjIBNHAuJIJ6m7ZAH8FDpIPydQwmbiH4qXOzw== X-YMail-OSG: ZEx4BOQVM1kxQzv1wlw.TJreZHe8BByOquud0zf7CqJonsSI.Vz6Dxi0JZLhmBt .GO3EEoyNSqhYMj2eHp_GAX_u1nF5fQdYmZ32yQ8sN3Y_anqJ1rqhi06GgM6P8GO.3NEvDF4UPB3 1Q_QZ3bagXECiCBsPifFlG.D6yaDfwsEZL0h_9dCfukdPlJbS65bXxOiL1wyHkjkQAjjAsLsRrIb 9Od8q0UbthaPF0PAmK2wbgGDUJG83Tv0KyoMXJpb3i4ImGIsB4ABz1jJ_DF2nYZWCLWK3ojRMp2P bjr2UPZWadx56J96XwV0TvFo4y8nz9cNC6HEXfNklYaAqa46oQxyHxEQdD5HRivZ_gch03zU_80d G62P30p.K32ZnRlSiUScYZUhf_hU9cO6y2A8XRNxObQopRqJhlzhZKm9nkof_SUiwr3me6WmCi5k poPjcSud3uA2RljpJsFtXOfVEQaOvhz2cCSGjD_CeDAIADuncU1pxZVKrnD9ylXiFIO16Acf9Y8u 7NMMVlDLySc1xLY_foN6_UwH3DSsTMuNxRKLSXf0pIKBuVnCU6fUhXJWC_uxdhXz6RH6l8wIWTFM wJRDUZOwtivecnzFKVZR19.udsOxb9ArGrWLvl4fCJbQO8LMbg05E8OVXZGKG_730_LKBFUu_dVk eE8h3oY5A8ZvO4LnvVA2jt6EoKlFbJsc2CxznbwQDsMDSC.28xlZHH0YXo5__BpnpE2rEXGBHuYm J0ek4vyW7xBKkg2arlO3m27nNJzSLc8fxtk1d2iDStsE_.eIzKJ_6sZ2w5GAK_m1LHw9K836bQHy 4S0blBENu6B3iy9Gl1KyUhlyE6oeayb73y3JDrZipknd0I9q0koCxpeyzyDpUPX.4zN2m8zLyNIT G.lbUD7lvHgFYNdP3PgZGiPvuC1m.6_jY1Xi7ODHvKP.Hx9h3cHq346lox2JexWbYEANK54UHtRG T50jEP5x1mUx9mFcU7QXBAkmTqcbigieJs8OUGI9eaSXtf_R34kcujWmdClI5d3qo9HzgInhS68X KRIDLa69yTusX83YuEJv.k.Zqx18znOEPaJcA5eddEsfxaJo0p9FaxpVFDXo63Ohouz5kVDXtqHa AnJze6FkXC3t0Uo9ibaMA4ZoQ_QCBBvu43v9E2ct42WNJEvI86wZWpzS1smKkcGtQxHROC5Q122l M1D0JOxbQqopeiMVMhbEDExeI8AWQw01y9ygT96kZYX28tFd7NA63nBuZ_DQRj_2DoelCNfRBGte ivJqbA8wTy9k24s1DyGphVqfYqVUgmGruSdSwaMKWSILoLHIfrQWuRV7qVFmnwSVOWVnLt_AxmEK h4j8v4iw1roji.mmxcJvemd82IeBXwYjppV.CHYCD4i9KE8U6WLyUyhXQ6lnhvrbCdK4qnJ5XblO oxw_5Jp46Rxz3i5TA7teSEiOOFaioeu_ZIotSj4jcnpx3WCIMBsEEYDJMrH5mLOFYbMSd4ksaUv0 2xArwyU_HsMhf5ZzTdWk.1T.ZKpyxSx15OQOFwgciUHkUADWQVetby5d.4DvrfGVox_jIlGotRiW 1YHSvdUJ4rsIZjIxbN.jVtRGpJmZgur7_2ayl_PA8Gmg4PnSVKx9xZHjwO8A26dlL_mhdrygB0aa ErzKMnRn4uefxV0c0pE2WiKHbWLXEGAVWfZbBhomG5Hp08g2oYL6JpWn1l6BSQu..P2Kvb_UeRWR AEpQMNQSDPs43cHZSKNun1R9kaBgBViaEbiF5Om.Rnt3IkViwUKELjmJDobMcwfWnaAzVInL1aKz tgo1Nkj18rY6cPkZeJA4WTD9LVXBFLcpaxDCb9qlDyVUGUCQxNJXv1KNWRaQiz6ZVIBQcZae3t4K _9ERLIO0lDv_4ep62KjTSYWIv0VF.Utx3nji5.K5ywSCGYC.j_G99OUAHCsfxjck0.HIxf1forjD _iZMfGSY_O4BoKDhbSpfWT3RC3hvG4NRUK_DmWQEPrLay9Kgp4SHAXNfJnUrH0GOmvUmUDq.LHIx Niu.Torw5VpRBfpImWwDDh7jphJmp4EnrF1_oazveElhldG7VPsHQlMsOm.AwOBpoVy5FLAYy7AF ew2dv4HJZM298Bsz3JSuj5HkLnCqCMJctbOOlMK8F8VyACj2QU9bTNZIP7zJUHONaK4DKVbDMjmb iJ5SdO1glmBB8wn_PU29XpTBMw9NXG9TQxDIxYexaoB2oth_Of.M4xt5dqWPaKEiQuZ8fBv2nx8M XuO48mBQO60TUz4La_o2DCcAH2RdysTM54.uzwHEeIQUbp8zql6m7RZPN_Ql59mP3dpTd5_buFnY zaBk5XF1oh1rdenHu X-Sonic-MF: X-Sonic-ID: 06851145-5804-421b-bb7e-4406c40da9c2 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Tue, 26 Nov 2024 13:21:46 +0000 Received: by hermes--production-gq1-5dd4b47f46-whghm (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID c7bc7af1bd60671337a8ffcde8537c81; Tue, 26 Nov 2024 13:21:42 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3776.700.51.11.1\)) Subject: Re: port binary dumping core on recent head in poudriere [tmpfs corruptions involving blocks of zeros that should not be all zeros] From: Mark Millard In-Reply-To: <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> Date: Tue, 26 Nov 2024 05:21:31 -0800 Cc: =?utf-8?Q?Dag-Erling_Sm=C3=B8rgrav?= , "jah@freebsd.org" , dougm@freebsd.org, Alan Somers , Mark Johnston , FreeBSD Current , Guido Falsi , Yasuhiro Kimura , ports@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <4EDAA149-2D45-40D3-A2CA-F6FBD06C18BF@yahoo.com> References: <7c9c3cf5-bbd1-4642-8d04-33aa07a4db02@madpilot.net> <9df256a8-c6ed-46d9-b955-fc2657c12d36@madpilot.net> <5c502054-7353-4a1e-8350-c403482e9c0d@madpilot.net> <3127C3BA-FC93-4636-ADDB-89518DE9C60D@FreeBSD.org> <86ed2zsp6l.fsf@ltc.des.dev> <5f24a570-26e0-4c0a-817f-591a234fd07b@madpilot.net> <5918C6A1-8FDB-40CA-8C86-EB7B7BE75A2E@yahoo.com> <86ed2zc8r5.fsf@ltc.des.dev> <45098ccf-4dc6-426c-849a-c923805d6723@madpilot.net> <38658C0D-CA33-4010-BBE1-E68D253A3DF7@FreeBSD.org> <1004a753-9a3c-4aa2-bfa8-4a0c471fe3ea@madpilot.net> <0690CFB1-6A6D-4B63-916C-BAB7F6256000@yahoo.com> <3660625A-0EE8-40DA-A248-EC18C734718C@yahoo.com> <865xoa2t6f.fsf@ltc.des.dev> <69A2E921-F5E3-40D2-977D-0964EE27349A@FreeBSD.org> <4AE5B316-D7EB-4290-8D52-7FBF244EA7A4@FreeBSD.org> To: Dimitry Andric X-Mailer: Apple Mail (2.3776.700.51.11.1) X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Queue-Id: 4XyNWv5vRJz4Hyh X-Spamd-Bar: ---- On Nov 26, 2024, at 04:58, Dimitry Andric wrote: > On 26 Nov 2024, at 13:32, Dimitry Andric wrote: >>=20 >> On 26 Nov 2024, at 11:19, Dag-Erling Sm=C3=B8rgrav = wrote: >>>=20 >>> Mark Millard writes: >>>> =46rom inside a bulk -i where I did a manual make command >>>> after it built and installed libsass.so.1.0.0 . The >>>> manual make produced a /wrkdirs/ : >>>> [...] >>>> So the original creation looks okay. But . . . >>>> [...] >>>> So: The later, staged copy is a bad copy. Both are in the >>>> tmpfs. So copying to the staging area makes a corrupted >>>> copy inside the same tmpfs. After that, further copies of >>>> staging's bad copy can be expected to be messed up. >>>=20 >>> This and the fact that it happens on 14 and 15 but not on 13 = strongly >>> suggests an issue wth `copy_file_range(2)`, since `install(1)` in 14 = and >>> 15 (but not in 13) now uses `copy_file_range(2)` if at all possible. >>>=20 >>> My educated guess is that hole detection doesn't work reliably for = files >>> that have had holes filled while memory-mapped, so = `copy_file_range(2)` >>> thinks there is a hole where there isn't one and skips some of the = data >>> when `install(1)` uses it to copy the library from `${WRKSRC}` to >>> `${STAGEDIR}`. This may or may not be specific to tmpfs. >>>=20 >>> You may want to try applying the attached patch to your FreeBSD 14 = and >>> 15 jails. It prevents `cp(1)` and `install(1)` from trying to use >>> `copy_file_range(2)`. >>=20 >> Yes, tmpfs is indeed the culprit (or at least involved). I have had = USE_TMPFS=3Dlocalbase in my poudriere.conf for a long time, since = otherwise my build machine would run out of memory very quickly, Use of TMPFS_BLACKLIST and TMPFS_BLACKLIST_TMPDIR can allow the use of USE_TMPFS=3Dall in many contexts. I'll later show my list that tries to exclude most everything using more than 7 or so GiBytes of tmpfs for the builder. If nothing else, it can help have a context for testing for the failure at hand for fairly general builds, including "bulk -a" . >> so I didn't encounter any issues. >>=20 >> Now I changed it to USE_TMPFS=3Dyes, rebuilt only textproc/libsass = and textproc/sassc, and then after reinstalling those packages: >>=20 >> $ /usr/local/bin/sassc >> Segmentation fault >=20 > And after applying Dag-Erling's patch to disable copy_file_range for = cp and install, it works correctly again. For reference (a very long line in its original form, noted in case something splits the line): TMPFS_BLACKLIST=3D"*-emacs_devel *-emacs_devel_nox *-emacs_nox *-gcc14 = *-rust-bootstrap 0ad RStudio aarch64-none-elf-gcc afni alliance anki = apache-openoffice apache-openoffice-devel arm-none-eabi-gcc binutils = biostar-tools blender boost-libs chezmoi chromium = chrono-physics-simulation-engine clickhouse cmake-core code_saturne deno = diaspora digikam dotnet dune-common dune-localfunctions dynare eclipse = eksctl electron[1-9][0-9] ess ess-emacs_canna firefox firefox-esr = foundationdb fr-aster freebsd-gcc14 gcc-arm-embedded = gcc-msp430-ti-toolchain gcc14 gcc1[45]-devel gdb geant4 ghc ghc810 ghc92 = ghc94 ghemical giacxcas grafana grafana-loki grafana9 gretl = gstreamer1-plugins-rust heyoka hs-cardano-db-sync = intel-graphics-compiler-llvm1[4321] iridium-browser julia kde5 kicad = kicad-devel kicad-doc kicad-library-packages3d* kosmorro kstars = libghemical libint2-psi4 libreoffice librewolf librsvg2-rust = libva-intel-media-driver llvm-devel llvm1[98764321] mesa-dri = mongodb[4-9][0-9] mpqc nerd-fonts nextcloudclient nextpnr octave = octave-forge octave-forge-bim octave-forge-msh octave-forge-sec*d = octave-forge-sole onlyoffice-documentserver paraview piglit = py39-orange3-single py39-pytorch pydio-cells qemu qemu-devel qemu-nox11 = qemu7 qemu7-nox11 qgis qgis-ltr qt*-webengine qt[56]*-webengine = qt[56]-tools quantum-espresso-pseudopotentials ringrtc rust rust-nightly = signal-desktop simpleitk telegraf tex-dvipdfmx tex-luatex tex-xetex = texlive-docs texlive-full thunderbird tor-browser trilinos trivy ttk = ungoogled-chromium vault vaultnextcloudclient virtualbox-ose = virtualbox-ose-legacy virtualbox-ose-nox11 vscode vuls = wasi-compiler-rt-* wasi-libcxx* webkit2-gtk3 wx30-gtk3 yazi ztop" =3D=3D=3D Mark Millard marklmi at yahoo.com