From nobody Sat Apr 15 22:49:32 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzT644kZVz45DyQ for ; Sat, 15 Apr 2023 22:49:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-24.consmr.mail.gq1.yahoo.com (sonic312-24.consmr.mail.gq1.yahoo.com [98.137.69.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzT634VSLz3nCn for ; Sat, 15 Apr 2023 22:49:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=EME8USLL; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681598985; bh=QFriFYybrt/DkPHkifqdhSpzObbt2yFMQDV+B0+0Gpk=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=EME8USLLQlz/XvV78ekYaCznMRKJjPLkW8FW51e6QnVoPwi49dygbbdxUWqWmPSB6qFfJZXYP5zgDG3kcY44VwEBIUYXlnVz5dB6Agw/S/mOXfuD8bJXhoWmdNmivSDwI6aeeYx/Xuf6Bsnr1CqWBqZIRIeQJ4BEXyfmXsws04MzQQWrxlD5pWINULJGOT/0TwzW/G2JFHIHtKXrpaXi0IDpX4+P0oHyWmcwQXl9B6jmwsdn14rnMglgS1GZJbM+lFp1N0kV5mnt8kZJWFLudkT4bzyHYoCDuO8bnRRbEaTbyUE6f1eHttDFYDeOIXT8z52I4Ld4MhTrkM+FdezfYQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681598985; bh=4LDDa5fezD0TEFFGxFNdBW3YyfZO9VxcraiwXPrPHgj=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=f51jeM6wt39jVRA5Ytk5To2KF1iRsFyePNxq04CsP6BtwR8b+l4dqu7/uPJ1jue9Ug1Fiau+yYkPl8ij9qsUxCjgpSSlgMb/49DwrJ7W6DOO7sIbuRJMz41sh7Y24nHcRbLjNMX461fxSg3uSpuY249R5tvltkjCRZ942ZPgec+TgCEeKugrTNJ7hWal6cylWEPt7I4eu+T3Pnq7Dx9ixcD+pprDkQsXM6N0jh4o3Fz0hYtGs0f3QnwKRE8jcYSor+W/Wy83JZoZkl8fOZldbAuHVubTc0KCI0f2P638hAOw4eMvaCegr3mZFDmWiz68YIIlK0QrxtbUgm/lkMqdxQ== X-YMail-OSG: 0K9_Kc4VM1lT3EgQBtL79GXSxYqrm6pvLoT0J7u3pFAIwL9JfbXmmTonepFPUO0 15BslKRImTRoWMqazFCTGBupxaD2eF1R._A5Wdf39zD22PrqzjRbLbQkbJzB_.dtRDiWIYfCQWew 9FnRKkD_VaiwRtt6TdrH6NJg5EDqdGqUUMdobsW56IewcUsbxbKg8_bqeGSpWLSI2rxFRejKRWaI ivTudRpKp2Lwvlx3QCwG.RxHtnjPSxaSxAq4WosZofBqXRDYdOMMTo7k7bSCHKcJo3mt.TxXBkc7 7UzXo4gnVochJ3o2PUmqA0rf3NLD.ZznQ9MN3SiBy7HEzrjnSSxv7QmInWdxBem82LFf873U2oU4 sBStdoi6cbon6GEMUDvVZuw4TGHSNy4Hz9cVowmBmxAwGJwcH0ZOpkkDKYbi5cfQxwQq8U6YlX5E vIupxm5jg9JzOOiD8eiCXnT8BWBc4w2H8T25_AE7JGxiuyZELtnnh7tp4WC9Pb0l5.7OXqlz7Xq8 C55lw9BFUi7HijHVU.9dmCsmJrHICGTte4Q3MjgmuHnoZtnQXCnwy8qd_Jneas8s5BuHH4KDpIAr cEwmQoyaIOL7XRI1_nutoSZvj82h.wigQNtsji2bwWdODgNbOf5ApLtlLFnWmAajEQOHytXXRmGq jmdrxehBE3Q6tDeyZ1HV5doXd_Ln4hgh_8xPl4HZE.RDj0kLLzxwhZ4vrkKLWswg3KOlkRA4Omrj TA7Pfkdk3lJIxTc4_Crk7omPehXbJtoFPKJLqad18ZoPjscYjrY0AAugale9mluLH5AQG3K9rh_Q j9W3GQB.dmv8OhVS4Y1zxjX5vmpXqQAnhfiE199f3eqdc0X0MXEE1DIwpZQ9cFSPqxLhposxiir9 wGFkxBLu9jFiVUYaW0KvV1u51hGrHkoHszYcwydIKQT9OnPt.gcG8dU6CateBGam3ItkyUrbdvGz Bf3scAL_YDRzzLAmZ0vAf5kFutQOfIlGfZiyIa2.S86QLLy8QjNB.QGRoAyywZaOHZBYDg2sVlMF NJD6BKIyLKDQwgh9Keblf1PDm1tM_Te5NemtCe7CBu6AbFB_UfEsKzmeF7v5O1kx.Hjk2e2JeK81 SreiU8agAcuIz8LrMb1pbMzMTREvHxorWs0uN0_px1Qdycvf5U5g.xDL5jnL0MN6SQ8LHmPotCZk zNn2KC7_36jCc0Mqx_0efVRMbweXXlYNZlFAp0jtVb8Gy1YwoCWoCkOntE1f8yy7R8AHTx7Krc1P c5dqVFYjSTP1kanWz1Xhg8MWdjMTmvods7ZmLAC6wjjY.JqsBiIJFps0nSzYmJVJ6M2qOumDfIMa TvxdZz.6syET_wxN6SRR1mSvNDbh1mFkXcahfhzWJcI_N9gc8AxkndL09zsSfl9OQNcSxST1zTxX I8FMmaMXMCHJck9Dv8Sqcn5Mdr50bJIJsakVCITacDgVOA6yXSBizkvHJ0zs6SO54Ezaxeqnshru xmtgaqpv2Bkn.baRz0AfI6rEf8Ty2F00HoNPs5dfJqzWlqfOVWxCkQFAxBtMb2f9vELWc27O5t68 sEvU1hxehcV0wScz0ONbth3z_2rDEBSIqTQ3paIMVaF1JdSYzw7CJX7.JrUFJv1OgdItszfTTPoS 0cznKDdt8n7pI4Bd3hO7BEcaXom9E8mSI60CqEEZemvZTuXMZyPRg8gQWvKAb7Nthhg3fUEIZ5Fb wYzNo87b0gPLfN4mGiq8LYrHDmw5CqihiZ5zUxVlBJiP0yq390_iOjp291PmsMJ3BweUWx1aN6xf QfS5Bge2wPWW.tMPgX3OKj27tyjuCtHMgP0May0AaXDLf9rQ3t02XJXj4xVnWk6Aj91jOGl2WVJq dO0t0Pj1QT92mTPSCPcf.C9ZvUAMTKox8we2qOWkb7ZC0DB9iftLlX08SWmV9qMMIuioNJoDRnXg IPllye2kq6XGsnrJKJ_z9qAP7aM1lt8jfdlE_jgP9Fa94EKCyd7P3pDiHLHsY.XCiMoTnTjJIS1E uwDLP2PHIofl.UNGmZgKc_rrl89yfumwJQvWNjecDFAM26BURvelxLV0tuRaWDe7DcPeEyWmsSq_ mHs99X4khE.0TnH8Itc_o3G_hPAONZWr6Gmwj4zlfhRP4mzNn0hPbHkCA0L8vdU7U14.ViomDUe2 a93C_N3Tx5o6O1whv3tAqtbxbN64rKgzUCaNts8NDQFxHY2KrImk4ovZP6sn7fSsxZvAIpEKJ0s9 migHFz.YZXGgKcSbfDEEDirLQbaNHIUJ4MS8Neah1SQYAFpzYm0P.xKMlVVmvS2_Ik_LTHUVQkPN 3Bn.K_C5k X-Sonic-MF: X-Sonic-ID: 0be95355-c582-4799-9fe4-a1c7a69b5fd6 Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sat, 15 Apr 2023 22:49:45 +0000 Received: by hermes--production-gq1-546798879c-sfj59 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID f4b3aba42658f92ddcaaff6db7af9bd6; Sat, 15 Apr 2023 22:49:43 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: <62F8DD62-8E09-4A07-926B-5DE3DB399609@yahoo.com> Date: Sat, 15 Apr 2023 15:49:32 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <0C286158-8F0D-45D0-8D85-B521278E9518@yahoo.com> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <62F8DD62-8E09-4A07-926B-5DE3DB399609@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.47 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.972]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.205:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_SEVEN(0.00)[7]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4PzT634VSLz3nCn X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 15, 2023, at 15:33, Mark Millard wrote: > On Apr 15, 2023, at 13:30, Mateusz Guzik wrote: >=20 >> On 4/15/23, FreeBSD User wrote: >>> Am Sat, 15 Apr 2023 07:36:25 -0700 >>> Cy Schubert schrieb: >>>=20 >>>> In message = <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de>, >>>> FreeBSD Us >>>> er writes: >>>>> Am Thu, 13 Apr 2023 22:18:04 -0700 >>>>> Mark Millard schrieb: >>>>>=20 >>>>>> On Apr 13, 2023, at 21:44, Charlie Li = wrote: >>>>>>=20 >>>>>>> Mark Millard wrote: >>>>>>>> FYI: in my original report for a context that has never had >>>>>>>> block_cloning enabled, I reported BOTH missing files and >>>>>>>> file content corruption in the poudriere-devel bulk build >>>>>>>> testing. This predates: >>>>>>>> https://people.freebsd.org/~pjd/patches/brt_revert.patch >>>>>>>> but had the changes from: >>>>>>>> https://github.com/openzfs/zfs/pull/14739/files >>>>>>>> The files were missing from packages installed to be used >>>>>>>> during a port's build. No other types of examples of missing >>>>>>>> files happened. (But only 11 ports failed.) >>>>>>> I also don't have block_cloning enabled. "Missing files" prior = to >>>>>>> brt_rev >>>>> ert may actually >>>>>>> be present, but as the corruption also messes with the file(1) >>>>>>> signature, >>>>> some tools like >>>>>>> ldconfig report them as missing. >>>>>>=20 >>>>>> For reference, the specific messages that were not explicit >>>>>> null-byte complaints were (some shown with a little context): >>>>>>=20 >>>>>>=20 >>>>>> =3D=3D=3D> py39-lxml-4.9.2 depends on shared library: = libxml2.so - not >>>>>> found >>>>>> =3D=3D=3D> Installing existing package = /packages/All/libxml2-2.10.3_1.pkg >>>>>>=20 >>>>>> [CA72_ZFS] Installing libxml2-2.10.3_1... >>>>>> [CA72_ZFS] Extracting libxml2-2.10.3_1: .......... done >>>>>> =3D=3D=3D> py39-lxml-4.9.2 depends on shared library: = libxml2.so - found >>>>>>=20 >>>>>> (/usr/local/lib/libxml2.so) . . . >>>>>> [CA72_ZFS] Extracting libxslt-1.1.37: .......... done >>>>>> =3D=3D=3D> py39-lxml-4.9.2 depends on shared library: = libxslt.so - found >>>>>>=20 >>>>>> (/usr/local/lib/libxslt.so) =3D=3D=3D> Returning to build of >>>>>> py39-lxml-4.9.2 >>>>>> . . . >>>>>> =3D=3D=3D> Configuring for py39-lxml-4.9.2 >>>>>> Building lxml version 4.9.2. >>>>>> Building with Cython 0.29.33. >>>>>> Error: Please make sure the libxml2 and libxslt development = packages >>>>>> are in >>>>> stalled. >>>>>>=20 >>>>>>=20 >>>>>> [CA72_ZFS] Extracting libunistring-1.1: .......... done >>>>>> =3D=3D=3D> libidn2-2.3.4 depends on shared library: = libunistring.so - not >>>>>> found >>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> [CA72_ZFS] Extracting gmp-6.2.1: .......... done >>>>>> =3D=3D=3D> mpfr-4.2.0,1 depends on shared library: libgmp.so - = not found >>>>>>=20 >>>>>>=20 >>>>>>=20 >>>>>> =3D=3D=3D> nettle-3.8.1 depends on shared library: libgmp.so - = not found >>>>>> =3D=3D=3D> Installing existing package = /packages/All/gmp-6.2.1.pkg >>>>>> [CA72_ZFS] Installing gmp-6.2.1... >>>>>> the most recent version of gmp-6.2.1 is already installed >>>>>> =3D=3D=3D> nettle-3.8.1 depends on shared library: libgmp.so - = not found >>>>>>=20 >>>>>> *** Error code 1 >>>>>>=20 >>>>>>=20 >>>>>> autom4te: error: need GNU m4 1.4 or later: /usr/local/bin/gm4 >>>>>>=20 >>>>>>=20 >>>>>> checking for GNU >>>>>> M4 that supports accurate traces... configure: error: no = acceptable m4 >>>>>> coul >>>>> d be found in >>>>>> $PATH. GNU M4 1.4.6 or later is required; 1.4.16 or newer is >>>>>> recommended. >>>>>> GNU M4 1.4.15 uses a buggy replacement strstr on some systems. >>>>>> Glibc 2.9 - 2.12 and GNU M4 1.4.11 - 1.4.15 have another strstr = bug. >>>>>>=20 >>>>>>=20 >>>>>> ld: error: /usr/local/lib/libblkid.a: unknown file type >>>>>>=20 >>>>>>=20 >>>>>> =3D=3D=3D >>>>>> Mark Millard >>>>>> marklmi at yahoo.com >>>>>>=20 >>>>>>=20 >>>>>=20 >>>>> Hello >>>>>=20 >>>>> whar is the recent status of fixing/mitigate this desatrous bug? >>>>> Especially f >>>>> or those with the >>>>> new option enabled on ZFS pools. Any advice? >>>>>=20 >>>>> In an act of precausion (or call it panic) I shutdown several = servers to >>>>> prev >>>>> ent irreversible >>>>> damages to databases and data storages. We face on one host with >>>>> /usr/ports r >>>>> esiding on ZFS >>>>> always errors on the same files created while staging (using = portmaster, >>>>> leav >>>>> es the system >>>>> with noninstalled software, i.e. www/apache24 in our case). = Deleting the >>>>> work >>>>> folder doesn't >>>>> seem to change anything, even when starting a scrubbing of the = entire >>>>> pool (R >>>>> AIDZ1 pool) - >>>>> cause unknown, why it affects always the same files to be = corrupted. >>>>> Same wit >>>>> h deve/ruby-gems. >>>>>=20 >>>>> Poudriere has been shutdown for the time being to avoid further = issues. >>>>>=20 >>>>>=20 >>>>> Are there any advies to proceed apart from conserving the boxes = via >>>>> shutdown? >>>>>=20 >>>>> Thank you ;-) >>>>> oh >>>>>=20 >>>>>=20 >>>>>=20 >>>>> -- >>>>> O. Hartmann >>>>=20 >>>> With an up-to-date tree + pjd@'s "Fix data corruption when cloning >>>> embedded >>>> blocks. #14739" patch I didn't have any issues, except for email = messages >>>>=20 >>>> with corruption in my sent directory, nowhere else. I'm still >>>> investigating >>>> the email messages issue. IMO one is generally safe to run = poudriere on >>>> the >>>> latest ZFS with the additional patch. >>>>=20 >>>> My tests of the additional patch concluded that it resolved my last >>>> problems, except for the sent email problem I'm still = investigating. I'm >>>> sure there's a simple explanation for it, i.e. the email thread was >>>> corrupted by the EXDEV regression which cannot be fixed by = anything, even >>>>=20 >>>> reverting to the previous ZFS -- the data in those files will = remain >>>> damaged regardless. >>>>=20 >>>> I cannot speak to the others who have had poudriere and other = issues. I >>>> never had any problems with poudriere on top of the new ZFS. >>>>=20 >>>> WRT reverting block_cloning pools to without, your only option is = to >>>> backup >>>> your pool and recreate it without block_cloning. Then restore your = data. >>>>=20 >>>>=20 >>>=20 >>> All right, I interpret the answer that way, that I need a most = recent source >>> tree (and >>> accordingly built and installed OS) AND a patch that isn't = officially >>> commited? >>>=20 >>> On a box I'm with: >>>=20 >>> FreeBSD 14.0-CURRENT #8 main-n262175-5ee1c90e50ce: Sat Apr 15 = 07:57:16 CEST >>> 2023 amd64 >>>=20 >>> The box is crashing while trying to update ports with the well known = issue: >>>=20 >>> Panic String: VERIFY(!zil_replaying(zilog, tx)) failed >>>=20 >>> At the moment all alarm bells are ringing and I lost track about = what has >>> been patched and >>> already commited and what is still as patch available but in the = phase of >>> evaluation or >>> inofficially emmited here. >>>=20 >>> According to the EXDEV issue: in cases of poudriere or ports trees = on ZFS, >>> what do I have to >>> do to ensure that those datasets are clean? The OS should detect = file >>> corruption but in my >>> case the box is crashing :-( >>>=20 >>> I did several times scrubbing, but this seems to be the action of a = helpless >>> and desperate man >>> ... ;-/ >>>=20 >>> Greetings >>>=20 >>=20 >> Using block cloning is still not safe, but somewhere in this thread >> pjd had a patch to keep it operatinal for already cloned files = without >> adding new ones. >>=20 >> Anyhow, as was indicated by vishwin@ there was data corruption >> *unrelated* to block cloning which also came with the import, I >> narrowed it down: https://github.com/openzfs/zfs/issues/14753 >>=20 >> That said now I'm testing a kernel which does not do block cloning = and >> does not have the other problematic commit, we will see if things >> work. >=20 > (Mostly written as I progressed but some material later > inserted into/around previously written material.) >=20 > Summary: >=20 > As stands, it looks like reverting the dnode_is_dirty > code is what fixes the corruptions that my type of > test context produced via poudriere bulk activity . >=20 >=20 > The details that lead to that summary . . . >=20 > Using my my build environment for updating my temporary, > experimental context, an environment running a world and > and kernel that predate the import: >=20 > # uname -apKU > FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #90 = main-n261544-cee09bda03c8-dirty: Wed Mar 15 20:25:49 PDT 2023 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400082 1400082 >=20 > (Note the "nondbg": I normally run non-debug main builds, > but with symbols not stripped.) >=20 > The kernel and world for this are what is in old-main-CA72: >=20 > # bectl list > BE Active Mountpoint Space Created > main-CA72 R - 3.98G 2023-04-12 20:29 > old-main-CA72 N / 1.08M 2023-02-06 19:44 >=20 > (Most everything else is outside the BE's and so is shared > across the BE's.) >=20 > I updated to also have (whitespace details likely > not preserved in this note): >=20 > # git -C /usr/main-src/ diff = /usr/main-src/sys/contrib/openzfs/module/zfs/dnode.c > diff --git a/sys/contrib/openzfs/module/zfs/dnode.c = b/sys/contrib/openzfs/module/zfs/dnode.c > index 367bfaa80726..49a7f59c0da4 100644 > --- a/sys/contrib/openzfs/module/zfs/dnode.c > +++ b/sys/contrib/openzfs/module/zfs/dnode.c > @@ -1772,17 +1772,7 @@ dnode_is_dirty(dnode_t *dn) > { > mutex_enter(&dn->dn_mtx); > for (int i =3D 0; i < TXG_SIZE; i++) { > - list_t *list =3D &dn->dn_dirty_records[i]; > - for (dbuf_dirty_record_t *dr =3D list_head(list); > - dr !=3D NULL; dr =3D list_next(list, dr)) { > - if (dr->dr_dbuf =3D=3D NULL || > - (dr->dr_dbuf->db_blkid !=3D = DMU_BONUS_BLKID && > - dr->dr_dbuf->db_blkid !=3D = DMU_SPILL_BLKID)) { > - mutex_exit(&dn->dn_mtx); > - return (B_TRUE); > - } > - } > - if (dn->dn_free_ranges[i] !=3D NULL) { > + if (multilist_link_active(&dn->dn_dirty_link[i])) { > mutex_exit(&dn->dn_mtx); > return (B_TRUE); > } >=20 >=20 >=20 >=20 > I did my usual buildworld buildkernel sequence and then > one of my normal install sequences into main-CA72 to > update it to have the change, as well as the prior > material involved in my first experiment that I'd > reported on. >=20 > I cleared the content of the jail that I use for > temporary experiments, such as the prior testing that > got the 11 builder failures: >=20 > # poudriere pkgclean -jmain-CA72-bulk_a -A >=20 > I then rebooted using the updated main-CA72 BE. >=20 > Then I started the: >=20 > # poudriere bulk -jmain-CA72-bulk_a -w -f ~/origins/CA72-origins.txt > . . . > [00:00:37] Building 476 packages using up to 16 builders > [00:00:37] Hit CTRL+t at any time to see build progress and stats > [00:00:38] [01] [00:00:00] Builder starting > [00:00:40] [01] [00:00:02] Builder started > [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 >=20 > In the prior experiment it got: >=20 > 476 =3D 252 success + 11 failed + 213 skipped >=20 > and it reported the time for that as: 00:37:52. >=20 > A normal from-scratch build takes many hours (multiple > compiler toolchains and such) so my first report after > this point will be for one of: >=20 > A) It got to, say, 00:40:00 or beyond with, or without > failures. > vs. > B) It got failures and stopped before that. >=20 > . . . TIME GOES BY . . . >=20 > At about 00:40:00 the status was: >=20 > [00:40:00] [06] [00:00:00] Building x11/libXv | libXv-1.0.12,1 > load: 30.73 cmd: sh 1508 [nanslp] 2400.88r 6.69u 11.90s 0% 3960k > [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 235 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 241 Time: 00:40:01 > ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% > [15] 00:07:44 devel/py-lxml@py39 | py39-lxml-4.9.2 = stage 00:00:08 40.00 KiB 0% 0% > [01] 00:00:34 x11/libXxf86vm | libXxf86vm-1.1.4_3 = build-depends 00:00:03 56.00 KiB 2.3% 0% > [16] 00:01:59 x11-toolkits/libXt | libXt-1.2.1,1 = configure 00:00:52 40.00 KiB 0.3% 0% > [02] 00:01:40 devel/dbus | dbus-1.14.6,1 = configure 00:00:05 36.00 KiB 0.5% 0% > [03] 00:02:20 x11/libXrender | libXrender-0.9.10_2 = configure 00:01:27 40.00 KiB 0% 0% > [04] 00:00:44 graphics/libglvnd | libglvnd-1.6.0 = build-depends 00:00:13 52.00 KiB 20.3% 0.1% > [05] 00:01:39 x11/xprop | xprop-1.2.6 = configure 00:00:45 56.00 KiB 0.7% 0.2% > [06] 00:00:14 x11/libXv | libXv-1.0.12,1 = pkg-depends 00:00:03 52.00 KiB 3.6% 0% > [07] 00:01:57 x11/libXfixes | libXfixes-6.0.0 = configure 00:00:42 40.00 KiB 0.1% 0% > [08] 00:03:01 devel/glib20 | glib-2.76.1,2 = configure 00:01:26 40.00 KiB 4.3% 0.1% > [09] 00:01:21 shells/bash-completion | bash-completion-2.11_2,2 = configure 00:00:13 32.00 KiB 5.7% 0% > [10] 00:06:26 devel/qt5-buildtools | qt5-buildtools-5.15.8p157 = package 00:01:57 44.00 KiB 76.1% 0.1% > [11] 00:01:20 print/psutils | psutils-1.17_5 = stage 00:00:03 40.00 KiB 0.2% 0% > [12] 00:02:09 x11/libxkbfile | libxkbfile-1.1.0 = configure 00:01:22 44.00 KiB 0.1% 0% > [13] 00:08:54 devel/cmake-core | cmake-core-3.25.1 = build 00:01:43 36.00 KiB 694.9% 5.2% > [14] 00:01:20 x11/xcb-util-image | xcb-util-image-0.4.0_1 = configure 00:00:14 48.00 KiB 0% 0% > [00:40:14] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s > [00:40:22] [11] [00:01:28] Finished print/psutils | psutils-1.17_5: = Success >=20 > So no failures so far, 235 ports built and a bunch in process. >=20 > Note: maximum observed ("MaxObs") load averages so far > as shown below (for there being only 16 cores, i.e., > 16 Cortex-A72 threads, one per core): >=20 > load averages: 43.75, 34.32, 27.40 MaxObs: 45.03, 34.32, 27.40 >=20 > I'll report again if it gets a corruption failure. > Otherwise it will be many hours before it would > complete without such failures (multiple compiler > toolchain builds and such). >=20 To be explicit, since I've now seen your commit: My test does not include the change (whitespace details not preserved in this note) @@ -2650,9 +2641,7 @@ dnode_next_offset(dnode_t *dn, int flags, uint64_t = *offset, rw_enter(&dn->dn_struct_rwlock, RW_READER); if (dn->dn_phys->dn_nlevels =3D=3D 0) { - if (!(flags & DNODE_FIND_HOLE)) { - error =3D SET_ERROR(ESRCH); - } + error =3D SET_ERROR(ESRCH); goto out; } I make no claims about which way dnode_next_offset should be. I was only providing some independent evidence that might prove of some use to folks that understand the alternative code sequences. For reference, at about the 1 hr point for what I'm testing: [00:57:49] [01] [00:00:51] Finished sysutils/u-boot-tools | = u-boot-tools-2022.10: Success [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 306 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 170 Time: 00:59:49 ID TOTAL ORIGIN PKGNAME PHASE PHASE = TMPFS CPU% MEM% [06] 00:16:54 devel/binutils@native | binutils-2.40_2,1 package 00:04:22 = 56.00 KiB 100% 0.2% [09] 00:15:50 lang/ruby31 | ruby-3.1.3_2,1 package 00:00:28 = 40.00 KiB 100% 0.2% [13] 00:28:43 devel/cmake-core | cmake-core-3.25.1 package 00:14:20 = 36.00 KiB 100% 0.2% [01:00:03] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s [01:00:06] [06] [00:16:57] Finished devel/binutils@native | = binutils-2.40_2,1: Success =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 00:27:18 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzWGv2Lg6z45N5Y for ; Sun, 16 Apr 2023 00:27:35 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzWGs3Vchz4GB6 for ; Sun, 16 Apr 2023 00:27:33 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=V9Wawu94; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681604851; bh=HzKzeqFfEZU8GkA8xvtnOl+Uv6pcPpi7Dpawb1ZY6/s=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=V9Wawu94V2w88elmmP3KWSchj0NT5aD2yfNr3438kbrvvJM35Dm5gMD/FNMUrdD++l7r+9fc51GC/cApykBvMCVYNijesj1/0TAubDXJpHV71ImVuUC/WNhODtVOsyV0M2gOJ3PaUB4Ts455c7jyFbZcb9RX4tApMdh5e79e9VX11bWo4MaUjgHgIkonIZ9vWlIpPqMdFGCZcPQvHX1x8SCXHiuZueU+RgkRyQN7yuDMXE58BDLZWUo6Vm5y9QeDub/wP8hyTGzivcegMto5fxAqPCbwxsPlkvvDyi6FAsf537QPpqzbjdWwAs7WbVEFuuF6dXWi2w4/Y22WEGHgaQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681604851; bh=twjiqslks8iuRCPQicLsY6XWJxIYl553OvTHs6XR7Vv=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Ri996beiQXbnMwAcjqfWSrMvn/T36ylttSDvISodsgFK+p1LXB+6iXI6bJ0dD1MC9W2+AgeoECrcoNXvMdXHB3rdPadAPjZ+mHGWdBuqGlnuKXV8qMahPcvPil/sD/RyI1hcDyv5ZJfnRlOJLIaBJoh2D3hwPcqzyJyLE1RmxFD4HZ1qirZCCK6Ryym2UmQ/2BbWkPUmz/zn7B7ExMtmEteFInJn6/E4POBf+QLXHlqDyWgE86a40hjJuI1DKLuaNiLPW9x0ZwlKdAW9EVkKCzO/ICpfDeaxKcfiPo+lWSmi0Or9XpkF3sjlLGzudmnNxqUpGmGnDNtyxBUPnthv4Q== X-YMail-OSG: IR9D8j0VM1naINQovorhVqfaEawZ1wcMMC4YG22hLjsz6akLbotX1i7rSRRxM3f M4lGr60yVAffy9WA1t.vAbRzGBwhXvxD.hTdh4jNDd.hIaIGh1nt4eE9jZnVk3wGjFG5ICIGnscv MuqSSfosnSIhyLUsgI664yXzpczlqdlliivJAuzeGPy0vzrdieJuvxHe9AMZSd626O3cK.jU7wXh fdXwPeeKHk1PywxBcqeSL8q94j0TYzybLDbCQ.8lljfP68_Rf2qSi40P96iWmNJax_fGVUDKzSye udy_1JEgG9TDv7zPPN7ik3YG0eEjQIa_tYjz9j3fgKINZT0fq3QlhuIekj.9wZU6Rsz9i7b2NbZu 4PWGZZkTz._cGSt_C.lVCsGTTL70jFcvQMj4lOQDfSzRuS8qebNpwYdPyj8fHIhi8A1CXCLmu2Dt D6FAeovgHc__pxmrYv383f.agsP09wn4v.g8g0eP0_SjV.PjFmoMlYUc_wQ9ezEFlDRuEujYUgql Z.MU3BwTtevLLOl9QAmPxxVXI8aw6FUPtvtiMuiBwCQdoviMjmrBQR9T.tVMeYOahDF0s9jLkmdk 9oZ74UDlilHCrOxlWo06_Lox78N21pYwflswEaMk3baRwQ5RbkqiN7FKvvNkjE.iipytoTRriCty AMpbD0LrYmWPVRlE4sIXQY3nN6BhOQi5I64k6Crvacjt.u8UVGKpAO0pCxerXuP8.0bg5OwFJ2k_ AoxNhEhd3Vo0cs3DF7JqhmICzQI397o1ezxsVj5x5X1ztCQH3USF8HsjqxYAOseBCSO6NqKxfnPo GNGOMMSJFlKgV34ZK.M8UBXZpDo88YvdTCTkdXqt5nxgLFVMoDIB_Wf2u2FKNTeVV88HrZmZPEJH _YlDNmUoCuq575zWm3kr.D1wpZ.x5qFObVCZcc0e_9eOCTxr8LB3EZvW3uw1zWBbV9bvDK5ZAsgy wq3GecMg5Z4AYGX9Dpnfi9Iw4Qyl_mO8Sry4zQkELjaUV9t.knTX2xrMqvvmvUeX8HcYcvlXYftL tqWCg0Im7Rr4norL_RJLmojWVn_tny3VAIDilhIIoX8CMG6BB0osL3KjcVTNdqXDeP2hBVUlLCaQ ygFTH80iQMjNWkiqxFrmzk0h0Ez5HffUIA2RUb5i2rmjqwONKsaYXgdn4xQBd.Mps9ZZBtrdGfrw mT1rOpijQW6EuKwzNGjoCwtK2ggx0y.nW4S8e5b9iRp5Ndta2Cw.FKe.WxvIAL6oOxRxznYsdhz0 EqyxHt1l_WA2wOT7ODxYSO7dxqLiEnuqo_nVylRP4oS3CM6eqhyZlVuUMoo4m1FKDe1p4QTk6b0w F4AoQtGjspgjChZcfUikzbFPLmyyJYcRLQGz3FAIlGk8wdtGOYmS6iB4BuEP6_rJiOu7XiPYt8Jp vBfC7ZBpjYokw9NcxTFyzooZi9cR3F7bIktI_YoEBvvZM68W6AUvQytisHoioX.y3uLVxs4nDj0b zv4kZsPOLYSGGan55KZ_xSgx61ZWsMBmApCUTizX9HXSMXnC3sRujrWv8MH1ugPOIa3pSZ6nmFLi 6LEMu.vtMfQXEyJwMBNZCL9ATTHQE7UIuC2fEs5YreVLDpIa.3aSVRMJWkp3xbLhszpo3g83P2rq oHWqH7EwIkZ0yQrFLUtrUiJLCZRoNDPE2VKZxlP2DDAAqBQGOult9.mifvc4DiwwQThuMD0ffD8Z _Nli8JfULtz50UhSfhDFU_YfW3jvgwT0MrDNWUpkPBg2Aa5UArTyhdSeEv3cB6J.jt2qxMZzynLv w1p2MpFlE86L7C8O5L1YmxEQweZ7TZEaLvlSjXFwUkwni0yIbUWzTBMC7t1nX3j9IoqEHGUmkFVU pQr8d3YtnRmo.BGnGHnoPtC_.whSfPIY69U7Be5orQ3l4eWNODgbsssoRuoWeYq4FldDZCXETBCz domARSImoZemFH4i_A488K1YpMZRjZufEqoQV2Yw0S9iK_TQI0sSQSWbhGmFO2zDbPL8sRk0Kxv9 diojiexd_.TtAn1eUVEA2EyHILfHcHBiyHSfnDbv8aQOsah6JrxLTknbtIIqKQOahHIyYAhmw3Z0 9skm8Mcz3zbl3M.HVjC4W8VzLzPkhy04RbrlYZIPqdke_jyAGHySisY7vWdZfvb9IPknbwzQc_G1 YIlcSx27ByR3pi1S6AfyVzu7Sxc6s5ZdM8tya6jBEekOgxj8t5Xfn4h_BDK82NP8HhtJPCSicwyy 652P6yMfCXDcgtriIi2Fkx72ibqd_ZSBz78hOUCsv.XVsMo2Z3947zbs_4qdo.xySJSS32gSoVg8 PYqbvvlBgErE. X-Sonic-MF: X-Sonic-ID: bfd27b42-e30f-4a66-8da3-1c3a47d71eec Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 00:27:31 +0000 Received: by hermes--production-gq1-546798879c-qx24x (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 64eda90cb7af52a8c1e7dcfcdaaa63fe; Sun, 16 Apr 2023 00:27:29 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: <0C286158-8F0D-45D0-8D85-B521278E9518@yahoo.com> Date: Sat, 15 Apr 2023 17:27:18 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <62F8DD62-8E09-4A07-926B-5DE3DB399609@yahoo.com> <0C286158-8F0D-45D0-8D85-B521278E9518@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.32:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_SEVEN(0.00)[7]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.32:from] X-Rspamd-Queue-Id: 4PzWGs3Vchz4GB6 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 15, 2023, at 15:49, Mark Millard wrote: > . . . >>=20 >>=20 >> (Mostly written as I progressed but some material later >> inserted into/around previously written material.) >>=20 >> Summary: >>=20 >> As stands, it looks like reverting the dnode_is_dirty >> code is what fixes the corruptions that my type of >> test context produced via poudriere bulk activity . >>=20 >>=20 >> The details that lead to that summary . . . >>=20 >> Using my my build environment for updating my temporary, >> experimental context, an environment running a world and >> and kernel that predate the import: >>=20 >> # uname -apKU >> FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #90 = main-n261544-cee09bda03c8-dirty: Wed Mar 15 20:25:49 PDT 2023 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400082 1400082 >>=20 >> (Note the "nondbg": I normally run non-debug main builds, >> but with symbols not stripped.) >>=20 >> The kernel and world for this are what is in old-main-CA72: >>=20 >> # bectl list >> BE Active Mountpoint Space Created >> main-CA72 R - 3.98G 2023-04-12 20:29 >> old-main-CA72 N / 1.08M 2023-02-06 19:44 >>=20 >> (Most everything else is outside the BE's and so is shared >> across the BE's.) >>=20 >> I updated to also have (whitespace details likely >> not preserved in this note): >>=20 >> # git -C /usr/main-src/ diff = /usr/main-src/sys/contrib/openzfs/module/zfs/dnode.c >> diff --git a/sys/contrib/openzfs/module/zfs/dnode.c = b/sys/contrib/openzfs/module/zfs/dnode.c >> index 367bfaa80726..49a7f59c0da4 100644 >> --- a/sys/contrib/openzfs/module/zfs/dnode.c >> +++ b/sys/contrib/openzfs/module/zfs/dnode.c >> @@ -1772,17 +1772,7 @@ dnode_is_dirty(dnode_t *dn) >> { >> mutex_enter(&dn->dn_mtx); >> for (int i =3D 0; i < TXG_SIZE; i++) { >> - list_t *list =3D &dn->dn_dirty_records[i]; >> - for (dbuf_dirty_record_t *dr =3D list_head(list); >> - dr !=3D NULL; dr =3D list_next(list, dr)) { >> - if (dr->dr_dbuf =3D=3D NULL || >> - (dr->dr_dbuf->db_blkid !=3D = DMU_BONUS_BLKID && >> - dr->dr_dbuf->db_blkid !=3D = DMU_SPILL_BLKID)) { >> - mutex_exit(&dn->dn_mtx); >> - return (B_TRUE); >> - } >> - } >> - if (dn->dn_free_ranges[i] !=3D NULL) { >> + if (multilist_link_active(&dn->dn_dirty_link[i])) { >> mutex_exit(&dn->dn_mtx); >> return (B_TRUE); >> } >>=20 >>=20 >>=20 >>=20 >> I did my usual buildworld buildkernel sequence and then >> one of my normal install sequences into main-CA72 to >> update it to have the change, as well as the prior >> material involved in my first experiment that I'd >> reported on. >>=20 >> I cleared the content of the jail that I use for >> temporary experiments, such as the prior testing that >> got the 11 builder failures: >>=20 >> # poudriere pkgclean -jmain-CA72-bulk_a -A >>=20 >> I then rebooted using the updated main-CA72 BE. >>=20 >> Then I started the: >>=20 >> # poudriere bulk -jmain-CA72-bulk_a -w -f ~/origins/CA72-origins.txt >> . . . >> [00:00:37] Building 476 packages using up to 16 builders >> [00:00:37] Hit CTRL+t at any time to see build progress and stats >> [00:00:38] [01] [00:00:00] Builder starting >> [00:00:40] [01] [00:00:02] Builder started >> [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 >>=20 >> In the prior experiment it got: >>=20 >> 476 =3D 252 success + 11 failed + 213 skipped >>=20 >> and it reported the time for that as: 00:37:52. >>=20 >> A normal from-scratch build takes many hours (multiple >> compiler toolchains and such) so my first report after >> this point will be for one of: >>=20 >> A) It got to, say, 00:40:00 or beyond with, or without >> failures. >> vs. >> B) It got failures and stopped before that. >>=20 >> . . . TIME GOES BY . . . >>=20 >> At about 00:40:00 the status was: >>=20 >> [00:40:00] [06] [00:00:00] Building x11/libXv | libXv-1.0.12,1 >> load: 30.73 cmd: sh 1508 [nanslp] 2400.88r 6.69u 11.90s 0% 3960k >> [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 235 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 241 Time: 00:40:01 >> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >> [15] 00:07:44 devel/py-lxml@py39 | py39-lxml-4.9.2 = stage 00:00:08 40.00 KiB 0% 0% >> [01] 00:00:34 x11/libXxf86vm | libXxf86vm-1.1.4_3 = build-depends 00:00:03 56.00 KiB 2.3% 0% >> [16] 00:01:59 x11-toolkits/libXt | libXt-1.2.1,1 = configure 00:00:52 40.00 KiB 0.3% 0% >> [02] 00:01:40 devel/dbus | dbus-1.14.6,1 = configure 00:00:05 36.00 KiB 0.5% 0% >> [03] 00:02:20 x11/libXrender | libXrender-0.9.10_2 = configure 00:01:27 40.00 KiB 0% 0% >> [04] 00:00:44 graphics/libglvnd | libglvnd-1.6.0 = build-depends 00:00:13 52.00 KiB 20.3% 0.1% >> [05] 00:01:39 x11/xprop | xprop-1.2.6 = configure 00:00:45 56.00 KiB 0.7% 0.2% >> [06] 00:00:14 x11/libXv | libXv-1.0.12,1 = pkg-depends 00:00:03 52.00 KiB 3.6% 0% >> [07] 00:01:57 x11/libXfixes | libXfixes-6.0.0 = configure 00:00:42 40.00 KiB 0.1% 0% >> [08] 00:03:01 devel/glib20 | glib-2.76.1,2 = configure 00:01:26 40.00 KiB 4.3% 0.1% >> [09] 00:01:21 shells/bash-completion | bash-completion-2.11_2,2 = configure 00:00:13 32.00 KiB 5.7% 0% >> [10] 00:06:26 devel/qt5-buildtools | qt5-buildtools-5.15.8p157 = package 00:01:57 44.00 KiB 76.1% 0.1% >> [11] 00:01:20 print/psutils | psutils-1.17_5 = stage 00:00:03 40.00 KiB 0.2% 0% >> [12] 00:02:09 x11/libxkbfile | libxkbfile-1.1.0 = configure 00:01:22 44.00 KiB 0.1% 0% >> [13] 00:08:54 devel/cmake-core | cmake-core-3.25.1 = build 00:01:43 36.00 KiB 694.9% 5.2% >> [14] 00:01:20 x11/xcb-util-image | xcb-util-image-0.4.0_1 = configure 00:00:14 48.00 KiB 0% 0% >> [00:40:14] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >> [00:40:22] [11] [00:01:28] Finished print/psutils | psutils-1.17_5: = Success >>=20 >> So no failures so far, 235 ports built and a bunch in process. >>=20 >> Note: maximum observed ("MaxObs") load averages so far >> as shown below (for there being only 16 cores, i.e., >> 16 Cortex-A72 threads, one per core): >>=20 >> load averages: 43.75, 34.32, 27.40 MaxObs: 45.03, 34.32, 27.40 >>=20 >> I'll report again if it gets a corruption failure. >> Otherwise it will be many hours before it would >> complete without such failures (multiple compiler >> toolchain builds and such). >>=20 >=20 > To be explicit, since I've now seen your commit: My > test does not include the change (whitespace details > not preserved in this note) >=20 > @@ -2650,9 +2641,7 @@ dnode_next_offset(dnode_t *dn, int flags, = uint64_t *offset, > rw_enter(&dn->dn_struct_rwlock, RW_READER); >=20 > if (dn->dn_phys->dn_nlevels =3D=3D 0) { > - if (!(flags & DNODE_FIND_HOLE)) { > - error =3D SET_ERROR(ESRCH); > - } > + error =3D SET_ERROR(ESRCH); > goto out; > } >=20 >=20 > I make no claims about which way dnode_next_offset should be. > I was only providing some independent evidence that might > prove of some use to folks that understand the alternative > code sequences. >=20 >=20 > For reference, at about the 1 hr point for what I'm testing: >=20 > [00:57:49] [01] [00:00:51] Finished sysutils/u-boot-tools | = u-boot-tools-2022.10: Success > [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 306 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 170 Time: 00:59:49 > ID TOTAL ORIGIN PKGNAME PHASE PHASE = TMPFS CPU% MEM% > [06] 00:16:54 devel/binutils@native | binutils-2.40_2,1 package = 00:04:22 56.00 KiB 100% 0.2% > [09] 00:15:50 lang/ruby31 | ruby-3.1.3_2,1 package = 00:00:28 40.00 KiB 100% 0.2% > [13] 00:28:43 devel/cmake-core | cmake-core-3.25.1 package = 00:14:20 36.00 KiB 100% 0.2% > [01:00:03] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s > [01:00:06] [06] [00:16:57] Finished devel/binutils@native | = binutils-2.40_2,1: Success Just a status update as of lang/gcc12 finishing up: [02:18:39] [03] [01:18:30] Finished lang/gcc12 | gcc12-12.2.0_5: Success [02:19:11] [02] [00:12:27] Finished print/harfbuzz-icu | = harfbuzz-icu-7.1.0: Success load: 63.40 cmd: sh 59209 [runnable] 0.03r 0.00u 0.00s 0% 4012k [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 404 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 72 Time: 02:19:04 ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% [15] 01:15:01 devel/llvm16 | llvm16-16.0.0_1 = build 00:55:16 4.00 KiB 428.6% 4.4% [03] 01:19:09 lang/gcc12 | gcc12-12.2.0_5 = build_port_done 00:00:39 4.00 KiB 27.8% 0% [04] 00:51:27 lang/rust | rust-1.68.0 = build 00:41:24 8.00 KiB 480.6% 4.3% [06] 00:12:09 print/ghostscript9-agpl-base | = ghostscript9-agpl-base-9.56.1_8 build 00:07:19 36.00 KiB = 0.7% 0% [09] 01:15:01 devel/llvm15 | llvm15-15.0.7_1 = build 00:57:14 4.00 KiB 436.2% 5% [02:19:18] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s So 404 packages built and 0 failures to build. 5 builders active at the time. FYI at about the same time: load averages: 64.31, 66.32, 66.68 MaxObs: 99.41, 83.15, 74.18=20 I'll note that I use non-default options to avoid LTO based builds of compiler toolchains and such and this testing is of my normal style of builds, not of FreeBSD default options. I'll also note that, unlike my normal builds on this machine, this is with USE_TMPFS=3Ddata ( instead of USE_TMPFS=3Dall ). This leads to a lot more ZFS I/O, which seemed appropriate for the type of testing context involved. I've one other machine that I normally use USE_TMPFS=3Dall with but the rest normally use USE_TMPFS=3Ddata . This is because only the 2 machines have the RAM to make the USE_TMPFS=3Dall reasonable. The machine in use has 64 GiBytes of RAM and "Swap: 241664Mi Total" for the 16 hardware thread context, not that it has ever used much of the swap, except when some process turned into a massive-size file generator. (64-bit offset limited other than my noticing and killing such?) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 02:13:23 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzYdJ3F8vz45Y7G for ; Sun, 16 Apr 2023 02:13:40 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic312-25.consmr.mail.gq1.yahoo.com (sonic312-25.consmr.mail.gq1.yahoo.com [98.137.69.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzYdJ0lWsz3rjH for ; Sun, 16 Apr 2023 02:13:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681611217; bh=aqUUVzNibHcQfdPF5PL/Lx3W2LKzoI/r+p4fQ5FsHz8=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=eg6Mym17B/oWjDbFwzyzbUNUSjv3WiXzQBQWBKmNQVEeiVZUTuMygMSN93y0JpRApP4nwRKj9Eru91VUAoDoOPF5YjYZDaIa2Y41mzQ0TNcjmLpCCqZj9tLAXKVdRL/S2F2VdksSRN/YObNphGT/HoTWd1HtVGfaGAOwGDkRFEX8OZfvWIwZRRcbePsiNLTTItTxFQpXW0X7z3DHpR1lsUO2oaAK+hQH/FkS7YDi4w8X9qkzsB++0Xi7GVJ/ysOWL+8VPyvvVTwJjdDNbnQK9ss9uf3cU7wh7JPQIsqNFJRwyu4+5x7x5z4YWWAKkTyLfck0aX+rsAADewfa3jq5JQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681611217; bh=OS2A8daYggY+vZ+wOdMXnKwVuHqNPyz8+8cW1gZJ6qH=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=O/8UceRTPFa/qkTerL7f7+CGtCZijQwrthf8xvmt7wRD1xeIME0HM7W6ctb9x0v7FK5kzLTQqePyIhQ4zsiaya8RZ6Mv9afL+ZFkUGeU7p3ZrWboOcevgwZwWk5zKwYtFl2vEaAM0oS17sVvGVGvxRA1Y5xqvIXzRXk+HSkcOc8qLvaO7rqB250ypP1L49Ko7k0hw9wtXbX8VCvCAQVkI4JzuqC/f1PwfIbjYSlW5+77+5CYy5UjPfzPvn1TLU8y+/s6nUJeADpG+0q4xvMhRc6LIrM1mFJwV+stvVebJ1jDpNp6wmf/JteKaloDwev+2I7dbCuGo2ER9hqy7ALPeQ== X-YMail-OSG: zg32g2sVM1mY2sI1HZRz2j9cpUQimJ_Kz0_rNKvofBS4DyiJLeK2QPtJcTwj_ml ZAWf6NYGzLxKkY2jHOnup_BPWjG7mF047NxNJiCr_.1Jp_7hqAjcS.kHWomR..OODrot07k45c3Q CXI85DCZF_sHOMQSw94YOTxJpXzjrgYpBbMgQpLYOPpHegxq4Oc8wY.Dp7eb4VZhgxw6qNTVNeNV j39lovYBnNoUkT8lCdjd0MPdlao0HgyYwLxJQ9A2g5hB7.p3NVvv7j1cuTC_6fjCbFY1plLlTOTv xITWVUkPHtb8ccDOHe3ioL2Zmo_DKZ0AG515jKvJYYK3Mj_FCWhdoLfiwgv8UXeupyZi7js8oCet 1Fg5zIW3kPGeupTCRB14F2HgZZK6c9OqRXMQvLh6uaEWaGeQnBveWSEcq4Uzb6KxO9jqPip8L8oz zd3v50uNtNFc.kZhlz1v2IU96BKcfDa_DevJmls38G7WEFM2uHJNuI5gSIy7Q0VlHb_5R7cXjjRZ JDIhV2_l7sy.9AQrkXI6eTNOfvJwtIsfkAlVxBNKjGK8ZcfV9R0u872sx6jTUhtrupJA7iB9EYd0 H8WaE7C51qY1UOab7fbi.SrvZl5ZMLHIDifzp.0w2oXMQd_kzadFbx5i2XB6k2w3sG8cKnorD.f7 MNaW90NTCeZjtuAi_hE6JUtTUsy9mhnydQVfLUXmPlPurWPIBU0TXDeIWawlMENUH3zRre8NCiRk nsoQbMMvzxAdGsFhWZixpSD0xulyUnHPJmV4m4J.jr4F6aEQfPWgs0oBIrvfGZS.Y2J8dXtCbSpT PS6Yrt7acG1Px8msIhS_RJvv6ZlHWP0D9Ag8.V8lRvJhgtG88PZGXf9DYe5Xmd3O7zE0b7oazipZ fZes8N7ey30vQOR3Y1IwSLTPrzghgxUWG2XS833Qy3YqT6uEe16R12NwW5lE6pgUW9fGi2gxmkpW etqQtrQlXGPfXcLFXkXsBRSpzAK72axdjXLbkppRPNK.t7KjFZSzS6BHPL.BVpR5omeJHHIKMfO9 .Pz4AanME0P4cXUEwR9FU4HQQfrBwGWUxkw624NDdlbWTQvbvN.J5ZW8FSizezDOZ7aIAKrdQ4v. 5LqTf9PryMjKa5dnrMg7AGIUwgV.ry5kTnSq.Lyz67RJRa0_fEPL5ERlrVaLTBWKQ1espMLOkUok Ygn.Xafpa0iGmSwO9yRP1h7EferFrVRDbE4ZTv7qa_tDoBBbwGbn8ZMmB5bwz7DGQU4K2foap0g8 dZX433YzA1tG42PoASJNpOL7_dUbRQEDHyDRBRGysJQwcg91dtdfOn8kv5JCGZM71VFOU159sF4g PASxlY1orpPTVFdpdSKegpYW3aR0djvON0iJxtr0Bpgmhlmf90WJEGaL4QFLMh0.VKWBX8ZVDvAi 9tEnh.QS6He888yppFuzobV9GQiqNhwhnRZZa3bW07YRmp1p7R7lJqr80bU.C5_Gr3LSc7yqu2Cj .7mBTD1Axj4IBoClwLITA90lG8f50cGvPG2O0tMjwg0EJe5xExuBG3C26afkIFWA1hBNQRBtQqxt MPaxfZfXYW9NStepIuPX5mU74scAUqJes7t7JFcy9YOGcpSCIQC1Pl79TIHevLCGGAEvEKtCsLh9 AAQ8aISd3u7ea6bVyBaFrLmx3AtECLqtRzH37iWFDH19GSRbjBEX8cyVingR_4ALfJeC27cy9Afx 4XcxwWt0P.9xTFvxiboYYVexkhbwJKH8IsFjabHyNr5PCCL7BiJwS9IaSzNK9zWs8uG1RglAVWoj n88bquH7tt12_6g8XB2pNqPVMZKJ5ztylm3jJDm91gM16bFKZjo0bhtMTT1e2kb3HC1N8czVWFmS ZjARJO9GmAKetGAOhvQmC86dSukTt2qcQjSSqZbjGZ1aiPnewMgDw9D.3hbF1xAr_Jyz74uGTk3Y ZubOBOpkzoEn6ZgjQeJDODIR70epRgrKrGiS4HMk0bVXMZFkpgeXkyqOgdKJ56ZyXhAFVWypRjc1 o.CclpZ0PsxNbF2kXlr7ML6su1KUjoTL6qClh1u58XJ1eBOt1lXZcSxWTIBIl3ZnhbZ5SK4.wScz dBVdsQx6iTZ5GtVmWHUdc5GKN0efOoF1fjAYGogoxXCUktG6qdGxWvG91.sxKzqVf.GOgkjf5pLZ E0RkYpjNI9VNJ37nDbrcT8Jfyrc_W6jzZUNI84E2nHMhOz6Ne0NedkOL6h.8MkcenpofA7GTYvoy zEsCtOdF1qVKTS6UGxCrYG8XvlJECARztV0tCALSancCXkIRUz58LDemYNHWNmFkYKBXaikBD8ir ZrfM87Q-- X-Sonic-MF: X-Sonic-ID: d0c37849-95c5-4e2a-b920-7d6d78dfd58f Received: from sonic.gate.mail.ne1.yahoo.com by sonic312.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 02:13:37 +0000 Received: by hermes--production-bf1-5f9df5c5c4-dxcv9 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID bd51cb664787f9ff0b56f6998654247d; Sun, 16 Apr 2023 02:13:35 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: Date: Sat, 15 Apr 2023 19:13:23 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: 7bit Message-Id: <3BB7D5DD-99A6-4D27-BBB6-4CD3294EEDF9@yahoo.com> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4PzYdJ0lWsz3rjH X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N A general question is all for this message. So far no commit to FeeeBSD's main seems to be analogous to the content of: https://github.com/openzfs/zfs/pull/14739/files After my existing poudriere bulk test finishes, should I avoid having the content of that change in place for future testing? Vs.: Should I keep using the content of that change? (The question is prompted by the 2 recent commits that I will update my test environment to be using, in part by fetching and updating to a new head, avoiding the "no dnode_next_offset change" status that my existing test has.) === Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 04:31:17 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzchS6Z8tz45lqF for ; Sun, 16 Apr 2023 04:31:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic306-21.consmr.mail.gq1.yahoo.com (sonic306-21.consmr.mail.gq1.yahoo.com [98.137.68.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzchR11z3z3HGZ for ; Sun, 16 Apr 2023 04:31:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=SRXfsojF; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681619492; bh=VH1SxqVMbw7yCBqh6QXUIPyvCA/NMrAPs7/ILT4EFO4=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=SRXfsojFuocKxdwDGTv0wbyQDyOUs7ACzIctNF8YuULR1qq80EG4Sg6t2x3/ARpwrQwSbXlZEzYctQ8srNQI7QoGOXzYtCrZiOPK1oOsCkPRiXM9P01Q+VD2RHAFq+MtW9mT+COYHrXKf8X+l/g+iNXGidwMv6M9hcSmLYWwGSSuKiBmoNMzmwCP1/uPbWnO/H8XVQ5/gmkbB1f16GpgqBdy3oo+Sca3tS5eDe91NC9ADsxirrClyHvsxi0rmoTUUkdiz+sgQjtc08UMqNDxcSTeSiS4tkkGif50mDtYWxnMJX6ME1K/PL/W4Eoo5+LffKaM1iRVKiVvlK3YFc7bqw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681619492; bh=Iih1oogeKZMX/ApEzAXh1TwWmpA5x9S4jw2g3olR9JN=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=BzujGknOJvTPT1Nu/4WjbGiS+zCkXJBtzsNx9xgRKTjvADrxBsVtBdNQngyNgaVJA+6089qUKDO7s8FCpLfpPsYwS5EbLcbkFLRzEDXoo67mQvjHHguatjNnPpc6/xawUdFGkUygAtG7EyCHRi2/XdhrjqHdtz22+daXUqlKm5wVzJiVEOJx99TAdnnFSrcmqpEgPOH1PqyqMluMAHez1eT28sDSNYn48mj2T3OFNCdSFVk7qdZ7JHN+/5pv4D0NZO7L9eVEgolg7jx5yx/l59FdybGcne+dMzLlvDznPYoRrb/3vLxiYffqo7grFzkB3lhwK1p+gAlcBwRi3Pye7A== X-YMail-OSG: Au586yAVM1lfriY_lsCTmQE53ArMF8ujJtwmcpjUHnFsKMD8zUpp7upaY1D9l9r cb.l_UDQ.k4Zc8Xl1RKJcURPT50joPiC.3WI28wNPNILyd5PM.1KnI.nJKD9Fq64A9Y7.D1lbaQQ .HHnxRnsqsY7Aq63_XdUqbQRS5nmV0Yx7IbMk7E8wUqMVS6Tkimt0xdO4UOT27LXBcISVSt7kkqX _to4U5Vs5t1L9JXUrJdIPpHbqbnO2dvff5WXlSfBijv3qo0JSaDQBnl4NbgjvOzj7gmzHsNr0zVe CgAYnUcmp7S9M3QJ3P.SFm6r7B067vGAUEO69W7TCn6Gv_yZTyqo3nzRjg.hI3SpByMiuCjcrnf_ BUSNMxRZiqMHPHGjHWUbrmcUACoHOHWlDvpPzhfRKjzNvGpZLwOoF4KIFIA2cdwnD0eTu554Y_PK lNTtjKz.tFDezbkAc2gyD8X9vyO2Qt.S8pmfStMrALC3ABC2ezz3Zh6aQE_Eihdu7A4s_dE4qsZO 4GukHVA_EVz3qBzuO2RRxZdJDobAwjPAmVb0xZooL63015KSwYuRSItEJIoUmFrfcEm5QxdYhBR5 hsK_MA3Zcc_8sARrx1j7NlHkGNPMv6TvC92c3crB2CnxxIhDKAXvvqmWjDvRSCj4KgOKC7xZec0J p..QH13pDNgldF1eGzxn1WiZbj.1Q6e2Pr_veMUDPvF3Ct6uRMy8Axv22utCi6YciH78Xbjqkqs_ gtaqBAAS3ccrRSyPIRW5y0qv9hZTeXHP2LfvxN1Tpzesl8j_Mmtw6.5qd_EJp3SDi4uypL32ANk3 ALhoMYgBS1t8hTSdd.I6E1mdYpAbdla1n_yxtxP1WkN0UEdNUW..q4.lxdmtrJvFIxyxna3Jyfbx GTLch42pmaX1zO783Bz9qK0pPIqlhmY9MozW.qS34ZyTzMntItUnsWjOfJYjTEAbszPnmIDqNjzq 32E2vJMWZUpTMw9fWc5IqPjfGK.SV2ByZr4cs9GD6qQvnlijYKB0zyj.s_Z8yk56OHI_NXmojJCM iRm.eIWAzgUwD60n3BySWeNrQurimBz9Rh8yEsu7zOUyTXL4eRG.K2zJ25W.fPLunKJeAGOhVZFr QTRkmnDxOOfsFUR95IszsnRBVFBjVpOV9NpgYBvihR0ehpiv.tur2G8xxfWDFO170ZJNN.l1WzqX .zVM11yJtBwSWSSod7W04V46QpiZ1mDZayML1ayOYSqYq.aSzhpWc40CTQSYRtwg18SK_fwWlkBy Hy5ANxPtMhZ75XYon0Ehsz5CWucppTJvJuplm31URIBCO0dSqEIddwZXlBtHGf0Ow6cwoSNUhkjF AWq.64avAGhOI8bw_0DAPx4TUx62KUuuNZnyQlvC.G.r1WlJJYsfHO7p5U9Ycu.qOPLIgVYq28wM WCElSsB0KJONMNzByrphaSy60l5UY7k31hSVJh3OS8eyInf05q5SSmBYp.SXnDHLqbuVamJjmQuF sc3spTvh0T4f70gt1F85wZvvKlndzio8YPzpp4c47bLY4yPe446fVBxXaNS05UTHyyRhyEzh3pyl StUh0vOr8dZ5Iu6ctRBm91Fa1eP..o.xoASislZg0u8ylmj8aJjwgeQLAejI4CxII6mC0lIRBbFJ 1XQZUYEMpIzE8CytfZ9noi606lqIrDtwt5gJBW8__AtcPoRyij1lWf72sFRkGhT1KAk7tT2GZ6zQ .4QGamP1olW7wdsLQs2GmnCllXzT7fUix.nfi0kYC7f2EOq2Rny31yFFb9Luewb5ySLS13ETLssP S1buQeZg3v9bjT5hLad_XIVc28zAKAI.9oSf7A4n4CW4HsjCYC_jCi7aAFQrpgWTXp1WFhRrbCe_ ah7wyp_SG.OwRYGBMaMJXSOdls3EIB3ZphR2HT.cBM.IOdkbNttJjV0Qo5am_fvMtTjdynSPJlsE z8g1VZia09LHJhaK0WDYA7cHMKixql0BAdja4OoyNnrCsKWmCYo.BdP6aTG_3wrnplc38qfeL_A3 KPjYsYM4yIl3ze48P1MpMAs6cyw_oAE0lX7C6E5FyWUQPIv22ur8Izr__gelozypqfSQviPf3218 yDydx0atIKg5U75e9sedHW6IkgqZxrLTu2mKRvILBUjkSJeEN3WYH8hZHewEb9cTjWvVFhIEcois o7t5W8LQ.fc_J3G_Qczs4OQjTZuYw2K183BcBK1w52ZFlAv1mhUcGuN.imyuQZ38TLIXGBZDA0Xo L6Wodn.RQCYWG9YaaEa.wIBgQ8YQt6zwKdPIX03twwjw7IMkRrRbBh3R_rXBWVP1J1iJ5VlaQN7I uxQpUccui X-Sonic-MF: X-Sonic-ID: b37f82d6-4907-4daf-9a12-7da44e040de0 Received: from sonic.gate.mail.ne1.yahoo.com by sonic306.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 04:31:32 +0000 Received: by hermes--production-ne1-7dbd98dd99-nn8pc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID b6dbd0f6bf8a8f721e1c68e10956cbb6; Sun, 16 Apr 2023 04:31:29 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: Date: Sat, 15 Apr 2023 21:31:17 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <78E9D9C8-6679-4E2F-87DA-43B5B0224D9C@yahoo.com> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <62F8DD62-8E09-4A07-926B-5DE3DB399609@yahoo.com> <0C286158-8F0D-45D0-8D85-B521278E9518@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.49 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.84:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_SEVEN(0.00)[7]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.84:from] X-Rspamd-Queue-Id: 4PzchR11z3z3HGZ X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 15, 2023, at 17:27, Mark Millard wrote: > On Apr 15, 2023, at 15:49, Mark Millard wrote: >=20 >> . . . >>>=20 >>>=20 >>> (Mostly written as I progressed but some material later >>> inserted into/around previously written material.) >>>=20 >>> Summary: >>>=20 >>> As stands, it looks like reverting the dnode_is_dirty >>> code is what fixes the corruptions that my type of >>> test context produced via poudriere bulk activity . >>>=20 >>>=20 >>> The details that lead to that summary . . . >>>=20 >>> Using my my build environment for updating my temporary, >>> experimental context, an environment running a world and >>> and kernel that predate the import: >>>=20 >>> # uname -apKU >>> FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #90 = main-n261544-cee09bda03c8-dirty: Wed Mar 15 20:25:49 PDT 2023 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400082 1400082 >>>=20 >>> (Note the "nondbg": I normally run non-debug main builds, >>> but with symbols not stripped.) >>>=20 >>> The kernel and world for this are what is in old-main-CA72: >>>=20 >>> # bectl list >>> BE Active Mountpoint Space Created >>> main-CA72 R - 3.98G 2023-04-12 20:29 >>> old-main-CA72 N / 1.08M 2023-02-06 19:44 >>>=20 >>> (Most everything else is outside the BE's and so is shared >>> across the BE's.) >>>=20 >>> I updated to also have (whitespace details likely >>> not preserved in this note): >>>=20 >>> # git -C /usr/main-src/ diff = /usr/main-src/sys/contrib/openzfs/module/zfs/dnode.c >>> diff --git a/sys/contrib/openzfs/module/zfs/dnode.c = b/sys/contrib/openzfs/module/zfs/dnode.c >>> index 367bfaa80726..49a7f59c0da4 100644 >>> --- a/sys/contrib/openzfs/module/zfs/dnode.c >>> +++ b/sys/contrib/openzfs/module/zfs/dnode.c >>> @@ -1772,17 +1772,7 @@ dnode_is_dirty(dnode_t *dn) >>> { >>> mutex_enter(&dn->dn_mtx); >>> for (int i =3D 0; i < TXG_SIZE; i++) { >>> - list_t *list =3D &dn->dn_dirty_records[i]; >>> - for (dbuf_dirty_record_t *dr =3D list_head(list); >>> - dr !=3D NULL; dr =3D list_next(list, dr)) { >>> - if (dr->dr_dbuf =3D=3D NULL || >>> - (dr->dr_dbuf->db_blkid !=3D = DMU_BONUS_BLKID && >>> - dr->dr_dbuf->db_blkid !=3D = DMU_SPILL_BLKID)) { >>> - mutex_exit(&dn->dn_mtx); >>> - return (B_TRUE); >>> - } >>> - } >>> - if (dn->dn_free_ranges[i] !=3D NULL) { >>> + if (multilist_link_active(&dn->dn_dirty_link[i])) { >>> mutex_exit(&dn->dn_mtx); >>> return (B_TRUE); >>> } >>>=20 >>>=20 >>>=20 >>>=20 >>> I did my usual buildworld buildkernel sequence and then >>> one of my normal install sequences into main-CA72 to >>> update it to have the change, as well as the prior >>> material involved in my first experiment that I'd >>> reported on. >>>=20 >>> I cleared the content of the jail that I use for >>> temporary experiments, such as the prior testing that >>> got the 11 builder failures: >>>=20 >>> # poudriere pkgclean -jmain-CA72-bulk_a -A >>>=20 >>> I then rebooted using the updated main-CA72 BE. >>>=20 >>> Then I started the: >>>=20 >>> # poudriere bulk -jmain-CA72-bulk_a -w -f ~/origins/CA72-origins.txt >>> . . . >>> [00:00:37] Building 476 packages using up to 16 builders >>> [00:00:37] Hit CTRL+t at any time to see build progress and stats >>> [00:00:38] [01] [00:00:00] Builder starting >>> [00:00:40] [01] [00:00:02] Builder started >>> [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 >>>=20 >>> In the prior experiment it got: >>>=20 >>> 476 =3D 252 success + 11 failed + 213 skipped >>>=20 >>> and it reported the time for that as: 00:37:52. >>>=20 >>> A normal from-scratch build takes many hours (multiple >>> compiler toolchains and such) so my first report after >>> this point will be for one of: >>>=20 >>> A) It got to, say, 00:40:00 or beyond with, or without >>> failures. >>> vs. >>> B) It got failures and stopped before that. >>>=20 >>> . . . TIME GOES BY . . . >>>=20 >>> At about 00:40:00 the status was: >>>=20 >>> [00:40:00] [06] [00:00:00] Building x11/libXv | libXv-1.0.12,1 >>> load: 30.73 cmd: sh 1508 [nanslp] 2400.88r 6.69u 11.90s 0% 3960k >>> [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 235 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 241 Time: 00:40:01 >>> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >>> [15] 00:07:44 devel/py-lxml@py39 | py39-lxml-4.9.2 = stage 00:00:08 40.00 KiB 0% 0% >>> [01] 00:00:34 x11/libXxf86vm | libXxf86vm-1.1.4_3 = build-depends 00:00:03 56.00 KiB 2.3% 0% >>> [16] 00:01:59 x11-toolkits/libXt | libXt-1.2.1,1 = configure 00:00:52 40.00 KiB 0.3% 0% >>> [02] 00:01:40 devel/dbus | dbus-1.14.6,1 = configure 00:00:05 36.00 KiB 0.5% 0% >>> [03] 00:02:20 x11/libXrender | libXrender-0.9.10_2 = configure 00:01:27 40.00 KiB 0% 0% >>> [04] 00:00:44 graphics/libglvnd | libglvnd-1.6.0 = build-depends 00:00:13 52.00 KiB 20.3% 0.1% >>> [05] 00:01:39 x11/xprop | xprop-1.2.6 = configure 00:00:45 56.00 KiB 0.7% 0.2% >>> [06] 00:00:14 x11/libXv | libXv-1.0.12,1 = pkg-depends 00:00:03 52.00 KiB 3.6% 0% >>> [07] 00:01:57 x11/libXfixes | libXfixes-6.0.0 = configure 00:00:42 40.00 KiB 0.1% 0% >>> [08] 00:03:01 devel/glib20 | glib-2.76.1,2 = configure 00:01:26 40.00 KiB 4.3% 0.1% >>> [09] 00:01:21 shells/bash-completion | bash-completion-2.11_2,2 = configure 00:00:13 32.00 KiB 5.7% 0% >>> [10] 00:06:26 devel/qt5-buildtools | qt5-buildtools-5.15.8p157 = package 00:01:57 44.00 KiB 76.1% 0.1% >>> [11] 00:01:20 print/psutils | psutils-1.17_5 = stage 00:00:03 40.00 KiB 0.2% 0% >>> [12] 00:02:09 x11/libxkbfile | libxkbfile-1.1.0 = configure 00:01:22 44.00 KiB 0.1% 0% >>> [13] 00:08:54 devel/cmake-core | cmake-core-3.25.1 = build 00:01:43 36.00 KiB 694.9% 5.2% >>> [14] 00:01:20 x11/xcb-util-image | xcb-util-image-0.4.0_1 = configure 00:00:14 48.00 KiB 0% 0% >>> [00:40:14] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >>> [00:40:22] [11] [00:01:28] Finished print/psutils | psutils-1.17_5: = Success >>>=20 >>> So no failures so far, 235 ports built and a bunch in process. >>>=20 >>> Note: maximum observed ("MaxObs") load averages so far >>> as shown below (for there being only 16 cores, i.e., >>> 16 Cortex-A72 threads, one per core): >>>=20 >>> load averages: 43.75, 34.32, 27.40 MaxObs: 45.03, 34.32, 27.40 >>>=20 >>> I'll report again if it gets a corruption failure. >>> Otherwise it will be many hours before it would >>> complete without such failures (multiple compiler >>> toolchain builds and such). >>>=20 >>=20 >> To be explicit, since I've now seen your commit: My >> test does not include the change (whitespace details >> not preserved in this note) >>=20 >> @@ -2650,9 +2641,7 @@ dnode_next_offset(dnode_t *dn, int flags, = uint64_t *offset, >> rw_enter(&dn->dn_struct_rwlock, RW_READER); >>=20 >> if (dn->dn_phys->dn_nlevels =3D=3D 0) { >> - if (!(flags & DNODE_FIND_HOLE)) { >> - error =3D SET_ERROR(ESRCH); >> - } >> + error =3D SET_ERROR(ESRCH); >> goto out; >> } >>=20 >>=20 >> I make no claims about which way dnode_next_offset should be. >> I was only providing some independent evidence that might >> prove of some use to folks that understand the alternative >> code sequences. >>=20 >>=20 >> For reference, at about the 1 hr point for what I'm testing: >>=20 >> [00:57:49] [01] [00:00:51] Finished sysutils/u-boot-tools | = u-boot-tools-2022.10: Success >> [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 306 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 170 Time: 00:59:49 >> ID TOTAL ORIGIN PKGNAME PHASE PHASE = TMPFS CPU% MEM% >> [06] 00:16:54 devel/binutils@native | binutils-2.40_2,1 package = 00:04:22 56.00 KiB 100% 0.2% >> [09] 00:15:50 lang/ruby31 | ruby-3.1.3_2,1 package = 00:00:28 40.00 KiB 100% 0.2% >> [13] 00:28:43 devel/cmake-core | cmake-core-3.25.1 package = 00:14:20 36.00 KiB 100% 0.2% >> [01:00:03] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >> [01:00:06] [06] [00:16:57] Finished devel/binutils@native | = binutils-2.40_2,1: Success >=20 > Just a status update as of lang/gcc12 finishing up: >=20 > [02:18:39] [03] [01:18:30] Finished lang/gcc12 | gcc12-12.2.0_5: = Success > [02:19:11] [02] [00:12:27] Finished print/harfbuzz-icu | = harfbuzz-icu-7.1.0: Success > load: 63.40 cmd: sh 59209 [runnable] 0.03r 0.00u 0.00s 0% 4012k > [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 404 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 72 Time: 02:19:04 > ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% > [15] 01:15:01 devel/llvm16 | llvm16-16.0.0_1 = build 00:55:16 4.00 KiB 428.6% 4.4% > [03] 01:19:09 lang/gcc12 | gcc12-12.2.0_5 = build_port_done 00:00:39 4.00 KiB 27.8% 0% > [04] 00:51:27 lang/rust | rust-1.68.0 = build 00:41:24 8.00 KiB 480.6% 4.3% > [06] 00:12:09 print/ghostscript9-agpl-base | = ghostscript9-agpl-base-9.56.1_8 build 00:07:19 36.00 KiB = 0.7% 0% > [09] 01:15:01 devel/llvm15 | llvm15-15.0.7_1 = build 00:57:14 4.00 KiB 436.2% 5% > [02:19:18] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >=20 > So 404 packages built and 0 failures to build. 5 builders active > at the time. >=20 > FYI at about the same time: > load averages: 64.31, 66.32, 66.68 MaxObs: 99.41, 83.15, 74.18=20= >=20 > I'll note that I use non-default options to avoid LTO based builds > of compiler toolchains and such and this testing is of my normal > style of builds, not of FreeBSD default options. >=20 > I'll also note that, unlike my normal builds on this machine, this > is with USE_TMPFS=3Ddata ( instead of USE_TMPFS=3Dall ). This leads to > a lot more ZFS I/O, which seemed appropriate for the type of > testing context involved. I've one other machine that I normally > use USE_TMPFS=3Dall with but the rest normally use USE_TMPFS=3Ddata . > This is because only the 2 machines have the RAM to make the > USE_TMPFS=3Dall reasonable. The machine in use has 64 GiBytes > of RAM and "Swap: 241664Mi Total" for the 16 hardware thread > context, not that it has ever used much of the swap, except when > some process turned into a massive-size file generator. (64-bit > offset limited other than my noticing and killing such?) >=20 The bjam command used for stage for devel/boost-libs is a long running, single process, single threaded context. So far over 50 CPU minutes worth for stage. [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 463 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 13 Time: 06:40:59 ID TOTAL ORIGIN PKGNAME PHASE PHASE TMPFS = CPU% MEM% [02] 01:06:33 devel/boost-libs | boost-libs-1.81.0 stage 00:55:57 16.00 = KiB 100% 0.5% [06:41:12] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s But, so far: 463 ports built, 0 failures. It may be a while before the last 12 get a chance, as they wait for devel/boost-libs to complete. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 06:59:29 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzgzN4D0lz44m4p for ; Sun, 16 Apr 2023 06:59:44 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzgzM2hHwz45Fy for ; Sun, 16 Apr 2023 06:59:43 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=VSOnYC6H; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.68.32 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681628381; bh=aVtza9sJl6rMjwUxR2OG1rFhWxGp4ZNhIKao2LVUJbY=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=VSOnYC6He077y0WfWARbEj8f3E1Va2CtdfD1ESRRCG7+ZNMHh3/UY+FLtHrHWu4CSzZtHjcjFJWzFqLQNvAZltdJwCDeJIOy0kud4abmIlWrkNC/ik+NINugyQICGArtemwNEQxV5f/fyJ24k30ZKSnt4mNAr2CljMm3OO4NAAejOC16mpygg3bp0pGvD3OuB3S9XImu3UMWXZ9MIFPLbMcOXnZtPIoeIJc7lv4R4Uua6kWjKc4tjGKy1TyxKKUuCKgMd0VclXbHh7jec/g6yMzD/EAJCZ7Y+tUcatLjMeNnSnwg7o4bbxX1U5LnjbDnCTC4Q7yas49PlZogng5x7A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681628381; bh=14mJWYiIdtppQqW/Ed77SlUNa9lbWUjfHhS8mOfs2dj=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=mgeuiMxiu5bso3S40qj6dvJD5SRJZmJf9AoHRBDbLjCW9E/uF7437j8/1pEeytntX5jebSq7b5Q9nfFy60SN8G8LLp6rSKbPH44fSowbbp/akKPRGC39h7FtrNp+BqLIYwcNF5vI30DvgFTU31y8v+Mqjx67uX8ukBDwW0Ny2MruWIYKf+wuGpCrOkamta1pF2wjDD7geC6x7nZY/3VxuOlvLDB9bHVN7nMzFnebhaLMYr4q872fsCUQqGAmGeq/ldh7s2QDOsRET37f/X2Oemui991OZt0NrcQTtR6PvPgi9G/WAwAudniL/Zuf3Ca5MyzWaBQzgjnFvy51HauKEw== X-YMail-OSG: ARhJU8YVM1mUPoquyxTIzEKfx0EWqEiQP7fFO9eIQ6s2jOZWRqy5XuBAcfBauYc Mon4L8B6vviKRzRlHCmhu9vYqN0Q65E50TpbcuAE9B5aG6TIH08U1DsN5u0l8pLxW_E955277n4Y CC5.Fvu7PC4TURspbc14cgZbVHgm.kpXU9xlEB9CO7t61A_.8rdFbJpo8zcK6ALL54rrebsRm3cI u2_c9PPOl2Vp9Q9oyLtmTYy8y6uINmOTvS0muk62dLzYutKWI.8v5pNNm.lrUITfFDMsTdZ.QNu6 n0Y6TtAIw2YeuKose7nRaNyEVi27lIW6IC9PGfo0h8F_4oL4xG6G2UgRZ1VvrN2YZy9Z1YbUls.G r23_BTjKr_kauwfu4CQb5rXwN_almBTan2LyYeS8OhF7qqQTBcHkClnQrMHPYka0SAKkBlDLZRzx K2H8ZlJs20DqOXq6Uns0ezQrBowAJX6IJAMBYR0yssB1LWL8.5Fpq.sC93oUJm_XP_GZtknlQlYp UzvkPbe5tFM0Dfe.SfxVOKxnVgS42k5FVouiOVmhQn7Hy4BgbfjSFQK7n3TTRikfdzmPLGih2r7B MUXT0QnncOd6BAS7J1ctKfqNsHw9s67aVe8i7m0dZvS.rO_W7KXU.80sN54bRnNgSbS41gh3tve7 dBrIH.q5Y38y3v0ANQCtRhajsphe9W5ha2SgzSnfALAONnIdhmqobiMvSveW9BPOZCkDKgWLJ82R 1yAb2b_k8Q6BwpVnyVH_pppu2NCfoWXec2KbZDG8fQrtezLujk9vSUzY7BezVrBqvES1minzftH. xY2rLDK18nxUPHd292tSNlKl9wfCpOfnwslCc0vBDIvLIaivUN1U_uQLE4sl3fNUiEkB9V7AdhPK 9QiQV8yxHXDFLHwWFYLD8Nhh.cl_sPfESGMt.LvLuPis_gIAoWH4OfqyBpZJiMn4qlk.j6UNOmDv vLWlExNGCNDpB9N676W9KASGHykbeAT_Qbz1s9DzGaxpSScooiHbbxz8dXh2dM9uImfKthoWsxPq JWQP.oSBgON7eTcR_9h33.POzk6Hptbn_2KmJL8XNc5_ZD81zbTgoF6WdoGTKcac8OT2UO__SWTR r2pNW0CWMF4MvrgOxDj0b_UqJmgy9XnuqEDjFENQ0OiSMu3382WlpRuNY01cXEi5lKoHnr8ftCi2 XVolWSu4AuR1wY9tv1RiaKw4TZdm5YFO1OJn3HEtVRghWiCcm0OGiZewMKbfcNbLQ5OMi1VdBweS TaX7RoAozeLMYascfHXXijdPApGHoIWY.38_7xwTx1xvOxTKFVwMXT.06wmjKUb8sLG9EwFgl7u6 UptOe2Mi38iwl.knygYeybY63nsk8jHr3gt0KKDwpQIxPwItz.0jYmfXhonDxGTm.ypq_hDXE52q DgMlLYLVK.81uUdVkBL7nHqKPzM5_gfnEC_qkSghlw4Ln2c7zubqVKiUYDVDvWtKiKXG8T0FS8Rh old6HJs__ChA8MAjY5UviQYjWuZuaum5UwDUnes5nyaFkx1XTu6R8GAD3qADHV8aXWBJTev1cidw 0OtmtpUmygbPnB8eiXmV5mMZUO6hZwOpVrFswLmrovtA6CTdC6G9lEbuZvMDCqDruQFA_NXSVaiV CPnQ9bpJt1uE41CZAlmmZYqiGWDBGHTvcMLZKqBMm1dHidR81Ncsdy53PFVOr311.6i1r76jra2f twmLS8bGthbpqMudCcEYEUAzzPn8wZd8HwlS2WGPeKiIDupeyGQDwSQZhMakJofJwrXl2d8Gai8x tNEg4OaHeSeXK1BjDAhxrHlgRmCiZxGTHp6XR9aoZOZJQey6TizYNJG2O8_wmdRaxdimj4OUgPh5 KWRYe.Y.YD9oG5IVhyQHIX9JAIWrP1g4O_p0a93m08oR9eYsNxRDC75CAfVO5Ru14ARV5R7H8VOy LPnz7mRk9T4gBcbyCUUMCDZXaNgis5_bThh6N4Ef.vxx60BYuVeFXBhLOlfA1zwXOty94IIX5soz f1P7i27kOEEpWDRPr0hMQ4Eur83hKci1LuyaYgyr8ZT_atzPFVLLvUmSEz1ZbrdWO9mYN5WJTarc wUkH3tZ6nWUIoi7iknHx1evhBUrgFICg4_ISujJCAEhbGXmOS.M3szZLw7mI02Kk2aHiAhOx6tAu 27xz6lMiikHIYNDLvy5Dj_yJs__QlgW9Q_lYCJpe45CNK4QEVygqd2zDETco6eGv3XDZFduGDxTN sdaW.ZDdHHxpo_hSTsaJkEzbDVN5XckjK0fDcR1BRLk494xnm_HVCKcuJAvfN1ziCbW_dDwYEBRn r4ybkg1gl X-Sonic-MF: X-Sonic-ID: 880975c6-5ad1-463e-b46c-e6193b7c0aaa Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 06:59:41 +0000 Received: by hermes--production-gq1-546798879c-9kfxl (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID cddc7540fe2761f9834233afda650d2e; Sun, 16 Apr 2023 06:59:40 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: <78E9D9C8-6679-4E2F-87DA-43B5B0224D9C@yahoo.com> Date: Sat, 15 Apr 2023 23:59:29 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <61A6CD66-2624-4A27-8F6A-0300D050F9AE@yahoo.com> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <62F8DD62-8E09-4A07-926B-5DE3DB399609@yahoo.com> <0C286158-8F0D-45D0-8D85-B521278E9518@yahoo.com> <78E9D9C8-6679-4E2F-87DA-43B5B0224D9C@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.44 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.94)[-0.939]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.68.32:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_SEVEN(0.00)[7]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.68.32:from] X-Rspamd-Queue-Id: 4PzgzM2hHwz45Fy X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 15, 2023, at 21:31, Mark Millard wrote: > On Apr 15, 2023, at 17:27, Mark Millard wrote: >=20 >> On Apr 15, 2023, at 15:49, Mark Millard wrote: >>=20 >>> . . . >>>>=20 >>>>=20 >>>> (Mostly written as I progressed but some material later >>>> inserted into/around previously written material.) >>>>=20 >>>> Summary: >>>>=20 >>>> As stands, it looks like reverting the dnode_is_dirty >>>> code is what fixes the corruptions that my type of >>>> test context produced via poudriere bulk activity . >>>>=20 >>>>=20 >>>> The details that lead to that summary . . . >>>>=20 >>>> Using my my build environment for updating my temporary, >>>> experimental context, an environment running a world and >>>> and kernel that predate the import: >>>>=20 >>>> # uname -apKU >>>> FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #90 = main-n261544-cee09bda03c8-dirty: Wed Mar 15 20:25:49 PDT 2023 = root@CA72_16Gp_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400082 1400082 >>>>=20 >>>> (Note the "nondbg": I normally run non-debug main builds, >>>> but with symbols not stripped.) >>>>=20 >>>> The kernel and world for this are what is in old-main-CA72: >>>>=20 >>>> # bectl list >>>> BE Active Mountpoint Space Created >>>> main-CA72 R - 3.98G 2023-04-12 20:29 >>>> old-main-CA72 N / 1.08M 2023-02-06 19:44 >>>>=20 >>>> (Most everything else is outside the BE's and so is shared >>>> across the BE's.) >>>>=20 >>>> I updated to also have (whitespace details likely >>>> not preserved in this note): >>>>=20 >>>> # git -C /usr/main-src/ diff = /usr/main-src/sys/contrib/openzfs/module/zfs/dnode.c >>>> diff --git a/sys/contrib/openzfs/module/zfs/dnode.c = b/sys/contrib/openzfs/module/zfs/dnode.c >>>> index 367bfaa80726..49a7f59c0da4 100644 >>>> --- a/sys/contrib/openzfs/module/zfs/dnode.c >>>> +++ b/sys/contrib/openzfs/module/zfs/dnode.c >>>> @@ -1772,17 +1772,7 @@ dnode_is_dirty(dnode_t *dn) >>>> { >>>> mutex_enter(&dn->dn_mtx); >>>> for (int i =3D 0; i < TXG_SIZE; i++) { >>>> - list_t *list =3D &dn->dn_dirty_records[i]; >>>> - for (dbuf_dirty_record_t *dr =3D list_head(list); >>>> - dr !=3D NULL; dr =3D list_next(list, dr)) { >>>> - if (dr->dr_dbuf =3D=3D NULL || >>>> - (dr->dr_dbuf->db_blkid !=3D = DMU_BONUS_BLKID && >>>> - dr->dr_dbuf->db_blkid !=3D = DMU_SPILL_BLKID)) { >>>> - mutex_exit(&dn->dn_mtx); >>>> - return (B_TRUE); >>>> - } >>>> - } >>>> - if (dn->dn_free_ranges[i] !=3D NULL) { >>>> + if (multilist_link_active(&dn->dn_dirty_link[i])) { >>>> mutex_exit(&dn->dn_mtx); >>>> return (B_TRUE); >>>> } >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>> I did my usual buildworld buildkernel sequence and then >>>> one of my normal install sequences into main-CA72 to >>>> update it to have the change, as well as the prior >>>> material involved in my first experiment that I'd >>>> reported on. >>>>=20 >>>> I cleared the content of the jail that I use for >>>> temporary experiments, such as the prior testing that >>>> got the 11 builder failures: >>>>=20 >>>> # poudriere pkgclean -jmain-CA72-bulk_a -A >>>>=20 >>>> I then rebooted using the updated main-CA72 BE. >>>>=20 >>>> Then I started the: >>>>=20 >>>> # poudriere bulk -jmain-CA72-bulk_a -w -f = ~/origins/CA72-origins.txt >>>> . . . >>>> [00:00:37] Building 476 packages using up to 16 builders >>>> [00:00:37] Hit CTRL+t at any time to see build progress and stats >>>> [00:00:38] [01] [00:00:00] Builder starting >>>> [00:00:40] [01] [00:00:02] Builder started >>>> [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 >>>>=20 >>>> In the prior experiment it got: >>>>=20 >>>> 476 =3D 252 success + 11 failed + 213 skipped >>>>=20 >>>> and it reported the time for that as: 00:37:52. >>>>=20 >>>> A normal from-scratch build takes many hours (multiple >>>> compiler toolchains and such) so my first report after >>>> this point will be for one of: >>>>=20 >>>> A) It got to, say, 00:40:00 or beyond with, or without >>>> failures. >>>> vs. >>>> B) It got failures and stopped before that. >>>>=20 >>>> . . . TIME GOES BY . . . >>>>=20 >>>> At about 00:40:00 the status was: >>>>=20 >>>> [00:40:00] [06] [00:00:00] Building x11/libXv | libXv-1.0.12,1 >>>> load: 30.73 cmd: sh 1508 [nanslp] 2400.88r 6.69u 11.90s 0% 3960k >>>> [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 235 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 241 Time: 00:40:01 >>>> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >>>> [15] 00:07:44 devel/py-lxml@py39 | py39-lxml-4.9.2 = stage 00:00:08 40.00 KiB 0% 0% >>>> [01] 00:00:34 x11/libXxf86vm | libXxf86vm-1.1.4_3 = build-depends 00:00:03 56.00 KiB 2.3% 0% >>>> [16] 00:01:59 x11-toolkits/libXt | libXt-1.2.1,1 = configure 00:00:52 40.00 KiB 0.3% 0% >>>> [02] 00:01:40 devel/dbus | dbus-1.14.6,1 = configure 00:00:05 36.00 KiB 0.5% 0% >>>> [03] 00:02:20 x11/libXrender | libXrender-0.9.10_2 = configure 00:01:27 40.00 KiB 0% 0% >>>> [04] 00:00:44 graphics/libglvnd | libglvnd-1.6.0 = build-depends 00:00:13 52.00 KiB 20.3% 0.1% >>>> [05] 00:01:39 x11/xprop | xprop-1.2.6 = configure 00:00:45 56.00 KiB 0.7% 0.2% >>>> [06] 00:00:14 x11/libXv | libXv-1.0.12,1 = pkg-depends 00:00:03 52.00 KiB 3.6% 0% >>>> [07] 00:01:57 x11/libXfixes | libXfixes-6.0.0 = configure 00:00:42 40.00 KiB 0.1% 0% >>>> [08] 00:03:01 devel/glib20 | glib-2.76.1,2 = configure 00:01:26 40.00 KiB 4.3% 0.1% >>>> [09] 00:01:21 shells/bash-completion | bash-completion-2.11_2,2 = configure 00:00:13 32.00 KiB 5.7% 0% >>>> [10] 00:06:26 devel/qt5-buildtools | qt5-buildtools-5.15.8p157 = package 00:01:57 44.00 KiB 76.1% 0.1% >>>> [11] 00:01:20 print/psutils | psutils-1.17_5 = stage 00:00:03 40.00 KiB 0.2% 0% >>>> [12] 00:02:09 x11/libxkbfile | libxkbfile-1.1.0 = configure 00:01:22 44.00 KiB 0.1% 0% >>>> [13] 00:08:54 devel/cmake-core | cmake-core-3.25.1 = build 00:01:43 36.00 KiB 694.9% 5.2% >>>> [14] 00:01:20 x11/xcb-util-image | xcb-util-image-0.4.0_1 = configure 00:00:14 48.00 KiB 0% 0% >>>> [00:40:14] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >>>> [00:40:22] [11] [00:01:28] Finished print/psutils | psutils-1.17_5: = Success >>>>=20 >>>> So no failures so far, 235 ports built and a bunch in process. >>>>=20 >>>> Note: maximum observed ("MaxObs") load averages so far >>>> as shown below (for there being only 16 cores, i.e., >>>> 16 Cortex-A72 threads, one per core): >>>>=20 >>>> load averages: 43.75, 34.32, 27.40 MaxObs: 45.03, 34.32, = 27.40 >>>>=20 >>>> I'll report again if it gets a corruption failure. >>>> Otherwise it will be many hours before it would >>>> complete without such failures (multiple compiler >>>> toolchain builds and such). >>>>=20 >>>=20 >>> To be explicit, since I've now seen your commit: My >>> test does not include the change (whitespace details >>> not preserved in this note) >>>=20 >>> @@ -2650,9 +2641,7 @@ dnode_next_offset(dnode_t *dn, int flags, = uint64_t *offset, >>> rw_enter(&dn->dn_struct_rwlock, RW_READER); >>>=20 >>> if (dn->dn_phys->dn_nlevels =3D=3D 0) { >>> - if (!(flags & DNODE_FIND_HOLE)) { >>> - error =3D SET_ERROR(ESRCH); >>> - } >>> + error =3D SET_ERROR(ESRCH); >>> goto out; >>> } >>>=20 >>>=20 >>> I make no claims about which way dnode_next_offset should be. >>> I was only providing some independent evidence that might >>> prove of some use to folks that understand the alternative >>> code sequences. >>>=20 >>>=20 >>> For reference, at about the 1 hr point for what I'm testing: >>>=20 >>> [00:57:49] [01] [00:00:51] Finished sysutils/u-boot-tools | = u-boot-tools-2022.10: Success >>> [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 306 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 170 Time: 00:59:49 >>> ID TOTAL ORIGIN PKGNAME PHASE PHASE = TMPFS CPU% MEM% >>> [06] 00:16:54 devel/binutils@native | binutils-2.40_2,1 package = 00:04:22 56.00 KiB 100% 0.2% >>> [09] 00:15:50 lang/ruby31 | ruby-3.1.3_2,1 package = 00:00:28 40.00 KiB 100% 0.2% >>> [13] 00:28:43 devel/cmake-core | cmake-core-3.25.1 package = 00:14:20 36.00 KiB 100% 0.2% >>> [01:00:03] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >>> [01:00:06] [06] [00:16:57] Finished devel/binutils@native | = binutils-2.40_2,1: Success >>=20 >> Just a status update as of lang/gcc12 finishing up: >>=20 >> [02:18:39] [03] [01:18:30] Finished lang/gcc12 | gcc12-12.2.0_5: = Success >> [02:19:11] [02] [00:12:27] Finished print/harfbuzz-icu | = harfbuzz-icu-7.1.0: Success >> load: 63.40 cmd: sh 59209 [runnable] 0.03r 0.00u 0.00s 0% 4012k >> [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 404 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 72 Time: 02:19:04 >> ID TOTAL ORIGIN PKGNAME = PHASE PHASE TMPFS CPU% MEM% >> [15] 01:15:01 devel/llvm16 | llvm16-16.0.0_1 = build 00:55:16 4.00 KiB 428.6% 4.4% >> [03] 01:19:09 lang/gcc12 | gcc12-12.2.0_5 = build_port_done 00:00:39 4.00 KiB 27.8% 0% >> [04] 00:51:27 lang/rust | rust-1.68.0 = build 00:41:24 8.00 KiB 480.6% 4.3% >> [06] 00:12:09 print/ghostscript9-agpl-base | = ghostscript9-agpl-base-9.56.1_8 build 00:07:19 36.00 KiB = 0.7% 0% >> [09] 01:15:01 devel/llvm15 | llvm15-15.0.7_1 = build 00:57:14 4.00 KiB 436.2% 5% >> [02:19:18] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >>=20 >> So 404 packages built and 0 failures to build. 5 builders active >> at the time. >>=20 >> FYI at about the same time: >> load averages: 64.31, 66.32, 66.68 MaxObs: 99.41, 83.15, 74.18=20= >>=20 >> I'll note that I use non-default options to avoid LTO based builds >> of compiler toolchains and such and this testing is of my normal >> style of builds, not of FreeBSD default options. >>=20 >> I'll also note that, unlike my normal builds on this machine, this >> is with USE_TMPFS=3Ddata ( instead of USE_TMPFS=3Dall ). This leads = to >> a lot more ZFS I/O, which seemed appropriate for the type of >> testing context involved. I've one other machine that I normally >> use USE_TMPFS=3Dall with but the rest normally use USE_TMPFS=3Ddata . >> This is because only the 2 machines have the RAM to make the >> USE_TMPFS=3Dall reasonable. The machine in use has 64 GiBytes >> of RAM and "Swap: 241664Mi Total" for the 16 hardware thread >> context, not that it has ever used much of the swap, except when >> some process turned into a massive-size file generator. (64-bit >> offset limited other than my noticing and killing such?) >>=20 >=20 > The bjam command used for stage for devel/boost-libs is a > long running, single process, single threaded context. So > far over 50 CPU minutes worth for stage. >=20 > [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [parallel_build:] = Queued: 476 Built: 463 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 13 Time: 06:40:59 > ID TOTAL ORIGIN PKGNAME PHASE PHASE TMPFS = CPU% MEM% > [02] 01:06:33 devel/boost-libs | boost-libs-1.81.0 stage 00:55:57 = 16.00 KiB 100% 0.5% > [06:41:12] Logs: = /usr/local/poudriere/data/logs/bulk/main-CA72-bulk_a-default/2023-04-15_14= h47m19s >=20 > But, so far: 463 ports built, 0 failures. It may be > a while before the last 12 get a chance, as they > wait for devel/boost-libs to complete. No failures overall: [main-CA72-bulk_a-default] [2023-04-15_14h47m19s] [committing:] Queued: = 476 Built: 476 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 0 Time: 09:08:52 (Several of the last builds were basically each a sequence of single threaded single processes that took notable time.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 08:34:19 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Pzk4r08Dpz44v79 for ; Sun, 16 Apr 2023 08:34:36 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-21.consmr.mail.gq1.yahoo.com (sonic313-21.consmr.mail.gq1.yahoo.com [98.137.65.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Pzk4p5h8zz4Dpv for ; Sun, 16 Apr 2023 08:34:34 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=FmqRCNBH; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681634072; bh=qk+AlQx6tvPcT2DL4eL8rjDD5QmlX5qD/49J5Ga/KHA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=FmqRCNBHTslJRe/Xj+XMvg6euix/JfW7sS6nsCdtWE3QTN0ESabIL1QJSvmtX6aGkBdk72XmbbWNoocyt+YCSjBF26uz+hffRgwrOiHURJkVbnPleOkvaPNh4k88GRkJb63flpo7KwawM3AqMEJrDF2vPyucl992lqQXjnOnv73drdT5WT9kiPNlfZOj8FcceqjTBka+c7nkCC/fbs7NHuaJfOBpKDjwjDli//3U81OqRGRDBPGMYOrPxFxg4GuCTSfP4nMhWMysjWrKifC3TjKwKFAFNIucluTyFuklkvfAL+fcdgdH3lZyDbUrhUx8SP+xbnj+ORB7IWlPJ+hiyQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681634072; bh=KrrCsAaJAQsc7qEH51HejWNbd7xY+vl10j3LdMH9vZ+=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=fNRmpmazwUSGqBCYhwGTzJ+T876mzRgzfDBppxQ01LcV5n0wj1RPqGh6BoWPKQRQJVeRc4+Ifq1N+CY6HtKPbNXGZx5LRzC17qb9A+m7Ut+E7OtgKPCqpwOBL6/EPGmbI81K/KwudERh8wmMXOg5JrkW7ohfM0K14JKKj7el4I3dPjske1zziTYOZRDJNeSTTsJFduZSPqS8/wU7eIZEfGdSwARmEZ7H5kTEVs277zYAPbVcHJlzp7dYG1YNFIVLqu1wzrc999Ok6wDQpvxqXHUg+wNSNLaK1ipGL7foPDrVZRoM+YiB87bblJO/HCJvzHnXJ+ki1TTNmX4Mi+tsew== X-YMail-OSG: rPujsqEVM1lrXNB03g8onDTOiSQUzo9VYuZfYG7Yl.sPDwSG2ovjeV4KS3w56kK O87zAh4fcSsJnMGsBddWXMFYIPs2mFmIDdRzP7lqULDPtchADlMyaoDBK3x896k_9TnG4BNpe55M 9J6W8J1FMkr.FYbUDb2lwUkQuKwhM1wfAP6p3ztZmTSF5FAjB2N_biMXJ6JA1G_Uy46i6J4nb7tt 8pvb5gpp8lrl5k.TJoXfAk.fEPZz9d35ku2yi0JK.cUz_uz4WEofFwu9umNe4FlZ4TGw82iBrmiX JRW59mKBQY3PqNCsWDd61WTp7Lc1NuHcDztg.8uGfe7acypj1FoWnA0joXJOtwZOoXkbJklLt09E 3zniOTDqJblDBf_KPnwMlXeeY6lFI8EJlD.RO0U2a6CM_y_ZF377uUCJ7BXyadV5yD7ZQy8wSiUh ka4qwNMmu3MQV.mluiVQoOAGiRHj9sNyMi_q6fH6jBW5MCZIWSdfQoUGv8DVHLbXiAasocuiPP0_ 5ozStJqGw7VXrquA2R3oXPuVU65v1yOlR6eUihpILMSLH1W.oq8OGEqo0o8Px5XggUh1gVMG5Riu kq5pG9BWnE2KhmPshQkxY9rs6tf.Jhqx4sWHHP1xXyx3QhW_knHxf9yL.7FS98TPTLNdr44JJI7j AaKlIPleFvGFngxnTyhRXlQ.IfE85k681CYQLyqSVJWeCssulrXNlt2_Gmy4cJW7yucsxKo8DS1D fLo6F78cQMq9JqsQsC_L5beudo6UdSdgQwLvxW6XyzehDXRttS2cLtGgEGM3skocIerRvBGnGckf pmd0G8b4evYozZ.mTSnBhgSlEcvXlk_8v1Gkq8WFSd3_YPPpvEvrATiqakBLjYN.e_I5Se5tjFkt SiFJeiypbQwHQOLgVWPe6jkkerfiYf71LxGIcQTYCvqAWpmWSwa4F._ozM9_YaiLhNHHDop34Pqk jnPULE4xGCu1r7MsCaYVZrck8yEb7PWgwjXwRg.gNQcUg8LmirjayRfvsXNJhWgtAbEcpibirxUS ms7VCgOGlZbN8yFSuMukP8q1fhNfuezpWJ6LSX1D_XqcxnbAK2zU5CyOu0i1iHmKefba.TaIpCtJ zGpPiD65tRJ.nfgEwG_yZ_4JMhWmo31i67f1MftU2g8dLQOrwWonsEYLdywi1c8cGdpfGRXd33bl MnLMxhtY7tTGWP2lTj1e6P71cke6GzwL__BI3IdzVQvtr04tyZi1R5JJQef.7JTZzf7WoMZrJo_o oCPrYWloSHtGVlKAaFdbLqpgQsf9mrviZvMTsjXf5d4kxL7b5NRB0.LNlwkB6DFf9lqb6nycfB7w _wvkFm97n1mlKnNeZgyUJ5YXHxnXCwzaFbJHBRA5CqmSTGYAJUmoqjBoD5rF_9y7oMTm1oxfmale RjCxHjXmVIa96Bi5434mqdpfs1cyc7rLysKfl0i68C5CUgFbXxBEDmjtGuRftu1Gm_MtBdo..jp. 0eVRG3pbRkm.juG6qylMnQTaPjYDo7ri5Mf1100Hmo9vli2xv8Y_ebASpx0tuUi.QPC9KYaEz4mq VKmFxt0He3iLBGXNquN4T08WA68jYuMRbwn0bX9MAahFcc0B_QABkNtJHTcQVKZE5WMHQXh.tWDJ oLfqaWSG1rKAwATgUR4v5ZK5LpjaN.nsCwXmLxv_RE1aZhV80BqispLNzr.Vy.NaC2Ia1jqmqPD1 dqxAmpMuHt3EBMFHm2oO_wGeRPaQIiEyXbee4hQjWusyUXPVeeqicErWujiKjssBWh2FomCdcEdq u5pPUlpPRAQWPBGCVorKQFs37JZl81LSHMjpRx5PbgRkDKtzEIUfd6gy7jXvP0MciEin6rXexcKj wpFSFQ6P3z0pqj5tSLiD2HZhCp15y2lr2Vx4.b54PJ012qKNbRuJEwsToRdv0B_grsS4CGcWRzxW 6c4ErGFjPWT_HsjI2xama.4T4scQp_I7uqu0A_MPNmxwZQfj3pptjitJ5JtN_C3yIexFpvJXy.JF FgwmT_K9QfFYonCjRp5B3Iw.n2LopuEDVJzsVcpPtrg6vFCDO1Ljgsc_LsFMeLTZZsX2_lgfC5ds Q6M_2omBkK52ypyCTuW4F9ayWLcrI9SRslLOkio4FtC7s72O5CjKOoQHuQJIoTeFfpOclDhcjD5p Fg_vRcxWV7rRzBV5s6tBKncDsOAN5jFQYuENM_Hd_BAcmdJSQZrVH6zdrnEFIt5yQ1BpnTO4rHM1 5HCwHyTWN8KlQyXSAXuxn7uBms7TZfCByJHAklzGFy1ooouk5OGHrvs9Id3IMI7EjdvoDX3ROKMi tcvlLo2Y- X-Sonic-MF: X-Sonic-ID: d66473bf-7814-46b1-89f7-1e487ce43c1d Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 08:34:32 +0000 Received: by hermes--production-ne1-7dbd98dd99-gdhzk (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 6859fad07ccc9bd7d5616f1889b708bf; Sun, 16 Apr 2023 08:34:31 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: <3BB7D5DD-99A6-4D27-BBB6-4CD3294EEDF9@yahoo.com> Date: Sun, 16 Apr 2023 01:34:19 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <96A74C34-22FE-4F7D-B032-16313B6E1612@yahoo.com> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <3BB7D5DD-99A6-4D27-BBB6-4CD3294EEDF9@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-2.44 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; NEURAL_SPAM_SHORT(0.06)[0.062]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.84:from]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_SEVEN(0.00)[7]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_TO(0.00)[gmail.com]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.84:from] X-Rspamd-Queue-Id: 4Pzk4p5h8zz4Dpv X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N On Apr 15, 2023, at 19:13, Mark Millard wrote: > A general question is all for this message. >=20 > So far no commit to FeeeBSD's main seems to be > analogous to the content of: >=20 > https://github.com/openzfs/zfs/pull/14739/files >=20 > After my existing poudriere bulk test finishes, > should I avoid having the content of that change > in place for future testing? Vs.: Should I keep > using the content of that change? >=20 > (The question is prompted by the 2 recent commits > that I will update my test environment to be using, > in part by fetching and updating to a new head, > avoiding the "no dnode_next_offset change" status > that my existing test has.) >=20 Not knowing, I updated to: # uname -apKU FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #92 = main-n262185-b1a00c2b1368-dirty: Sun Apr 16 00:10:51 PDT 2023 = root@CA72_4c8G_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400086 1400086 with the following still in place: # git -C /usr/main-src/ diff sys/contrib/openzfs/ diff --git a/sys/contrib/openzfs/module/zfs/dmu.c = b/sys/contrib/openzfs/module/zfs/dmu.c index ce985d833f58..cda1472a77aa 100644 --- a/sys/contrib/openzfs/module/zfs/dmu.c +++ b/sys/contrib/openzfs/module/zfs/dmu.c @@ -2312,8 +2312,10 @@ dmu_brt_clone(objset_t *os, uint64_t object, = uint64_t offset, uint64_t length, dl->dr_overridden_by.blk_phys_birth =3D 0; } else { dl->dr_overridden_by.blk_birth =3D dr->dr_txg; - dl->dr_overridden_by.blk_phys_birth =3D - BP_PHYSICAL_BIRTH(bp); + if (!BP_IS_EMBEDDED(bp)) { + dl->dr_overridden_by.blk_phys_birth =3D + BP_PHYSICAL_BIRTH(bp); + } } mutex_exit(&db->db_mtx); and booted the update. I've done a: # poudriere pkgclean -jmain-CA72-bulk_a -A and started another package build run based on that combination: # poudriere bulk -jmain-CA72-bulk_a -w -f ~/origins/CA72-origins.txt . . . [main-CA72-bulk_a-default] [2023-04-16_00h38m01s] [balancing_pool:] = Queued: 476 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 476 Time: 00:00:24 [00:00:37] Recording filesystem state for prepkg... done [00:00:37] Building 476 packages using up to 16 builders [00:00:37] Hit CTRL+t at any time to see build progress and stats [00:00:37] [01] [00:00:00] Builder starting [00:00:40] [01] [00:00:03] Builder started [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 . . . If there are no failures, it will be about 9 hrs before I know that. Given that I'll be trying to sleep soon, it may be about that long either way. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 17:40:41 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzyCJ3BHxz45kJW for ; Sun, 16 Apr 2023 17:41:00 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic313-20.consmr.mail.gq1.yahoo.com (sonic313-20.consmr.mail.gq1.yahoo.com [98.137.65.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzyCF6dVdz3h8F for ; Sun, 16 Apr 2023 17:40:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="nUmdHHe/"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681666856; bh=E3clUVG2WtHSx3suGFRBhW/QN2ow0Kt/i7KLvcxeu94=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=nUmdHHe/tpHn5OTXWBmH127xfUxRbdyyLGNKtOl5Mg2TEyWo52/RqMbn6E2qmUkFJlejUltj7YZsxTeS2txchrNVzr8y3YnQ9VzV1hfS8qnwUi9vF148WfttlwV9W7E2CiL6CtWKkPgVxlVz5nKEYnDLKSbgb9SHLFA1ClzHoAk1znZ+qX3CNywnpIisb8QIzRrLatvRUGY/1v2PathgonBW9W1wscxXrhn5jD8HkE/ssoxfE/NiCJiKVGbIdV7WwR+ba/8SuJ2+YgrXeadLXSZD21Hwc/IuyOB30pWAnIKY5FLQsKIEb1yI4Aon4qV6HSuTlmj4N2sathnbjQGwkw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681666856; bh=hsbIm7w+/q60S75bWk8C7SF7ZiQT4NqK1u+kKrXKrhF=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=Kg5sIZZ8tUWDpc5wnnyF0G2GijPEpxuQ7t67fhRkYfExA3jJswoVrXBf6+uLjImAj/uPeB3hPLD3KhnvzggUx6QS6nPqNevmkPApWLTb7NAQ0i7+kLS10uHWeSDkS6dVgL31f4vZNjt4ClqMktVGOOXsO7XAcSDrhZ5uemOyrOWQPq8T1wBgU6vOqaFdtHZKUvRoNyI+KEkhTvMFnM4gSQUyBwlVrIFXU8280R9ek8uxhFwnksEMfWb4dSZEelNJ8hQzxDTytRNF8XqCHF6EGzAEq23kwrNdCW37z+rmKuJ0jClI/MgyShcnGUzePfvubA9WpsQjUJjwBG2LjfV2Qg== X-YMail-OSG: gE1kqdIVM1mWM08PgC.uvYTYJpr0ht8txhHarpqchpEIDxEyt94SC2sEGxIgn2q pbBoh93Zh7sywFuytdK2jMvWOjQL9pUBeSR7vGW0jun3fmJIUbpGs7XrTjdprS0MXDHzodapc59T CLbBSCBhRj0Po7PY_lFog4J9QKxZbQw0i7t2vZws_9Rb08UO3snVJPDyf1PcveampAwT839Y_cIJ ylJq_fBL7dpC7YhslMMaIahwWV29jgMQKWFXPJONl0f2VnvagTxSpavasyuT2kLcWwo_EeNDlhrj SQzihJCZpb0gzO1Kbjj8FCO5YviTu0jvEcKzruMoVXfr83ye1dWf5JH_I1ei8MIswLXHaC6FsgXL Wq0F2uyabCsaM4pJzpe2Kj6TaScfY.qjMVkK2t6LEX2YjYCSsYfMOAyvcqsWTQovYQ9IO_3_vnYX VOef_oMScM5Vg76bjdTif0gUIlUu5iAihAocCAvQKYK_C3eAd.97wgtPIdly5JXdbZuwvIUq.LH7 BFPFHxaWFJCymocYFdr53VnkWs._MoShtehoF4enV9fUdId2._O94vOBfLPUB06XT5rXV3j7W4Ox w0mH3cCx0CnxRcXYrJpWPkZNgJcapXrBxHrg5Z1_IAHtjganIq2qcvMarIPm.JEocvkbmG8T5PEO bE_SwzOYOD9IXgrTphpsYBFeb9EyYhFL6cil1jGarnt.Mj8QwwGvyAm0HSfGIt.8gJazw7d7xzbo puZKzap9z7yzspRY0iSVROu4JmxARz6bCq1vEVxP0wnFwojMp32l3TMkGQG54O2HtViuF9QIGDlK Hlt5.nzOh1QZBRksND.FD_pdIUyihmhUwBOWWP7XDXpvpzx3a9Slo7N0LnZYlMaa_QsHyH73s53c IsDtEQ.W0qH8arXRwMQJf6hp45zlfGnWFyLIBoyxMdXDPmIdYyEOXGiR9ESsTZG2oNPyrDf8qJ1G toqMo9QcJOAXTWWI3EmfKgOVe2takC6M6xnwukfSg2ClW7LOWCIkFgFD.tZzZJzXPp.luJpv5sMt PC_hnl3W.hnZKeVzPWXYtfVwpsXr4elvaNKQLoqtHk8fpbmL5iVOX.UNvenPtRPGD.XRODOwtMHa gJqbVQfPGsxBexAyV_ebCcDLv7j05zRXowzkznbXxRT6dMg8CsCCzrZXd6DjdPxkMJztOFwIzMpC 7mLP9WAi8sJwe19.LaGiGxDyHSPCiKv3MMqQ3M1q5Aintj_Bz5HjDeTPoUPrtNJKv6mqveTeWvas hOphjEJf7v3BniD99zY7hnZSIlS7qA7UI30Rn0VIaf9Qh3kXuT3xG82ArcjdwAymhxqhsRmrwaCW qFr9vlri6bM_qDOm8IkIPoUNdW1x3jDChV5DUIIcxcVwgVkibV7aYYAYZJ0bxg8lIWRSfrObfvJl fnDUtNAR88OqWENX175ga0MHNcXNrOO0iKVrUV7ybyJgOGMd.VUfRYyFKTgeRLBlE1y8tL7bhhwa dTIn1u9pd8pj61RpcOKl6AMFWgxbxsC.z.vkpY36bsJGt3lFN0NTFQ9XKaRy5mtMeM6lLPMEnUSO J_5EuYunq9vDUFaH.ADcSBrGANk_4pmxJnWloUI3U3uNnJg5XpGOhcw9MGL9uazV7nAalRm6eMOI 6XI856PA2Qv4QANBaUzCKnASDLB71QVlSCn6DUhnaSbIUpoJX_RYUP6HFFcmYnltdFPlyUsI7QQm AKvWh6n_YmLwblWEdrd5hoMh6Bf2FttF75AhIldxIpFNIrG_AmbwS0klFZpaPIxXCSa3w.1VG5CP a9U8syFLOHWMPmUqam_L08sUr6sPysP3KT63.1OrgVZ_N4Y0cabEgPWPNe3GMce.jQULVofTc9Mz 4roqVdJhWXFfdvPsBAUvlSImjFBYfrIie.g8RSLoBBuP1gGyKPAPPHCZGh41nFKqistNGDTQ2oyD bGzw9DyK5IFAsVBJDzkJphb7slRlfowVKFK3FQm4TDtgrbqo_e9geS4Cv2DGmhdLcAXH2lkyfOvl 5Ml21gNRlhi8QdWDJwkRdD_FvUi5NJa_mrdFwS4G7aavkVTSLuoiRnNVEGYiZDIAvCd8v1QlK5yO bn5_jstW67kpniVMAS4Mk1D3g1nETKUy4je7yjU46mQC0VBKdhXuuMND6RGB4_44zKg2lVkexaDR LLS7UPMEvfYl5fHcJ3rxfR.v511hlRLKDT1pdxN9cvFv11ARVyeCtdY5zmRzsqwlFcvqUS0FSSk9 S89La1HVyjlRtkAImRLLzsok7_I1fZUDSee2IKke8.B57JSlS9Y.hd_r3fYPwXB.5OMCQYWQFpj3 W.xW2snM- X-Sonic-MF: X-Sonic-ID: 623c22d3-d39f-4263-8edc-32432c5065b2 Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 17:40:56 +0000 Received: by hermes--production-ne1-7dbd98dd99-vd22t (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 269ea699665a5aba4d8c800a34be52ef; Sun, 16 Apr 2023 17:40:53 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: <96A74C34-22FE-4F7D-B032-16313B6E1612@yahoo.com> Date: Sun, 16 Apr 2023 10:40:41 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <3BB7D5DD-99A6-4D27-BBB6-4CD3294EEDF9@yahoo.com> <96A74C34-22FE-4F7D-B032-16313B6E1612@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.83:from]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_SEVEN(0.00)[7]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.83:from] X-Rspamd-Queue-Id: 4PzyCF6dVdz3h8F X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 16, 2023, at 01:34, Mark Millard wrote: > On Apr 15, 2023, at 19:13, Mark Millard wrote: >=20 >> A general question is all for this message. >>=20 >> So far no commit to FeeeBSD's main seems to be >> analogous to the content of: >>=20 >> https://github.com/openzfs/zfs/pull/14739/files >>=20 >> After my existing poudriere bulk test finishes, >> should I avoid having the content of that change >> in place for future testing? Vs.: Should I keep >> using the content of that change? >>=20 >> (The question is prompted by the 2 recent commits >> that I will update my test environment to be using, >> in part by fetching and updating to a new head, >> avoiding the "no dnode_next_offset change" status >> that my existing test has.) >>=20 >=20 > Not knowing, I updated to: >=20 > # uname -apKU > FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #92 = main-n262185-b1a00c2b1368-dirty: Sun Apr 16 00:10:51 PDT 2023 = root@CA72_4c8G_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400086 1400086 >=20 > with the following still in place: >=20 > # git -C /usr/main-src/ diff sys/contrib/openzfs/ > diff --git a/sys/contrib/openzfs/module/zfs/dmu.c = b/sys/contrib/openzfs/module/zfs/dmu.c > index ce985d833f58..cda1472a77aa 100644 > --- a/sys/contrib/openzfs/module/zfs/dmu.c > +++ b/sys/contrib/openzfs/module/zfs/dmu.c > @@ -2312,8 +2312,10 @@ dmu_brt_clone(objset_t *os, uint64_t object, = uint64_t offset, uint64_t length, > dl->dr_overridden_by.blk_phys_birth =3D 0; > } else { > dl->dr_overridden_by.blk_birth =3D dr->dr_txg; > - dl->dr_overridden_by.blk_phys_birth =3D > - BP_PHYSICAL_BIRTH(bp); > + if (!BP_IS_EMBEDDED(bp)) { > + dl->dr_overridden_by.blk_phys_birth =3D > + BP_PHYSICAL_BIRTH(bp); > + } > } > mutex_exit(&db->db_mtx); >=20 >=20 >=20 > and booted the update. I've done a: >=20 > # poudriere pkgclean -jmain-CA72-bulk_a -A >=20 > and started another package build run based > on that combination: >=20 > # poudriere bulk -jmain-CA72-bulk_a -w -f ~/origins/CA72-origins.txt > . . . > [main-CA72-bulk_a-default] [2023-04-16_00h38m01s] [balancing_pool:] = Queued: 476 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 476 Time: 00:00:24 > [00:00:37] Recording filesystem state for prepkg... done > [00:00:37] Building 476 packages using up to 16 builders > [00:00:37] Hit CTRL+t at any time to see build progress and stats > [00:00:37] [01] [00:00:00] Builder starting > [00:00:40] [01] [00:00:03] Builder started > [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 > . . . >=20 > If there are no failures, it will be about 9 hrs before I know that. > Given that I'll be trying to sleep soon, it may be about that long > either way. [Reminder: All my testing has been of a "block_cloning was never enabled" context. This one has the dnode_next_offset change involved, unlike the prior one.] There was one failed fetch but no other failures: [01:25:02] [04] [00:01:07] Finished ports-mgmt/fallout | = fallout-1.0.4_8: Failed: fetch . . . [09:13:58] Failed ports: ports-mgmt/fallout:fetch [main-CA72-bulk_a-default] [2023-04-16_00h38m01s] [committing:] Queued: = 476 Built: 475 Failed: 1 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 0 Time: 09:13:45 Running the bulk again: . . . [00:00:22] Building 1 packages using up to 1 builders [00:00:22] Hit CTRL+t at any time to see build progress and stats [00:00:22] [01] [00:00:00] Builder starting [00:00:24] [01] [00:00:02] Builder started [00:00:24] [01] [00:00:00] Building ports-mgmt/fallout | fallout-1.0.4_8 [00:01:04] [01] [00:00:40] Finished ports-mgmt/fallout | = fallout-1.0.4_8: Success . . . I do not expect the fetch issue is evidence of a problem. I'm counting this as: No evidence of corruption problems. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Sun Apr 16 17:47:39 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PzyML5hS7z45kqr for ; Sun, 16 Apr 2023 17:47:58 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-21.consmr.mail.gq1.yahoo.com (sonic309-21.consmr.mail.gq1.yahoo.com [98.137.65.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4PzyMH4TM7z3st6 for ; Sun, 16 Apr 2023 17:47:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=VJ3DVNha; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681667272; bh=a+1XI1u9e1NUwUnr5pmVDGPJpWxdvB2bXZ1D7T0iKcQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=VJ3DVNha96fVeeFvz9w4qp0SuSK8R/Bk6F/f0SXEWYSLWvcU32vT+vT5yFRjOqF34E6vtcTX7tgpRpKOOCPwvx6uJKDtlR67HddoxbfG1wRcBfMBJg/ZTxpqMj5dOA+/TjsYgbPV4Ia+394uMZQDfjzZ/MPYq65oX3IMsUsh4EDahoeuXV7EW7d9iSNsZeeOh/tQdt4qXFg9CTbTNb1lfPPIpNukjLIiResr1mrjtKVtA/26Ayj3ID8Gi+aQogeDzYkCuIyg4/yD/FF1a6XRiO6el6O3ogEdtDq7gCqwCyfDEQ3++kTterzoau7sYbFZuaSn6ywgclERv44NMqmrJg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681667272; bh=NO4man3yZIkQ/fG35OtsOH+1gkOrsrTEd0xmNi+B2v6=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=eTLEyEAnP/P9CezTQOStLR0lhHqkYpPsGFOwudvEHHXcaDAmWX+jQIvDk8h8A9gqrlvSrrvrQ1gF6jaNES9sfhh4/8Wk5agQSo9GSuo2M2pt9NwpUOOSLu0uuiSXVXJaSVxDyD4kNrJfZqL0qFX4bW8JESQzZ0sT07iT5fqQzJmfRj8WJnOfj0n0s7dTOjZ7uZX15RIhng4wH5/+ZFMy969YJDVe7p6aYXZ1QRB3zs9g6YsCjltmXK8hKDbymaKtiXMD2M/nXTc37xkggBt3+0EXohy+lmSGxpghRfhDJji6OVEC2HVrF8OPf8YHn0V/aU9wAQA/10by7dYFfdT7kA== X-YMail-OSG: c43aYt4VM1mRR6_wxx4U4Kg0_9_JC1S_z_Gwg3eSjOC.v.SFzRmMQg.D1tRPFXa qOzM7MMD652fIHODwLCo6_mecK0DkglK5DIAkfcZ1vkG.xeQiaqZojZpWrRC1enrW7qh0nxEMSu6 q_47cyr_6qWwD7QO25SPCsw_8GGi9xbuAVT4K0fXD9Dgj0wD9XN1riVJAxL9HdKzmKd3qCSvY6nv tv5vHxK.sDEPBXfDBiyxWAAE1bz.oWGZ0VHwnFWSQ66_kmP3Eta_XJH7GtIMxBnfslE79qT4jSaV DPhTseecWX8h3jMQ3VP0IKB5QbGaisNBRH6VEv_F4RKYHt_EZm7gs5IM0JMoatOkFICTAtylv_2N 4RbkmFndLP.JrY3uxyVRHQWoHP2pE61Xk3ImxUoy69Id3j0oQx.vmEiv0FGrgE8hmcNLg7Z48TTM Yfvg57gMFtomXmqXHvfdaVc3.v4OLH4W4I45dRG0NEUeLWuMBAnelYxKxgrbuwZm72Ei49_mNNF8 s5iwbHgWlJmw5myoungF3OutUcqkD2bcl1ot69Q.Ycv5ptHXd2rRbsEYPrgZnrfx1ZSE1v6keIcA eBqT8Dhyv9RkME3IfF_nqQ25JmzV_WBWCvFlWPuEWB1YZFjzf7k6yox0s011WjbRDseXyMCiKS1P dk7bZZTGnOoL0deYusmDbYao.oDIgok0NzA7Wj6iNmdcwsl96G6MMIvkOajNEiOsY63CDomyND_H c6N0Yx7OiYylO9liGBUco5n.yoPSx9ft50IO2fD2hu9IfHQXNWrlKy_aZOJfUG5so7BkbFL9Vv_T eFxmUg4ULHMzRMuEhAV4dRnpm8JYeaNe.D8mzHcvB2_lzp9bGXai7sVe6djkRBU.AyPdJHtcdBOl Nx7pomsErmqa1lfR5JWhtA4FV.xL0MMGf5d31O9Rsli5kybsgQ5LqBtJnix2gQOFH06iXnEl2fbQ mcGGwo3R2Eex6zzp9Qp_gxvsrDufjfX6UcRDOVPAmwMRd1hd0IOSnjv46hrrdPuid5SjCoB3Yfv9 4pHEChyaB5pMJ0RZqGQXt.4LPKJbHUk4sH6HEkY6kl.p19g3xrTjZ45NSEYew2tQV2KLU.IJApvv jVi5Yc0HT4aXxbeKKPeBlAo2HWyQYwysGhenJuDweihTHXkSphNawwwNa49X9ffL1e7ZU8U_lFUK 3660OR8Y5YA0C4fVns8T9EP181swPOb4kXCe8VQC423so1K6yLaOTRHUGNfZlk2OEC2TnoeKQvi5 EyOj_idftSmJnj1us3onAvmon9FkQ92ZLXOmP_wd_p1ZyJkww9ZP8Pl.mLZgnaWfRnGFHDC143t4 Y0scohydY1U8tsFbWXB9Ji5X5CEHj5qq82vgqIlWea02Am4dX.eLd4o51yyXL8zW7iw8M6yH70nQ nXNmW9LkCPpVuAo9SmnwMLZIohhD9v6pY1WPNYSiVUr2fRrFKmD.FNy46JurO5.0tf.X6eHnHKIS iUkJy9GZesXR0MvRpYXxAYT0ed1jpw8iE21Nj7o6V1xWgotQWubWfT3iWTx6mijBnVhiIEIc.HFO xhRmxBMkXD48vKEoMr00lVxXkLtQI0wZtk.39gB6hUYk5g531QBCPjCBULKWUUqnngA3cgMoUjns 4rJGUmEtNT2IJKP9BsDiGOIYMORnd88DI5jLhM1NQaR4nLlptAsC5bfSW7yjy8N9H08WxQ3Jeygl B26jCBsw2CULg4SmZezesiHQ1N62xNDdyZQUBPWdoyIZsEAlmjVaOlyE4PdoeTC5YWvpaEW460Ci .fgtiiw94EmpUYMTb.n6b50ZwuZmULCMqd2g6k2N0EcInbT456BEbXF5gKCOIlMD5BMiBIej0Avy YdFoGwYBTECRiC.k2AiGmqA1.Wl91CdC7CGetBVSwe0QXJg_YggV4u4ISttcDa9RB.l_DPgyQNdm XAqQ06Faw3uYvhCYsj3Bpsku9UB8KrF09xISPseJRM4FovqEYRPieITaQriunkk5wlKV4tnzZEh4 QyfHuZjPZgos7sq5ZnyWVACX3Akbe5d.FesQGW_M263J_nrTzfdU2y.dLt3hbqRObVvMyomfYDae Bf24MF3NRjuNhFu7m0kQ.pfpnkl3_pYXuWCHQRg5BQwKSq7SwbZdfA8uYEq03E8nurmPesadmFKd TO97JUHR0oPCyxurfuIOZykeXBr52u_XlxZ.7OEOnucWGpHGZMepQ5yAmDbslGICNCWfD.HpQvBe QHTbgs7D47Ueu6Om6ytjJ1avdLR80LNoDMNlNDdQIN1uvA.CrcduMrI_Jb6lMSCb.Rc2u55zkuOy TbWOATfCC6g-- X-Sonic-MF: X-Sonic-ID: 7dd32b23-b909-4e9e-96de-e283e79f73cd Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Sun, 16 Apr 2023 17:47:52 +0000 Received: by hermes--production-gq1-546798879c-g88f5 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 20b51038bb310f7fb5688a55e694d07e; Sun, 16 Apr 2023 17:47:50 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: Date: Sun, 16 Apr 2023 10:47:39 -0700 Cc: FreeBSD User , Cy Schubert , Charlie Li , Pawel Jakub Dawidek , dev-commits-src-main@freebsd.org, Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <00F473F6-ECE9-41F9-BA87-A021572E9776@yahoo.com> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <3BB7D5DD-99A6-4D27-BBB6-4CD3294EEDF9@yahoo.com> <96A74C34-22FE-4F7D-B032-16313B6E1612@yahoo.com> To: Mateusz Guzik X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.996]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.147:from]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_TO(0.00)[gmail.com]; RCPT_COUNT_SEVEN(0.00)[7]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4PzyMH4TM7z3st6 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 16, 2023, at 10:40, Mark Millard wrote: > On Apr 16, 2023, at 01:34, Mark Millard wrote: >=20 >> On Apr 15, 2023, at 19:13, Mark Millard wrote: >>=20 >>> A general question is all for this message. >>>=20 >>> So far no commit to FeeeBSD's main seems to be >>> analogous to the content of: >>>=20 >>> https://github.com/openzfs/zfs/pull/14739/files >>>=20 >>> After my existing poudriere bulk test finishes, >>> should I avoid having the content of that change >>> in place for future testing? Vs.: Should I keep >>> using the content of that change? >>>=20 >>> (The question is prompted by the 2 recent commits >>> that I will update my test environment to be using, >>> in part by fetching and updating to a new head, >>> avoiding the "no dnode_next_offset change" status >>> that my existing test has.) >>>=20 >>=20 >> Not knowing, I updated to: >>=20 >> # uname -apKU >> FreeBSD CA72_4c8G_ZFS 14.0-CURRENT FreeBSD 14.0-CURRENT #92 = main-n262185-b1a00c2b1368-dirty: Sun Apr 16 00:10:51 PDT 2023 = root@CA72_4c8G_ZFS:/usr/obj/BUILDs/main-CA72-nodbg-clang/usr/main-src/arm6= 4.aarch64/sys/GENERIC-NODBG-CA72 arm64 aarch64 1400086 1400086 >>=20 >> with the following still in place: >>=20 >> # git -C /usr/main-src/ diff sys/contrib/openzfs/ >> diff --git a/sys/contrib/openzfs/module/zfs/dmu.c = b/sys/contrib/openzfs/module/zfs/dmu.c >> index ce985d833f58..cda1472a77aa 100644 >> --- a/sys/contrib/openzfs/module/zfs/dmu.c >> +++ b/sys/contrib/openzfs/module/zfs/dmu.c >> @@ -2312,8 +2312,10 @@ dmu_brt_clone(objset_t *os, uint64_t object, = uint64_t offset, uint64_t length, >> dl->dr_overridden_by.blk_phys_birth =3D 0; >> } else { >> dl->dr_overridden_by.blk_birth =3D dr->dr_txg; >> - dl->dr_overridden_by.blk_phys_birth =3D >> - BP_PHYSICAL_BIRTH(bp); >> + if (!BP_IS_EMBEDDED(bp)) { >> + dl->dr_overridden_by.blk_phys_birth =3D= >> + BP_PHYSICAL_BIRTH(bp); >> + } >> } >> mutex_exit(&db->db_mtx); >>=20 >>=20 >>=20 >> and booted the update. I've done a: >>=20 >> # poudriere pkgclean -jmain-CA72-bulk_a -A >>=20 >> and started another package build run based >> on that combination: >>=20 >> # poudriere bulk -jmain-CA72-bulk_a -w -f ~/origins/CA72-origins.txt >> . . . >> [main-CA72-bulk_a-default] [2023-04-16_00h38m01s] [balancing_pool:] = Queued: 476 Built: 0 Failed: 0 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 476 Time: 00:00:24 >> [00:00:37] Recording filesystem state for prepkg... done >> [00:00:37] Building 476 packages using up to 16 builders >> [00:00:37] Hit CTRL+t at any time to see build progress and stats >> [00:00:37] [01] [00:00:00] Builder starting >> [00:00:40] [01] [00:00:03] Builder started >> [00:00:40] [01] [00:00:00] Building ports-mgmt/pkg | pkg-1.19.1_1 >> . . . >>=20 >> If there are no failures, it will be about 9 hrs before I know that. >> Given that I'll be trying to sleep soon, it may be about that long >> either way. >=20 > [Reminder: All my testing has been of a "block_cloning was > never enabled" context. This one has the dnode_next_offset > change involved, unlike the prior one.] >=20 > There was one failed fetch but no other failures: >=20 > [01:25:02] [04] [00:01:07] Finished ports-mgmt/fallout | = fallout-1.0.4_8: Failed: fetch > . . . > [09:13:58] Failed ports: ports-mgmt/fallout:fetch > [main-CA72-bulk_a-default] [2023-04-16_00h38m01s] [committing:] = Queued: 476 Built: 475 Failed: 1 Skipped: 0 Ignored: 0 Fetched: 0 = Tobuild: 0 Time: 09:13:45 >=20 > Running the bulk again: >=20 > . . . > [00:00:22] Building 1 packages using up to 1 builders > [00:00:22] Hit CTRL+t at any time to see build progress and stats > [00:00:22] [01] [00:00:00] Builder starting > [00:00:24] [01] [00:00:02] Builder started > [00:00:24] [01] [00:00:00] Building ports-mgmt/fallout | = fallout-1.0.4_8 > [00:01:04] [01] [00:00:40] Finished ports-mgmt/fallout | = fallout-1.0.4_8: Success > . . . >=20 > I do not expect the fetch issue is evidence of a problem. By omission, I was too vague about that. The log's error message was: go: golang.org/x/text@v0.3.7: read = "https:/proxy.golang.org/@v/v0.3.7.zip": read tcp = 192.168.1.110:47155->142.251.215.241:443: read: connection reset by peer > I'm counting this as: No evidence of corruption problems. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Apr 17 04:35:27 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0Dkf48jTz45Jm0 for ; Mon, 17 Apr 2023 04:35:38 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from mail.dawidek.net (mail.dawidek.net [94.130.64.56]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q0Dkc6j5tz49k7 for ; Mon, 17 Apr 2023 04:35:36 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 94.130.64.56 is neither permitted nor denied by domain of pjd@FreeBSD.org) smtp.mailfrom=pjd@FreeBSD.org; dmarc=none Received: from [192.168.250.133] (c-73-241-172-196.hsd1.ca.comcast.net [73.241.172.196]) by mail.dawidek.net (Postfix) with ESMTPSA id 18A2B4F93C; Mon, 17 Apr 2023 06:35:29 +0200 (CEST) Message-ID: <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> Date: Sun, 16 Apr 2023 21:35:27 -0700 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 Content-Language: en-US To: freebsd-current@freebsd.org References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> From: Pawel Jakub Dawidek In-Reply-To: <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [1.75 / 15.00]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_LONG(-1.00)[-0.999]; NEURAL_HAM_MEDIUM(-0.89)[-0.889]; NEURAL_SPAM_SHORT(0.14)[0.140]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; MIME_TRACE(0.00)[0:+]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; R_SPF_SOFTFAIL(0.00)[~all:c]; FREEFALL_USER(0.00)[pjd]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Rspamd-Queue-Id: 4Q0Dkc6j5tz49k7 X-Spamd-Bar: + X-ThisMailContainsUnwantedMimeParts: N On 4/16/23 01:07, Florian Smeets wrote: > On the pool that has block_cloning enabled I see the above insta panic > when poudriere starts building. I found a workaround though: > > --- /usr/local/share/poudriere/include/fs.sh.orig    2023-04-15 > 18:03:50.090823000 +0200 > +++ /usr/local/share/poudriere/include/fs.sh    2023-04-15 > 18:04:04.144736000 +0200 > @@ -295,7 +295,6 @@ >          fi > >          zfs clone -o mountpoint=${mnt} \ > -            -o sync=disabled \ >              -o atime=off \ >              -o compression=off \ >              ${fs}@${snap} \ > > With this workaround I was able to build thousands of packages without > panics or failures due to data corruption. Thank you, Florian, that was very helpful! This should fix the problem: https://github.com/openzfs/zfs/pull/14758 -- Pawel Jakub Dawidek From nobody Mon Apr 17 08:57:32 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0LY741f8z45bS7 for ; Mon, 17 Apr 2023 08:57:47 +0000 (UTC) (envelope-from fbl@aoek.com) Received: from mail.yourbox.net (mail.yourbox.net [IPv6:2001:41d0:1:767d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.yourbox.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0LY717TYz4NMp; Mon, 17 Apr 2023 08:57:46 +0000 (UTC) (envelope-from fbl@aoek.com) Authentication-Results: mx1.freebsd.org; none Received: from mail.yourbox.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail.yourbox.net (8.17.1/8.17.1) with ESMTP id 33H8vbuU054600; Mon, 17 Apr 2023 10:57:37 +0200 (CEST) (envelope-from fbl@aoek.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=aoek.com; s=mailbox; t=1681721857; bh=0caX5SZ9WdCAe47hdPEa0B9p4WQRuIk3dMmz5TLliyY=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=Jzb0GZnfLy4Lxi7deWQNfhGhqHoRMB0rsefFh+4eA1pTDCRhj9UUJ0r4jWnGKNbZo A2sLYlXH2AEjKDNS++9KJ67HE9IWbtMjLQYrHwVUY6orzdtdi2C2K5POicPILilJ42 7HoO3XkjDCX6sjHxJ7a3jRslvWKNv2AsilQtx+yxzdeckhgfuK6hAIbZXr+VzOLAeK 4NFJqj5r3i7oeHFc7npiq5knZ44WBN52Cw+LtVBZknVLS0ic7d24rdd2dINsb1oK/y F1T6WPYnRlsULcTFs4zTb5yjh2gBAFWppSMP/1uNEMVcKUNtKSUkv4fzefTCc38kr5 W3kW6OBHQYSlg== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 17 Apr 2023 10:57:32 +0200 From: =?UTF-8?Q?Jos=C3=A9_P=C3=A9rez?= To: Pawel Jakub Dawidek Cc: freebsd-current@freebsd.org Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 In-Reply-To: <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> Message-ID: X-Sender: fbl@aoek.com User-Agent: Roundcube Webmail/1.2.0 X-Rspamd-Queue-Id: 4Q0LY717TYz4NMp X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16276, ipnet:2001:41d0::/32, country:FR] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Hi Pawel, thank you for the patch. Can you please elaborate a little more? Did you run any tests? Is it safe to use your patch to access pools with feature@block_cloning active? Is it possible to build a kernel from such a pool? Asking for others: is this fixing any corrupted data? Thank you. BR, El 2023-04-17 06:35, Pawel Jakub Dawidek escribió: > On 4/16/23 01:07, Florian Smeets wrote: >> On the pool that has block_cloning enabled I see the above insta panic >> when poudriere starts building. I found a workaround though: >> >> --- /usr/local/share/poudriere/include/fs.sh.orig    2023-04-15 >> 18:03:50.090823000 +0200 >> +++ /usr/local/share/poudriere/include/fs.sh    2023-04-15 >> 18:04:04.144736000 +0200 >> @@ -295,7 +295,6 @@ >>          fi >> >>          zfs clone -o mountpoint=${mnt} \ >> -            -o sync=disabled \ >>              -o atime=off \ >>              -o compression=off \ >>              ${fs}@${snap} \ >> >> With this workaround I was able to build thousands of packages without >> panics or failures due to data corruption. > > Thank you, Florian, that was very helpful! > > This should fix the problem: > > https://github.com/openzfs/zfs/pull/14758 -- José Pérez From nobody Mon Apr 17 10:43:09 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0Ntp4Gc9z45jWW for ; Mon, 17 Apr 2023 10:43:14 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from mail.dawidek.net (mail.dawidek.net [94.130.64.56]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q0Ntn5yJHz43dc for ; Mon, 17 Apr 2023 10:43:13 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 94.130.64.56 is neither permitted nor denied by domain of pjd@FreeBSD.org) smtp.mailfrom=pjd@FreeBSD.org; dmarc=none Received: from [192.168.250.133] (c-73-241-172-196.hsd1.ca.comcast.net [73.241.172.196]) by mail.dawidek.net (Postfix) with ESMTPSA id 5D9C84F949; Mon, 17 Apr 2023 12:43:12 +0200 (CEST) Message-ID: <0164e42a-e7cd-a1e8-295c-21f414edf67b@dawidek.net> Date: Mon, 17 Apr 2023 03:43:09 -0700 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 Content-Language: en-US To: freebsd-current@freebsd.org References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> <01430095-33a3-a949-3772-2ec90b4c3fe6@dawidek.net> From: Pawel Jakub Dawidek In-Reply-To: <01430095-33a3-a949-3772-2ec90b4c3fe6@dawidek.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [2.45 / 15.00]; VIOLATED_DIRECT_SPF(3.50)[]; NEURAL_HAM_LONG(-1.00)[-0.996]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; NEURAL_HAM_MEDIUM(-0.08)[-0.083]; NEURAL_SPAM_SHORT(0.03)[0.027]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; R_SPF_SOFTFAIL(0.00)[~all:c]; FREEFALL_USER(0.00)[pjd]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Rspamd-Queue-Id: 4Q0Ntn5yJHz43dc X-Spamd-Bar: ++ X-ThisMailContainsUnwantedMimeParts: N On 4/17/23 18:15, Pawel Jakub Dawidek wrote: > There were three issues that I know of after the recent OpenZFS merge: > > 1. Data corruption unrelated to block cloning, so it can happen even > with block cloning disabled or not in use. This was the problematic commit: >     https://github.com/openzfs/zfs/commit/519851122b1703b8445ec17bc89b347cea965bb9 > > It was reverted in 63ee747febbf024be0aace61161241b53245449e. > > 2. Data corruption with embedded blocks when block cloning is enabled. > It can happen when compression is enabled and the block contains between > 60 to 112 bytes (this might be hard to determine). Fix exists, it is > merged to OpenZFS already, but isn't in FreeBSD yet. >     OpenZFS pull request: https://github.com/openzfs/zfs/pull/14739 > > 3. Panic on VERIFY(zil_replaying(zfsvfs->z_log, tx)). This is triggered > when block cloning is enabled, the sync property is set to disabled and > copy_file_range(2) is used. Easy fix exists, it is not yet merged to > OpenZFS and not yet in FreeBSD HEAD. >     OpenZFS pull request: https://github.com/openzfs/zfs/pull/14758 > > Block cloning was disabled in 46ac8f2e7d9601311eb9b3cd2fed138ff4a11a66, > so 2 and 3 should not occur. As of 068913e4ba3dd9b3067056e832cefc5ed264b5cc all known issues are fixed, as far as I can tell. Block cloning remains disabled for now just to be on the safe side, but can be enabled by setting sysctl vfs.zfs.bclone_enabled to 1. Don't relay on this sysctl as it will be removed in 2-3 weeks. -- Pawel Jakub Dawidek From nobody Mon Apr 17 12:28:40 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0RDc3Nm2z45qG0 for ; Mon, 17 Apr 2023 12:28:48 +0000 (UTC) (envelope-from fbl@aoek.com) Received: from mail.yourbox.net (mail.yourbox.net [IPv6:2001:41d0:1:767d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.yourbox.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0RDc0Ytnz3NHD; Mon, 17 Apr 2023 12:28:47 +0000 (UTC) (envelope-from fbl@aoek.com) Authentication-Results: mx1.freebsd.org; none Received: from mail.yourbox.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail.yourbox.net (8.17.1/8.17.1) with ESMTP id 33HCSjL5029378; Mon, 17 Apr 2023 14:28:45 +0200 (CEST) (envelope-from fbl@aoek.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=aoek.com; s=mailbox; t=1681734526; bh=VKyPY4x1en3p34OfbfXvO2HmGbDGxO+BCRv3U4VFkVM=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=NTC4+qBl7kvQg2+0K7sbzrRJapdZqitnFZdRbqG/T1kZJcYoAsVphpk3+BsjumqFN dyAiy8YBaTmMaWcawRggXSTw6tyR5r1mKaw1QIOFVBC+iN8/2JxaYpUzENuvFsqWLZ gYakDm6fQajqjCETZAzsGR8opjMMieQn6iFRbsQzuR4oWbCQcqfzP2ofuhtnJ1MCnk oFwmPQUqNvNT4/ohK8bTvgsjBGJYknIh1VT5PwayZTNlZld0FNkq7pzyezF6inzvT2 QsE8Zl4Qzd8z0op00h5d1U3EmuwCVvzQj403ID78wOIVAb55EFNMbC/LY3dea5skKq GnZ5ZJmtaC/uw== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 17 Apr 2023 14:28:40 +0200 From: =?UTF-8?Q?Jos=C3=A9_P=C3=A9rez?= To: Pawel Jakub Dawidek Cc: freebsd-current@freebsd.org Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 In-Reply-To: <0164e42a-e7cd-a1e8-295c-21f414edf67b@dawidek.net> References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413063321.60344b1f@cschubert.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> <01430095-33a3-a949-3772-2ec90b4c3fe6@dawidek.net> <0164e42a-e7cd-a1e8-295c-21f414edf67b@dawidek.net> Message-ID: X-Sender: fbl@aoek.com User-Agent: Roundcube Webmail/1.2.0 X-Rspamd-Queue-Id: 4Q0RDc0Ytnz3NHD X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16276, ipnet:2001:41d0::/32, country:FR] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N El 2023-04-17 12:43, Pawel Jakub Dawidek escribió: > On 4/17/23 18:15, Pawel Jakub Dawidek wrote: >> There were three issues that I know of after the recent OpenZFS merge: >> >> 1. Data corruption unrelated to block cloning, so it can happen even >> with block cloning disabled or not in use. This was the problematic >> commit: >> >>     https://github.com/openzfs/zfs/commit/519851122b1703b8445ec17bc89b347cea965bb9 >> >> It was reverted in 63ee747febbf024be0aace61161241b53245449e. >> >> 2. Data corruption with embedded blocks when block cloning is enabled. >> It can happen when compression is enabled and the block contains >> between 60 to 112 bytes (this might be hard to determine). Fix exists, >> it is merged to OpenZFS already, but isn't in FreeBSD yet. >>     OpenZFS pull request: https://github.com/openzfs/zfs/pull/14739 >> >> 3. Panic on VERIFY(zil_replaying(zfsvfs->z_log, tx)). This is >> triggered when block cloning is enabled, the sync property is set to >> disabled and copy_file_range(2) is used. Easy fix exists, it is not >> yet merged to OpenZFS and not yet in FreeBSD HEAD. >>     OpenZFS pull request: https://github.com/openzfs/zfs/pull/14758 >> >> Block cloning was disabled in >> 46ac8f2e7d9601311eb9b3cd2fed138ff4a11a66, so 2 and 3 should not occur. > > As of 068913e4ba3dd9b3067056e832cefc5ed264b5cc all known issues are > fixed, as far as I can tell. > > Block cloning remains disabled for now just to be on the safe side, > but can be enabled by setting sysctl vfs.zfs.bclone_enabled to 1. > > Don't relay on this sysctl as it will be removed in 2-3 weeks. Hi Pawel, thank you for your reply and for the fixes. I think there is a 4th issue that needs to be addressed: how do we recover from the worst case scenario which is a machine with a kernel > 2a58b312b62f and ZFS root upgraded with block cloning enabled. In particular, is it safe to turn such a machine on in the first place, and what are the risks involved in doing so? Any potential data loss? Would such a machine be able to fix itself by compiling a kernel, or would compilation fail and might data be corrupted in the process? I have two poudriere builders powered off (I am not alone in this situation) and I need to recover them, ideally minimizing data loss. The builders are also hosting current and used to build kernels and worlds for 13 and current: as of now all my production machines are stuck on the 13 they run, I cannot update binaries nor packages and I would like to be back online. Whatever the fixing procedure, it shall be outlined in the UPDATING document. Thank you. BR, -- José Pérez From nobody Mon Apr 17 13:45:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0Swv4L3fz45vmF for ; Mon, 17 Apr 2023 13:45:19 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-24.consmr.mail.gq1.yahoo.com (sonic303-24.consmr.mail.gq1.yahoo.com [98.137.64.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0Swt0gZMz3qXP for ; Mon, 17 Apr 2023 13:45:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="GH/2UHQg"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681739115; bh=m82l40pSy9aewkC/xNc4oiOhh2AvjDRYTytA7da+Qu0=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=GH/2UHQgHOXHXHY2WO8LAKAElGbxe3uj/lkgiL3rCCwsySqUH0heONzy8HsMahuj+04pofzv246yclM/ywcSQ+i4ED9x1iJoCM6uOzbt8oASkya519TPMzXQ2bLR9si8HUdHtprBOKV7Z0uQz4scQNm3buUIOhXW2HyPRQwe4VG51fMWWzPzsUKP1l+iu5uchJ4GD9hO8UfFzgmsDRK30U1Z7D63Q6giN2raKV8tV5CU/UuJMlukn0atn5hFVyi7CdQ24x8p3qa0XdHFXAhzILu+1RzpQvQd4vYJuFXvbCJCswbA1lxHHxcYVQfgUJI7ev4wc9mC6FAhxEhj5gv/fg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681739115; bh=WfxCAQqiwZ2g4uprMy641LUovY/3PKez4nzjwglG1hN=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=WfbF+0zD4g12W3XTTXftXl1jy0y3XNrdTzxYkAKMXn/xnznkCvDbqneBqk4leDccAbzCv4DsX/kufZbjADd1jL5NI2M9yAhI67rkEopofQDT2TMV6L2SMAzb+0ZppXdKoBpc3yVrN3OnEp8aTwr8wyBBN1uum92NbQiTyAJRmkO8cR++0QUOwpmP7QZMP+fhbl2cxMNZPLH+W+1cspYyY6jnN7t4rbWo3ihinLXOmcPiLKyriu0aojkodSIPed4csQLGnjOSdv+jQvMmqQDoIYAX1tOoHWvgsQZxTuEEgEYb+RyBIpNpyHUn7eJMZ5NPxcUX1Qn5nnYJAUKHyHXpMQ== X-YMail-OSG: IVfWDWsVM1l_G9CiOWZ9J2Ce7_jsuqJQ6wRp3RS9vVu8qoQ6nCr5ROSfi0zJAXt pBS6IsWLSOOBUubv6sO2d.4wCLB9zX3f8J90oT4syVqhQeC3S710r7cmnY0Pi.bU5V6oIpO5Nyvn SKYBkPhGwA0Kw6ncoayl75NEohmMRYsbmijcIcNlnDRzj8QXS.Cb3hYjdErSJx1nb08UcbrMkKUd 7C7UXL2rNKOHro7L_A385aa9oVslUCM9wAsuVvsfkg6FkuJqVRHg9xRzv2RgewwAuHlDOoUNvEan 1JXKQBM781f3dHi2E1bMc.hXyZjuY.Eo.15VX8K5pmdc41CxNs6GWMbRq0bUIdRRZYDSEQ7g496s PfQrXTz_Beq559n91_Jp0fUL_jsaX_iyURaeZTRt5AxYi3Xs.KsBMa_RMwYCXKVmK0b.9JOL3SCw Goh0PqPT7ZEnwVYzxZD4cRm8uIDTWy_jy.RZkh3IRQ2d5_mjXyXfhxGpEqVHxLGPIj5fHe6p.SYX 2vrDZOOBug46rf4J6CEU6.C8B3bdezQzJOQx.5KxCVj4x_kJ8tbl1nPo1jsIfd5Qi_elI3MebVCU f_Dz2dPIKScY21mqMomvj9qgJpEDENKsZB4VWZumgW1KmUzRYvxOcphLVUbsMOE2SzCWua2PN8aF 5j8WBP.d9RatzeR_KSM8LPxFePohQEd4eJHPbpkBRkLgnEOBYxqcSO0u2VouBJ7P5n_GuPoGqkIf T54aDE1sthC7PLoZ27iUy5uIQGbVFZ9ZsrGHTcIrSiRT2zyHx3csSlJljwQviSCfafeizPEt1uxt 3gn0Gr5tiTTTuamrm5PME2LJNWfUjq.C7Eo4qAO4kECOlFJDOtbkp_WAvGsZM5yWadILvwyewbu6 pSVAUZ4UiXzaH1r3VvAkXx4A.G1NuQDxRjHPZoU.uzHNGIRSIGTtGMsSI3iUx4wmTTr0Kge1C4GX VXC7_mkq0BqQ71Rw0aPHrbFUWJHYq7zZ2vbWv4VUmLIjWuLZ3UN2btZOEOSf2wXDX4FYh07_aCf2 LelzAucxlFTb0yduyYe7kbxmLgYCzHqPAu0XldgewVQXhyQP6GEZwoOJ5bYVJ9OLq6MoUJYAY1zX ev3AA_FgV1ZrY8Y94a58Klg1b9aZPw5QlquFFseG5Eph2L7wtQIZlwhHk8WuWUD4_.IqZpGjmTbD MtKYCUUbGNudJdpjECUM5mEhFNz.5FYU6.imeblxF8tiiPS7f8aiYsycNTqGW.ypEtNnPGIF9_8Y kUNFDnFRnzZyBZYJbK0fyRhMLSnki5xMuersdJYeMfTnmiP1odFkp.oQaTF9arY8PjHdG9u8VjJX jirs6i1NdK0qdH7Rc1mdXdNHHXcuOf1Ua4VP4DVCVT8XDrIY2rkQmd3xVFL4PG.mHYnpXPDTVYBC E9fte7lggZNwpSqdMmddnksM_Vnw.1qx_4gHiN2YlmYQTg.Vi.bFlNUgZYBrlorBJFB3qXOuSsTY F2nc4vVibbp2XeMjw9yo4TzH8tLVQG6WEBHtSB7kgmDzqBNUpIrDg7.R2w9PtlxAxI7eJO.b3hI8 ON_K6baeJXQ02vlUGDkV0Vp8VfkNoAtRu4UL598OAe.TzEKewxrK2FhPHmCznNvq6D11cizUhiy6 Y1rxfe6uo7RlLnzpJJ8DIrBdCpmjJG.TBA3h9g_GXyfXARkEc9Bbhgu7dv63UGbiD3jtjWf0RK0y yf70VJkOh4QSmQd3c97LRKJGdpBfT3Z7ZLU9x0NG_v1IVUpdJmABh.A3sEwSI.YVCWlPWhx32c5D ldi7bGABX0b8ICfdnrsS.EahAGAkegesqQhzqwXVn.1n5stsNZeXSx8SEMmfAKWxiDZKf599zdA. kcD5_eaV4k8oxVfYu2EqEOqiSmig0sh6tbp_Z9OIAIRXh6T4S4117E.izg7aXB5JipCRC5IzvZ1x 3FkDIACYUa1sr.73U9VKXvuySJdbuI_OBm.i9_4Qfh.IuyilLwSyrpMqM4Y3tjhsMpeV6V_Q_jhK XzChFO5RSySUO2z1MxJZ59ff7w1MMW2B8kukYQZw1kOS6ptu42QI3xtZjfnLBdf_scue7H8X9EJc p6NsYDs3fpLJR1REGRlaihnfDATll441PI8zdNLyAmIWDKMvIZkXlZYSOaPvCeLaN3aqqi7XfJEY A8RPUDwNGh9IDq4FpMXypB2IWxe1ypnQZ14iJROugwCVH8J5TzgaqgTL2SQv5Kdr_j0dslHYLizZ Dntfo7gbDTJU0eMFZ2R8HIR1jmhjpMC5mSoBrWgtxH85b19T3qggYR_EOcJJMuIqikhebeLswXJa Rs56Klw-- X-Sonic-MF: X-Sonic-ID: 1d412532-860f-40a4-a47a-addadcf09552 Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Mon, 17 Apr 2023 13:45:15 +0000 Received: by hermes--production-ne1-7dbd98dd99-gdhzk (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID cff7dca568c16d032836b6d985b776c0; Mon, 17 Apr 2023 13:45:13 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 Message-Id: <61987FE2-BE4E-45C5-A731-C7C6EED4D875@yahoo.com> Date: Mon, 17 Apr 2023 06:45:01 -0700 To: =?utf-8?B?Sm9zw6kgUMOpcmV6?= , Pawel Jakub Dawidek , Current FreeBSD X-Mailer: Apple Mail (2.3731.400.51.1.1) References: <61987FE2-BE4E-45C5-A731-C7C6EED4D875.ref@yahoo.com> X-Spamd-Result: default: False [-3.47 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.972]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.205:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4Q0Swt0gZMz3qXP X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Jos=C3=A9_P=C3=A9rez wrote on Date: Mon, 17 Apr 2023 12:28:40 UTC : > El 2023-04-17 12:43, Pawel Jakub Dawidek escribi=C3=B3: > > On 4/17/23 18:15, Pawel Jakub Dawidek wrote: > >> There were three issues that I know of after the recent OpenZFS = merge: > >>=20 > >> 1. Data corruption unrelated to block cloning, so it can happen = even=20 > >> with block cloning disabled or not in use. This was the problematic=20= > >> commit: > >>=20 > >> = https://github.com/openzfs/zfs/commit/519851122b1703b8445ec17bc89b347cea96= 5bb9 > >>=20 > >> It was reverted in 63ee747febbf024be0aace61161241b53245449e. > >>=20 > >> 2. Data corruption with embedded blocks when block cloning is = enabled.=20 > >> It can happen when compression is enabled and the block contains=20 > >> between 60 to 112 bytes (this might be hard to determine). Fix = exists,=20 > >> it is merged to OpenZFS already, but isn't in FreeBSD yet. > >> OpenZFS pull request: https://github.com/openzfs/zfs/pull/14739 > >>=20 > >> 3. Panic on VERIFY(zil_replaying(zfsvfs->z_log, tx)). This is=20 > >> triggered when block cloning is enabled, the sync property is set = to=20 > >> disabled and copy_file_range(2) is used. Easy fix exists, it is not=20= > >> yet merged to OpenZFS and not yet in FreeBSD HEAD. > >> OpenZFS pull request: https://github.com/openzfs/zfs/pull/14758 > >>=20 > >> Block cloning was disabled in=20 > >> 46ac8f2e7d9601311eb9b3cd2fed138ff4a11a66, so 2 and 3 should not = occur. > >=20 > > As of 068913e4ba3dd9b3067056e832cefc5ed264b5cc all known issues are > > fixed, as far as I can tell. > >=20 > > Block cloning remains disabled for now just to be on the safe side, > > but can be enabled by setting sysctl vfs.zfs.bclone_enabled to 1. > >=20 > > Don't relay on this sysctl as it will be removed in 2-3 weeks. >=20 > Hi Pawel, > thank you for your reply and for the fixes. >=20 > I think there is a 4th issue that needs to be addressed: how do we=20 > recover from the worst case scenario which is a machine with a kernel = >=20 > 2a58b312b62f and ZFS root upgraded with block cloning enabled. >=20 > In particular, is it safe to turn such a machine on in the first = place,=20 > and what are the risks involved in doing so? Any potential data loss? >=20 > Would such a machine be able to fix itself by compiling a kernel, or=20= > would compilation fail and might data be corrupted in the process? >=20 > I have two poudriere builders powered off (I am not alone in this=20 > situation) and I need to recover them, ideally minimizing data loss. = The=20 > builders are also hosting current and used to build kernels and worlds=20= > for 13 and current: as of now all my production machines are stuck on=20= > the 13 they run, I cannot update binaries nor packages and I would = like=20 > to be back online. >=20 > Whatever the fixing procedure, it shall be outlined in the UPDATING=20 > document. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D270811 is an example issue where a FreeBSD powerpc package building server can not boot --after patching so it no longer gets a boot time "panic: floating-point unavailable trap" (that jhibbits patch is still not committed): QUOTE from the description: . . . nda1: 953869MB (1953525168 512 byte sectors) GEOM_MIRROR: Device mirror/swap0 launched (2/2). Mounting from zfs:zroot failed with error 6; retrying for 3 more seconds Mounting from zfs:zroot failed with error 6. Loader variables: vfs.root.mountfrom=3Dzfs:zroot Manual root filesystem specification: : [options] Mount using filesystem and with the specified (optional) option list. eg. ufs:/dev/da0s1a zfs:zroot/ROOT/default cd9660:/dev/cd0 ro (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /) ? List valid disk boot devices . Yield 1 second (for background tasks) Abort manual input mountroot> This machine is part of the FreeBSD cluster for building PowerPC = packages, so we can build kernels to test anytime necessary. END QUOTE =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Apr 17 18:51:05 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0bjm4P9lz4531L for ; Mon, 17 Apr 2023 18:51:08 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oi1-x233.google.com (mail-oi1-x233.google.com [IPv6:2607:f8b0:4864:20::233]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0bjm095Tz40M4; Mon, 17 Apr 2023 18:51:08 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=OU2AbQh6; spf=pass (mx1.freebsd.org: domain of mjguzik@gmail.com designates 2607:f8b0:4864:20::233 as permitted sender) smtp.mailfrom=mjguzik@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-oi1-x233.google.com with SMTP id j12so17460094oij.3; Mon, 17 Apr 2023 11:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681757466; x=1684349466; h=cc:to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=OOADMwcRUH/jyAP85Hg4yCRdIpLfv19E5UI8eNSNqt8=; b=OU2AbQh6U0JAAUToXORe4hM51bVjH9WXW8QzPEFde5A6iPEFA55k300C70wKnZllGa jpCXDPJVALr1lWM6HcsmS03LYMGMMsobWgEkAZjSntQSn/HvRa3vOQMVMWQ7sReUAzoP tCvXXgog233Rto4oLw3CNGRLwWCEt7d8u/S2qYfspdUwsYibUHr1ZK9tWX9KViADRikR nBl0Tgk/nGsQhd8S1q/RZ2bTaO6jFaUB3VfYr9tOQum3TA8Cq8tUc1KdF3m52IgWRJNy DOiZ6JLV/en2m6pF3ep9FZeQl9390VYHQh+tmgnha7U9qJPHsnAUnPccoCszQyecUQnh Mapw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681757466; x=1684349466; h=cc:to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=OOADMwcRUH/jyAP85Hg4yCRdIpLfv19E5UI8eNSNqt8=; b=Rhl/z/gZbBR7SfQzco3bCCsRIbk/4YaDlQPeeFgRM2I9EBPF6miS2tweh1IC2KSGjW WCsepgmDAfUgWGPI8JJHOkZMuT028OAOiS1HNXIoOt3p3lolUAG446yQsz3fSl7/rVA2 6f3N83ggPrgv74tNoOwwMPPL/6VgoIVy/C8wUNvAtZTT2ZCA8uPXUJxWjtRJ78Qpl6i6 cJpFDG0XQVmVaIChgTc/dbCC5pO6QCkBBFhW6Vx7i4uDW8vVTsVBIM4NMIv2ca0/Up6N t1Z2YJrRoo7ybJXFLgCfT3eE9Fu08tYHOOj2k6d/malhqvq/vvIjN28vUflBVaJkTPfr 8Svw== X-Gm-Message-State: AAQBX9duBVAExZMr9E9XnzQ4+oyTvoq2P8Z7XXImv+3VyxzhbnFsF8Rb BO2Lm2C9/OlvD9P9QO77XpG3wDZSr6XrdXOugaE3UsOR X-Google-Smtp-Source: AKy350YPSLTKgyBH0XWV/Bf7nsQn7EOhOvaw2fm2dDBM4ClQszOKTvETlVRcGMFpEs0nkQDI7gVExjuwAovPDAdWFpY= X-Received: by 2002:aca:f043:0:b0:38e:30:1225 with SMTP id o64-20020acaf043000000b0038e00301225mr961960oih.4.1681757466033; Mon, 17 Apr 2023 11:51:06 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:a8a:46:0:b0:49c:b071:b1e3 with HTTP; Mon, 17 Apr 2023 11:51:05 -0700 (PDT) From: Mateusz Guzik Date: Mon, 17 Apr 2023 20:51:05 +0200 Message-ID: Subject: another crash and going forward with zfs To: freebsd-current@freebsd.org Cc: Pawel Jakub Dawidek , Glen Barber Content-Type: text/plain; charset="UTF-8" X-Spamd-Result: default: False [-3.97 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.97)[-0.973]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::233:from]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; FREEMAIL_FROM(0.00)[gmail.com]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-Rspamd-Queue-Id: 4Q0bjm095Tz40M4 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N After bugfixes got committed I decided to zpool upgrade and sysctl vfs.zfs.bclone_enabled=1 vs poudriere for testing purposes. I very quickly got a new crash: panic: VERIFY(arc_released(db->db_buf)) failed cpuid = 9 time = 1681755046 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0a90b8e5f0 vpanic() at vpanic+0x152/frame 0xfffffe0a90b8e640 spl_panic() at spl_panic+0x3a/frame 0xfffffe0a90b8e6a0 dbuf_redirty() at dbuf_redirty+0xbd/frame 0xfffffe0a90b8e6c0 dmu_buf_will_dirty_impl() at dmu_buf_will_dirty_impl+0xa2/frame 0xfffffe0a90b8e700 dmu_write_uio_dnode() at dmu_write_uio_dnode+0xe9/frame 0xfffffe0a90b8e780 dmu_write_uio_dbuf() at dmu_write_uio_dbuf+0x42/frame 0xfffffe0a90b8e7b0 zfs_write() at zfs_write+0x672/frame 0xfffffe0a90b8e960 zfs_freebsd_write() at zfs_freebsd_write+0x39/frame 0xfffffe0a90b8e980 VOP_WRITE_APV() at VOP_WRITE_APV+0xdb/frame 0xfffffe0a90b8ea90 vn_write() at vn_write+0x325/frame 0xfffffe0a90b8eb20 vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0a90b8eb80 vn_io_fault1() at vn_io_fault1+0x161/frame 0xfffffe0a90b8ecc0 vn_io_fault() at vn_io_fault+0x1b5/frame 0xfffffe0a90b8ed40 dofilewrite() at dofilewrite+0x81/frame 0xfffffe0a90b8ed90 sys_write() at sys_write+0xc0/frame 0xfffffe0a90b8ee00 amd64_syscall() at amd64_syscall+0x157/frame 0xfffffe0a90b8ef30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0a90b8ef30 --- syscall (4, FreeBSD ELF64, write), rip = 0x103cddf7949a, rsp = 0x103cdc85dd48, rbp = 0x103cdc85dd80 --- KDB: enter: panic [ thread pid 95000 tid 135035 ] Stopped at kdb_enter+0x32: movq $0,0x9e4153(%rip) The posted 14.0 schedule which plans to branch stable/14 on May 12 and one cannot bet on the feature getting beaten up into production shape by that time. Given whatever non-block_clonning and not even zfs bugs which are likely to come out I think this makes the feature a non-starter for said release. I note: 1. the current problems did not make it into stable branches. 2. there was block_cloning-related data corruption (fixed) and there may be more 3. there was unrelated data corruption (see https://github.com/openzfs/zfs/issues/14753), sorted out by reverting the problematic commit in FreeBSD, not yet sorted out upstream As such people's data may be partially hosed as is. Consequently the proposed plan is as follows: 1. whack the block cloning feature for the time being, but make sure pools which upgraded to it can be mounted read-only 2. run ztest and whatever other stress testing on FreeBSD, along with restoring openzfs CI -- I can do the first part, I'm sure pho will not mind to run some tests of his own 3. recommend people create new pools and restore data from backup. if restoring from backup is not an option, tar or cp (not zfs send) from the read-only mount block cloning beaten into shape would use block_cloning_v2 or whatever else, key point that the current feature name would be considered bogus (not blocking RO import though) to prevent RW usage of the current pools with it enabled. Comments? -- Mateusz Guzik From nobody Mon Apr 17 19:41:26 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0cqw4qhTz456g4 for ; Mon, 17 Apr 2023 19:41:32 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from mail.dawidek.net (mail.dawidek.net [94.130.64.56]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q0cqw2gR7z4J7r; Mon, 17 Apr 2023 19:41:32 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.250.133] (c-73-241-172-196.hsd1.ca.comcast.net [73.241.172.196]) by mail.dawidek.net (Postfix) with ESMTPSA id 880DC4FAC4; Mon, 17 Apr 2023 21:41:29 +0200 (CEST) Message-ID: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> Date: Mon, 17 Apr 2023 12:41:26 -0700 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: another crash and going forward with zfs Content-Language: en-US To: Mateusz Guzik , freebsd-current@freebsd.org Cc: Glen Barber References: From: Pawel Jakub Dawidek In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q0cqw2gR7z4J7r X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/18/23 03:51, Mateusz Guzik wrote: > After bugfixes got committed I decided to zpool upgrade and sysctl > vfs.zfs.bclone_enabled=1 vs poudriere for testing purposes. I very > quickly got a new crash: > > panic: VERIFY(arc_released(db->db_buf)) failed > > cpuid = 9 > time = 1681755046 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0a90b8e5f0 > vpanic() at vpanic+0x152/frame 0xfffffe0a90b8e640 > spl_panic() at spl_panic+0x3a/frame 0xfffffe0a90b8e6a0 > dbuf_redirty() at dbuf_redirty+0xbd/frame 0xfffffe0a90b8e6c0 > dmu_buf_will_dirty_impl() at dmu_buf_will_dirty_impl+0xa2/frame > 0xfffffe0a90b8e700 > dmu_write_uio_dnode() at dmu_write_uio_dnode+0xe9/frame 0xfffffe0a90b8e780 > dmu_write_uio_dbuf() at dmu_write_uio_dbuf+0x42/frame 0xfffffe0a90b8e7b0 > zfs_write() at zfs_write+0x672/frame 0xfffffe0a90b8e960 > zfs_freebsd_write() at zfs_freebsd_write+0x39/frame 0xfffffe0a90b8e980 > VOP_WRITE_APV() at VOP_WRITE_APV+0xdb/frame 0xfffffe0a90b8ea90 > vn_write() at vn_write+0x325/frame 0xfffffe0a90b8eb20 > vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0a90b8eb80 > vn_io_fault1() at vn_io_fault1+0x161/frame 0xfffffe0a90b8ecc0 > vn_io_fault() at vn_io_fault+0x1b5/frame 0xfffffe0a90b8ed40 > dofilewrite() at dofilewrite+0x81/frame 0xfffffe0a90b8ed90 > sys_write() at sys_write+0xc0/frame 0xfffffe0a90b8ee00 > amd64_syscall() at amd64_syscall+0x157/frame 0xfffffe0a90b8ef30 > fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe0a90b8ef30 > --- syscall (4, FreeBSD ELF64, write), rip = 0x103cddf7949a, rsp = > 0x103cdc85dd48, rbp = 0x103cdc85dd80 --- > KDB: enter: panic > [ thread pid 95000 tid 135035 ] > Stopped at kdb_enter+0x32: movq $0,0x9e4153(%rip) > > The posted 14.0 schedule which plans to branch stable/14 on May 12 and > one cannot bet on the feature getting beaten up into production shape > by that time. Given whatever non-block_clonning and not even zfs bugs > which are likely to come out I think this makes the feature a > non-starter for said release. > > I note: > 1. the current problems did not make it into stable branches. > 2. there was block_cloning-related data corruption (fixed) and there may be more > 3. there was unrelated data corruption (see > https://github.com/openzfs/zfs/issues/14753), sorted out by reverting > the problematic commit in FreeBSD, not yet sorted out upstream > > As such people's data may be partially hosed as is. > > Consequently the proposed plan is as follows: > 1. whack the block cloning feature for the time being, but make sure > pools which upgraded to it can be mounted read-only > 2. run ztest and whatever other stress testing on FreeBSD, along with > restoring openzfs CI -- I can do the first part, I'm sure pho will not > mind to run some tests of his own > 3. recommend people create new pools and restore data from backup. if > restoring from backup is not an option, tar or cp (not zfs send) from > the read-only mount > > block cloning beaten into shape would use block_cloning_v2 or whatever > else, key point that the current feature name would be considered > bogus (not blocking RO import though) to prevent RW usage of the > current pools with it enabled. > > Comments? Correct me if I'm wrong, but from my understanding there were zero problems with block cloning when it wasn't in use or now disabled. The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to exactly avoid mess like this and give us more time to sort all the problems out while making it easy for people to try it. If there is no plan to revert the whole import, I don't see what value removing just block cloning will bring if it is now disabled by default and didn't cause any problems when disabled. -- Pawel Jakub Dawidek From nobody Mon Apr 17 19:59:14 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0dLv1m76z457nb for ; Mon, 17 Apr 2023 20:04:55 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from mail.dawidek.net (mail.dawidek.net [94.130.64.56]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q0dLt6yzXz3Qv7 for ; Mon, 17 Apr 2023 20:04:54 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.250.133] (c-73-241-172-196.hsd1.ca.comcast.net [73.241.172.196]) by mail.dawidek.net (Postfix) with ESMTPSA id 920784F9D3; Mon, 17 Apr 2023 21:59:16 +0200 (CEST) Message-ID: Date: Mon, 17 Apr 2023 12:59:14 -0700 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 Content-Language: en-US To: =?UTF-8?B?Sm9zw6kgUMOpcmV6?= Cc: freebsd-current@freebsd.org References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> <01430095-33a3-a949-3772-2ec90b4c3fe6@dawidek.net> <0164e42a-e7cd-a1e8-295c-21f414edf67b@dawidek.net> From: Pawel Jakub Dawidek In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Q0dLt6yzXz3Qv7 X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/17/23 21:28, José Pérez wrote: > Hi Pawel, > thank you for your reply and for the fixes. > > I think there is a 4th issue that needs to be addressed: how do we > recover from the worst case scenario which is a machine with a kernel > > 2a58b312b62f and ZFS root upgraded with block cloning enabled. > > In particular, is it safe to turn such a machine on in the first place, > and what are the risks involved in doing so? Any potential data loss? > > Would such a machine be able to fix itself by compiling a kernel, or > would compilation fail and might data be corrupted in the process? > > I have two poudriere builders powered off (I am not alone in this > situation) and I need to recover them, ideally minimizing data loss. The > builders are also hosting current and used to build kernels and worlds > for 13 and current: as of now all my production machines are stuck on > the 13 they run, I cannot update binaries nor packages and I would like > to be back online. José, I can only speak of block cloning in details, but I'll try to address everything. The easiest way to avoid block_cloning-related corruption on the kernel after the last OpenZFS merge, but before e0bb199925 is to set the compress property to 'off' and the sync property to something other than 'disabled'. This will avoid the block_cloning-related corruption and zil_replaying() panic. As for the other corruption, unfortunately I don't know the details, but my understanding is that it is happening under higher load. Not sure I'd trust a kernel built on a machine with this bug present. What I would do is to compile the kernel as of 068913e4ba somewhere else, boot the problematic machine in single-user mode and install the newly built kernel. As far as I can tell, contrary to some initial reports, none of the problems introduced by the recent OpenZFS merge corrupt the pool metadata, only file's data. You can locate the files modified with the bogus kernel using find(1) with a proper modification time, but you have to decide what to do with them (either throw them away, restore them from backup or inspect them). -- Pawel Jakub Dawidek From nobody Mon Apr 17 20:14:43 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0dZJ6Yzxz458B7 for ; Mon, 17 Apr 2023 20:14:48 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oa1-x2b.google.com (mail-oa1-x2b.google.com [IPv6:2001:4860:4864:20::2b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0dZG6XSBz3xgC; Mon, 17 Apr 2023 20:14:46 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-1879502e2afso15805790fac.5; Mon, 17 Apr 2023 13:14:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681762484; x=1684354484; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=WfEO+uCynKjLNwzp6HdsoFetG/Svoy+9wbtn0CuOOR8=; b=SQ14lBCxafbYcDbeOw2wL801GTL9dqWp7qEueDyHWSBE6faPbh3DRoa9qBheH3Nwo5 dBRoT0D0xo0rMuETWJZfsoq+972CdyTnR6GfbDxdMVtiSJ7g179Yd+hkLEJOeNBFO3Nv R8qwkh9stfU+fw2JP5n0rwbi5Wtf6W6AvELSdzjnWszPT8j2wJ6EHyKzkd35RF9a18Ig /8WuWba4XSKyInAT+BmgWgpFmU9YPp9KPx4K8SFTADiMZwjNi8UKj6KgjnMiXcIvKMZj hEuFE4cDwIiX7GK6rwss40T1m/BF5xkXss/BGea6Fp0O8asihLd0tMVY44bmKD8pGPeY ISig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681762484; x=1684354484; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WfEO+uCynKjLNwzp6HdsoFetG/Svoy+9wbtn0CuOOR8=; b=kg7ZbwAJAJSiWU9e3QW4qEXFZp2elxKPka2StivUF/RuYH9Yno0gC8GKB9vVmpOzKr dkQJivylaQXNLKLHnzHQhV95O0cTezrviqhxh1HQl4/PjtLnCyiUqeoohK01Bg8aN8Le xZri5mey+VS+YhWzNLJVqfQ40+UKz6nXYX6KFnRDKXKDDXRk25E0/RbkSduFNH9ocNOv AD2oq/UtPYamC7Q1x9/jeFL9t+6lQRdLITbAFoyzlp7tddKaD8vHe0gM8HKY6KijTpTl +xzSwzHawHAIkKCvVCZT6k0rcuwR3UbhMZ9IS5nt0mEPFhOabi/OAcwKnRZ3ySP6l4cf FKPw== X-Gm-Message-State: AAQBX9e3jgOXvEcshiqGPad5G4tEuGzJauF1F75tZpDNn8a+PZth8lji J6ujdggt+w/ke4lykJJTq0kJ5anAJJzjtRgGfXq95LIT X-Google-Smtp-Source: AKy350bGHPVj+J9VlFjr9a95MWS49+UPeU/yt7kv8lPi/YqdmnIPXZQqSp/hjjf9+cWxiQgM2KSSxCAQiJFeE4OKd/A= X-Received: by 2002:a05:6870:b513:b0:187:8ace:7427 with SMTP id v19-20020a056870b51300b001878ace7427mr7119870oap.4.1681762484573; Mon, 17 Apr 2023 13:14:44 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:a8a:11cf:0:b0:49c:b071:b1e3 with HTTP; Mon, 17 Apr 2023 13:14:43 -0700 (PDT) In-Reply-To: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> From: Mateusz Guzik Date: Mon, 17 Apr 2023 22:14:43 +0200 Message-ID: Subject: Re: another crash and going forward with zfs To: Pawel Jakub Dawidek Cc: freebsd-current@freebsd.org, Glen Barber Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Q0dZG6XSBz3xgC X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/17/23, Pawel Jakub Dawidek wrote: > On 4/18/23 03:51, Mateusz Guzik wrote: >> After bugfixes got committed I decided to zpool upgrade and sysctl >> vfs.zfs.bclone_enabled=1 vs poudriere for testing purposes. I very >> quickly got a new crash: >> >> panic: VERIFY(arc_released(db->db_buf)) failed >> >> cpuid = 9 >> time = 1681755046 >> KDB: stack backtrace: >> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >> 0xfffffe0a90b8e5f0 >> vpanic() at vpanic+0x152/frame 0xfffffe0a90b8e640 >> spl_panic() at spl_panic+0x3a/frame 0xfffffe0a90b8e6a0 >> dbuf_redirty() at dbuf_redirty+0xbd/frame 0xfffffe0a90b8e6c0 >> dmu_buf_will_dirty_impl() at dmu_buf_will_dirty_impl+0xa2/frame >> 0xfffffe0a90b8e700 >> dmu_write_uio_dnode() at dmu_write_uio_dnode+0xe9/frame >> 0xfffffe0a90b8e780 >> dmu_write_uio_dbuf() at dmu_write_uio_dbuf+0x42/frame 0xfffffe0a90b8e7b0 >> zfs_write() at zfs_write+0x672/frame 0xfffffe0a90b8e960 >> zfs_freebsd_write() at zfs_freebsd_write+0x39/frame 0xfffffe0a90b8e980 >> VOP_WRITE_APV() at VOP_WRITE_APV+0xdb/frame 0xfffffe0a90b8ea90 >> vn_write() at vn_write+0x325/frame 0xfffffe0a90b8eb20 >> vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0a90b8eb80 >> vn_io_fault1() at vn_io_fault1+0x161/frame 0xfffffe0a90b8ecc0 >> vn_io_fault() at vn_io_fault+0x1b5/frame 0xfffffe0a90b8ed40 >> dofilewrite() at dofilewrite+0x81/frame 0xfffffe0a90b8ed90 >> sys_write() at sys_write+0xc0/frame 0xfffffe0a90b8ee00 >> amd64_syscall() at amd64_syscall+0x157/frame 0xfffffe0a90b8ef30 >> fast_syscall_common() at fast_syscall_common+0xf8/frame >> 0xfffffe0a90b8ef30 >> --- syscall (4, FreeBSD ELF64, write), rip = 0x103cddf7949a, rsp = >> 0x103cdc85dd48, rbp = 0x103cdc85dd80 --- >> KDB: enter: panic >> [ thread pid 95000 tid 135035 ] >> Stopped at kdb_enter+0x32: movq $0,0x9e4153(%rip) >> >> The posted 14.0 schedule which plans to branch stable/14 on May 12 and >> one cannot bet on the feature getting beaten up into production shape >> by that time. Given whatever non-block_clonning and not even zfs bugs >> which are likely to come out I think this makes the feature a >> non-starter for said release. >> >> I note: >> 1. the current problems did not make it into stable branches. >> 2. there was block_cloning-related data corruption (fixed) and there may >> be more >> 3. there was unrelated data corruption (see >> https://github.com/openzfs/zfs/issues/14753), sorted out by reverting >> the problematic commit in FreeBSD, not yet sorted out upstream >> >> As such people's data may be partially hosed as is. >> >> Consequently the proposed plan is as follows: >> 1. whack the block cloning feature for the time being, but make sure >> pools which upgraded to it can be mounted read-only >> 2. run ztest and whatever other stress testing on FreeBSD, along with >> restoring openzfs CI -- I can do the first part, I'm sure pho will not >> mind to run some tests of his own >> 3. recommend people create new pools and restore data from backup. if >> restoring from backup is not an option, tar or cp (not zfs send) from >> the read-only mount >> >> block cloning beaten into shape would use block_cloning_v2 or whatever >> else, key point that the current feature name would be considered >> bogus (not blocking RO import though) to prevent RW usage of the >> current pools with it enabled. >> >> Comments? > > Correct me if I'm wrong, but from my understanding there were zero > problems with block cloning when it wasn't in use or now disabled. > > The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to exactly > avoid mess like this and give us more time to sort all the problems out > while making it easy for people to try it. > > If there is no plan to revert the whole import, I don't see what value > removing just block cloning will bring if it is now disabled by default > and didn't cause any problems when disabled. > The feature definitely was not properly stress tested and what not and trying to do it keeps running into panics. Given the complexity of the feature I would expect there are many bug lurking, some of which possibly related to the on disk format. Not having to deal with any of this is can be arranged as described above and is imo the most sensible route given the timeline for 14.0 -- Mateusz Guzik From nobody Mon Apr 17 21:06:27 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0fk02hKGz45CM6 for ; Mon, 17 Apr 2023 21:06:32 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0fjy75SDz3DXJ for ; Mon, 17 Apr 2023 21:06:30 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of phk@critter.freebsd.dk designates 130.225.244.222 as permitted sender) smtp.mailfrom=phk@critter.freebsd.dk; dmarc=none Received: from critter.freebsd.dk (unknown [192.168.55.3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by phk.freebsd.dk (Postfix) with ESMTPS id 651A589281 for ; Mon, 17 Apr 2023 21:06:28 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.17.1/8.16.1) with ESMTPS id 33HL6SwA051408 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO) for ; Mon, 17 Apr 2023 21:06:28 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.17.1/8.16.1/Submit) id 33HL6RUX051407; Mon, 17 Apr 2023 21:06:27 GMT (envelope-from phk) Message-Id: <202304172106.33HL6RUX051407@critter.freebsd.dk> To: current@freebsd.org Subject: find(1): I18N gone wild ? From: Poul-Henning Kamp List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-ID: <51405.1681765587.1@critter.freebsd.dk> Date: Mon, 17 Apr 2023 21:06:27 +0000 X-Spamd-Result: default: False [-0.39 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-0.99)[-0.992]; NEURAL_HAM_SHORT(-0.98)[-0.984]; NEURAL_SPAM_MEDIUM(0.58)[0.581]; FORGED_SENDER(0.30)[phk@phk.freebsd.dk,phk@critter.freebsd.dk]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[current@freebsd.org]; ASN(0.00)[asn:1835, ipnet:130.225.0.0/16, country:EU]; MID_RHS_MATCH_FROMTLD(0.00)[]; MIME_TRACE(0.00)[0:+]; DMARC_NA(0.00)[freebsd.dk]; RCVD_COUNT_THREE(0.00)[4]; RCVD_TLS_LAST(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEFALL_USER(0.00)[phk]; ARC_NA(0.00)[]; R_DKIM_NA(0.00)[]; FROM_HAS_DN(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; FROM_NEQ_ENVFROM(0.00)[phk@phk.freebsd.dk,phk@critter.freebsd.dk] X-Rspamd-Queue-Id: 4Q0fjy75SDz3DXJ X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N This surprised me: # mkdir /tmp/P # cd /tmp/P # touch FOO # touch bar # env LANG=C.UTF-8 find . -name '[A-Z]*' -print ./FOO # env LANG=en_US.UTF-8 find . -name '[A-Z]*' -print ./FOO ./bar Really ?! -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From nobody Mon Apr 17 21:24:59 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0g7Y30Pcz45DQ7 for ; Mon, 17 Apr 2023 21:25:13 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-io1-xd29.google.com (mail-io1-xd29.google.com [IPv6:2607:f8b0:4864:20::d29]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0g7X4bXlz43SL for ; Mon, 17 Apr 2023 21:25:12 +0000 (UTC) (envelope-from delphij@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-io1-xd29.google.com with SMTP id s6so2038849iow.11 for ; Mon, 17 Apr 2023 14:25:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681766711; x=1684358711; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=Z2hVXUd9jTQCUq7oB7gNxYi6a7Eio5tQcu8W27K+sdk=; b=AJQRsNG7z6KHqF0C+AKciWU1PqhPLLKA5Ifu5y2HyRJTxeNrLsIF2PYMDr/MOhakCB 00HaAYxqZ6uKLBUpdGBRI/Vpw1sleUj7l2o6rkBduoHyrJaN9pzMsauWOlcua0nAzwUl hyI7oRto5FnHelcf0Y+bX8y0ZMb9GUE98K791CEuZCylCdjkDnfyM+Xj4SrNXRNbbHm0 JCJefW/Y5q5/NzWyrWyoNL0jCml03OLWZqzRuQvUWjE/LBTveiP+hJhq2N5Td5bl8zsz 5Q0h/Tbk9X3+lGDSJVgh8RAMcHYYTIyVfBOiFVzAG271cOxNHIL5oW3E3YsUOxFakVi9 4xLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681766711; x=1684358711; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=Z2hVXUd9jTQCUq7oB7gNxYi6a7Eio5tQcu8W27K+sdk=; b=FSnHQoPtZWquN7ZLX6NCK+zNaSyKPC1ovrS8K0EcV137gZPlF53Z3Qb8EMipDm4D6m 9OBN194UiPl7rT2TgxGxtEkwAqxYwaHwqifj19tgFmInjBeUug35lzsuYLEbi4PUwEQ8 foy2L+WGIbIT/Bs80kdYCjt7sBx42RhSQt0AnIV5pMW8r09E5pn//Pm1BC2V5v9IeWhj IN1xwOhoexNducD/NKZZdNB4ukxbu+sRRtNXmD5DizrShgkFv9r2FYwac8JEhmtB6/ms k+8IbPu1pH570Hu42Ge/R9Gz9eKQIOWwnX4gKZDDPBeIs+q2fuvs1thcJcaS9bKEwzRb IQmg== X-Gm-Message-State: AAQBX9eiHS3lhq34bl+7kT2yVuNs0CPMEdFFi1xhna2oZBQsLK7i3jR3 Q2TjHT8M2hJalZRVhm0GYCo/hMRrKaKUyvUL/nLrv8d9UmU= X-Google-Smtp-Source: AKy350aRZ8iQvkByTYRsxBIjESNuOPtbxpuTzFLrUv33XvoeJcTNG7WSQQDNHZc549NnnF52HZtI7xk+XsCkWm/7Yco= X-Received: by 2002:a5e:c008:0:b0:753:568:358e with SMTP id u8-20020a5ec008000000b007530568358emr11242819iol.20.1681766710771; Mon, 17 Apr 2023 14:25:10 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <202304172106.33HL6RUX051407@critter.freebsd.dk> In-Reply-To: <202304172106.33HL6RUX051407@critter.freebsd.dk> From: Xin LI Date: Mon, 17 Apr 2023 14:24:59 -0700 Message-ID: Subject: Re: find(1): I18N gone wild ? To: Poul-Henning Kamp Cc: current@freebsd.org Content-Type: multipart/alternative; boundary="000000000000640e9605f98ed3f4" X-Rspamd-Queue-Id: 4Q0g7X4bXlz43SL X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --000000000000640e9605f98ed3f4 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable This is expected behavior (in en_US.UTF-8 the ordering is AaBb, not ABab). You might want to set LC_COLLATE to C if C behavior is desirable. On Mon, Apr 17, 2023 at 2:06=E2=80=AFPM Poul-Henning Kamp wrote: > This surprised me: > > # mkdir /tmp/P > # cd /tmp/P > # touch FOO > # touch bar > # env LANG=3DC.UTF-8 find . -name '[A-Z]*' -print > ./FOO > # env LANG=3Den_US.UTF-8 find . -name '[A-Z]*' -print > ./FOO > ./bar > > Really ?! > > -- > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. > > --000000000000640e9605f98ed3f4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
This is expected behavior (in en_US.UTF-8 the ordering is AaBb= , not ABab).=C2=A0 You might want to set=C2=A0LC_COLLATE to C if C behavior= is desirable.

On Mon, Apr 17, 2023 at 2:06=E2=80=AFPM Poul-Henning Ka= mp <phk@phk.freebsd.dk> wro= te:
This surpris= ed me:

=C2=A0 =C2=A0 =C2=A0 =C2=A0 # mkdir /tmp/P
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # cd /tmp/P
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # touch FOO
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # touch bar
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # env LANG=3DC.UTF-8 find . -name '[A-Z]*&#= 39; -print
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ./FOO
=C2=A0 =C2=A0 =C2=A0 =C2=A0 # env LANG=3Den_US.UTF-8 find . -name '[A-Z= ]*' -print
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ./FOO
=C2=A0 =C2=A0 =C2=A0 =C2=A0 ./bar

Really ?!

--
Poul-Henning Kamp=C2=A0 =C2=A0 =C2=A0 =C2=A0| UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0| TCP/IP since RFC 956
FreeBSD committer=C2=A0 =C2=A0 =C2=A0 =C2=A0| BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.=

--000000000000640e9605f98ed3f4-- From nobody Mon Apr 17 21:33:04 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0gJh5w6Rz45DVp for ; Mon, 17 Apr 2023 21:33:08 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0gJh4BWTz4L5j for ; Mon, 17 Apr 2023 21:33:08 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; none Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id CAC275C0038; Mon, 17 Apr 2023 17:33:07 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Mon, 17 Apr 2023 17:33:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1681767187; x=1681853587; bh=jqP7xwgzsB3O9w2f17V2Voy/M4rCWzIKVt7 nv8vY/uo=; b=WbbINQRNayGV4EVBoKrUmq8D1foEjw11Lpje/7Nm3gKr7FK+XXG e0lhI6Ukwyu9BrtCRwFtPQqfqtiyOT9g1wZtHSY2JZ3PCNpRSaGYDGGoeXm/cUFT 4sMJvRgNGoeBS/IKt8ckgBj229NxoiL6/72rEHQAPTU16FGq0EAULlzuTUpu1zdK emHvR4KJzAHIofuFBDqkWN8oG6QxFVZ4YEdoN8Ekfq+AmaRoCv8/ZA2PQ9eOAsM+ JX+q4L3mz2t7D3HyXcfiA0VNFGQDeOQMOKO3RwS9HmFIyxdEKh5q2CRvQiAqDY8D Ta33dNGUnzstDzZiTvedYSGkPfjfCSpV/DA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t= 1681767187; x=1681853587; bh=jqP7xwgzsB3O9w2f17V2Voy/M4rCWzIKVt7 nv8vY/uo=; b=Xqb0T0RZtTzMAC05iB3q9u9y0i/MFsocBu5EobuVRimti54J8LH +VMO//J1h67526qKs4tfYz6jZuW1HQl9LP2vYz40QUpz1+cJkot8CNBNtC18Hj2b pEIj/S/K03vo5Nio62IjMXC3/hxmaAI5wxFwDUlHjdwO0fVU9wZG4scCIrq4mRIc hU/5lsuaSOW2UohLJx04gVLAECTN51jpV5iRREWnpkDBZPzOpZ/rQb03maR8UN96 YmK9ikbA1UE2djDyO282L4GdZm9WQxjdimCvd9Rr9yryrj1xbH0PHXwcK/G4xHjo EBVYBqDsyUeHvCRbYTZ6GZWpeGL1J7rOyqg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrvdeliedgudeiudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffvvehfhfgjtgfgse htkeertddtfeejnecuhfhrohhmpegjuhhrihcuoeihuhhrihesrggvthgvrhhnrdhorhhg qeenucggtffrrghtthgvrhhnpefghfffleejhfefhfettddtjeejjeefheeiuefhtdeihf duueegfffhteehueegffenucffohhmrghinhepohhpvghnghhrohhuphdrohhrghenucev lhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihuhhrihesrg gvthgvrhhnrdhorhhg X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 17 Apr 2023 17:33:06 -0400 (EDT) Message-ID: <6dd71202-4144-8587-b42c-8db44a4b737e@aetern.org> Date: Mon, 17 Apr 2023 23:33:04 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild ? Content-Language: en-US To: Xin LI , Poul-Henning Kamp Cc: current@freebsd.org References: <202304172106.33HL6RUX051407@critter.freebsd.dk> From: Yuri In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Q0gJh4BWTz4L5j X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:19151, ipnet:66.111.4.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Xin LI wrote: > This is expected behavior (in en_US.UTF-8 the ordering is AaBb, not > ABab).  You might want to set LC_COLLATE to C if C behavior is desirable. > > On Mon, Apr 17, 2023 at 2:06 PM Poul-Henning Kamp > wrote: > > This surprised me: > >         # mkdir /tmp/P >         # cd /tmp/P >         # touch FOO >         # touch bar >         # env LANG=C.UTF-8 find . -name '[A-Z]*' -print >         ./FOO >         # env LANG=en_US.UTF-8 find . -name '[A-Z]*' -print >         ./FOO >         ./bar > > Really ?! A bit more detail: find uses fnmatch(3) here, where the RE Bracket Expression rules apply (except for ! instead of ^, but that's unrelated): https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05 ...which has the following note: 7. In the POSIX locale, a range expression represents the set of collating elements that fall between two elements in the collation sequence, inclusive. In other locales, a range expression has unspecified behavior: strictly conforming applications shall not rely on whether the range expression is valid, or on the set of collating elements matched. Indeed, it's unfortunate that collations in non-POSIX are not that... linear and range expressions can break, but I don't see an easy way of "fixing" this. From nobody Mon Apr 17 22:38:47 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0hmX3bXlz45JGt for ; Mon, 17 Apr 2023 22:38:52 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Received: from mail.dawidek.net (mail.dawidek.net [94.130.64.56]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q0hmW6YSGz415q; Mon, 17 Apr 2023 22:38:51 +0000 (UTC) (envelope-from pjd@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from [192.168.250.133] (c-73-241-172-196.hsd1.ca.comcast.net [73.241.172.196]) by mail.dawidek.net (Postfix) with ESMTPSA id 24F6E4F8F8; Tue, 18 Apr 2023 00:38:49 +0200 (CEST) Message-ID: Date: Mon, 17 Apr 2023 15:38:47 -0700 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: another crash and going forward with zfs Content-Language: en-US To: Mateusz Guzik Cc: freebsd-current@freebsd.org, Glen Barber References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> From: Pawel Jakub Dawidek In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q0hmW6YSGz415q X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:94.130.0.0/16, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/18/23 05:14, Mateusz Guzik wrote: > On 4/17/23, Pawel Jakub Dawidek wrote: >> Correct me if I'm wrong, but from my understanding there were zero >> problems with block cloning when it wasn't in use or now disabled. >> >> The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to exactly >> avoid mess like this and give us more time to sort all the problems out >> while making it easy for people to try it. >> >> If there is no plan to revert the whole import, I don't see what value >> removing just block cloning will bring if it is now disabled by default >> and didn't cause any problems when disabled. >> > > The feature definitely was not properly stress tested and what not and > trying to do it keeps running into panics. Given the complexity of the > feature I would expect there are many bug lurking, some of which > possibly related to the on disk format. Not having to deal with any of > this is can be arranged as described above and is imo the most > sensible route given the timeline for 14.0 Block cloning doesn't create, remove or modify any on-disk data until it is in use. Again, if we are not going to revert the whole merge, I see no point in reverting block cloning as until it is enabled, its code is not executed. This allow people who upgraded the pools to do nothing special and it will allow people to test it easily. -- Pawel Jakub Dawidek From nobody Mon Apr 17 23:28:58 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0jtP58gMz45MMX for ; Mon, 17 Apr 2023 23:29:01 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0jtP422Gz4Dgf; Mon, 17 Apr 2023 23:29:01 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4003a.ext.cloudfilter.net ([10.228.9.183]) by cmsmtp with ESMTP id oMS6p9vX5uZMSoYHNpdR1N; Mon, 17 Apr 2023 23:29:01 +0000 Received: from spqr.komquats.com ([70.66.148.124]) by cmsmtp with ESMTPA id oYHLpZO3icyvuoYHMpFItf; Mon, 17 Apr 2023 23:29:01 +0000 X-Authority-Analysis: v=2.4 cv=VbHkgXl9 c=1 sm=1 tr=0 ts=643dd63d a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=kj9zAlcOel0A:10 a=dKHAf1wccvYA:10 a=13WrDtVnAAAA:8 a=6I5d2MoRAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=lNOpu8cdCLkuyS9JaL0A:9 a=CjuIK1q_8ugA:10 a=qcMfyop8IQhGkljw9-nY:22 a=IjZwj45LgO3ly-622nXo:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 634BB333; Mon, 17 Apr 2023 16:28:59 -0700 (PDT) Received: by slippy.cwsent.com (Postfix, from userid 1000) id 18262E2; Mon, 17 Apr 2023 16:28:59 -0700 (PDT) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.8+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Pawel Jakub Dawidek cc: Mateusz Guzik , freebsd-current@freebsd.org, Glen Barber Subject: Re: another crash and going forward with zfs In-reply-to: References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> Comments: In-reply-to Pawel Jakub Dawidek message dated "Mon, 17 Apr 2023 15:38:47 -0700." List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Date: Mon, 17 Apr 2023 16:28:58 -0700 Message-Id: <20230417232859.18262E2@slippy.cwsent.com> X-CMAE-Envelope: MS4xfB1rLH0BY+4kopOQIqPS/8bbAyh6LyL3cD7NZ0bslSyhHk8Xzvm2rRVqwQWXBwXcQ5gQBiGsUH7jdbNmf7YWl0bAON44tKEB15jsCs2U5zYzF09O9FsH abX2cC1xhpUuBewcKoIebJI+VwE/mDsFMkAc03l0Ko8ONkwvy7Jxxi9YGtdeVbf4fvcBRsEurcqit+nqEdIqDucbCgiW9IzjibyCHPvYYnAf7920KzNy2hIe g8Et7EqPvIsnkJvTc4RxKg8ZE+hL+HO6COB3jw/L5kv2MR0UmeasEyC3UaEuTU5B X-Rspamd-Queue-Id: 4Q0jtP422Gz4Dgf X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N In message , Pawel Jakub Dawi dek writes: > On 4/18/23 05:14, Mateusz Guzik wrote: > > On 4/17/23, Pawel Jakub Dawidek wrote: > >> Correct me if I'm wrong, but from my understanding there were zero > >> problems with block cloning when it wasn't in use or now disabled. > >> > >> The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to exactly > >> avoid mess like this and give us more time to sort all the problems out > >> while making it easy for people to try it. > >> > >> If there is no plan to revert the whole import, I don't see what value > >> removing just block cloning will bring if it is now disabled by default > >> and didn't cause any problems when disabled. > >> > > > > The feature definitely was not properly stress tested and what not and > > trying to do it keeps running into panics. Given the complexity of the > > feature I would expect there are many bug lurking, some of which > > possibly related to the on disk format. Not having to deal with any of > > this is can be arranged as described above and is imo the most > > sensible route given the timeline for 14.0 > > Block cloning doesn't create, remove or modify any on-disk data until it > is in use. > > Again, if we are not going to revert the whole merge, I see no point in > reverting block cloning as until it is enabled, its code is not > executed. This allow people who upgraded the pools to do nothing special > and it will allow people to test it easily. In this case zpool upgrade and zpool status should return no feature upgrades are available instead of enticing users to zpool upgrade. The userland zpool command should test for this sysctl and print nothing regarding block_cloning. I can see a scenario when a user zpool upgrades their pools, notices the sysctl and does the unthinkable. Not only would this fill the mailing lists with angry chatter but it would spawn a number of PRs plus give us a lot of bad press for data loss. Should we keep the new ZFS in 14, we should: 1. Make sure that zpool(8) does not mention or offer block_cloning in any way if the sysctl is disabled. 2. Print a cautionary note in release notes advising people not to enable this experimental sysctl. Maybe even have it print "(experimental)" to warn users that it will hurt. 3. Update the man pages to caution that block_cloning is experimental and unstable. It's not enough to have a sysctl without hiding block_cloning completely from view. Only expose it in zpool(8) when the sysctl is enabled. Let's avoid people mistakenly enabling it. -- Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0 From nobody Mon Apr 17 23:36:55 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0k3m2q3fz45Mrf for ; Mon, 17 Apr 2023 23:37:08 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0k3m0Ht7z3FBH; Mon, 17 Apr 2023 23:37:08 +0000 (UTC) (envelope-from rick.macklem@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-pj1-x102d.google.com with SMTP id 98e67ed59e1d1-24736992dd3so444991a91.1; Mon, 17 Apr 2023 16:37:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681774626; x=1684366626; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=hX293cZyviiYnWe1PM/ddZahmUJCBNMyZQ1bg6qScaE=; b=Tp9+4x8Qum1y/IQqY2L+5qLAmDYQQCKzghAQndiCsdqVRVqRVtE4sGCR8XYN0M5d6g 8Ewe+qZEhyO2Xn/8LPKmXNEV6/RjwprxvcXLRj1pL8UMupIxZDtuXVSxKrsbWYt/pd+W PPBYtYrgRpeKcMEutVF+dbKSkngbaLpMyZC6spkHN966WqWDXNAtyyNRAKm2/RjvaWr+ 3Dl2OUqeb1TuZjCE/DxSMaFfsR4Aq55KivCt7d5TDyCDkODV29q7PoT2mzj1JWpeXvrG eZzaczCQjdG1uJ8ExvT+zkTsEB5PqmeagXWRUR2H73utAUiomRGjOlRAMnlOubg1pOFe BQEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681774626; x=1684366626; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hX293cZyviiYnWe1PM/ddZahmUJCBNMyZQ1bg6qScaE=; b=FQy9gMXcwOQugJmgUtBL9NU8Sn54GZKsnniQF8Z+gOXGtNhLJEo79LT4RGUWKtl68B eX6FzpfaAGOD4pW2DAWe0YoBij3BfcJGEfYGgMtZMIkajtjjIVQoAVWygL7uvMfWVaXC ybX8VBAt0iUSsvzfn9D11eRhR+x20l+/GG6IupX+ZrYyQ6fItpQmClJ9LenBL/oI7ZtH X4NrNw9XTDalxKgORXsgUSPyWNSD6fKFivwglKNiAiCrKrShaj4thUeO2DGsYhBX0zQu QL6v1WNIr3cuCkqkO91Y7hfdoyYTrpXif0G0rldBqHfomtWeKqnyT2pToVnuZtnT4FZq HE2g== X-Gm-Message-State: AAQBX9c2EBsOzCbbf5Zi3n+Fn9LpmEUOfGTeb9oLf3KoFzTCRsCnxMQ1 pk475CvgSgua3VQF2sSUfXiq7jr0wqmQwk8iUA== X-Google-Smtp-Source: AKy350aRoX2vxcL7cACS1bKARFX/SxNXxkCLbI31/1UsGhmMRZMuIyKQn524fLr4+91N2tOxxK1lSF/ZvqiMY37HOSk= X-Received: by 2002:a17:90b:ecb:b0:23f:3f9c:7878 with SMTP id gz11-20020a17090b0ecb00b0023f3f9c7878mr227712pjb.2.1681774626158; Mon, 17 Apr 2023 16:37:06 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> <20230417232859.18262E2@slippy.cwsent.com> In-Reply-To: <20230417232859.18262E2@slippy.cwsent.com> From: Rick Macklem Date: Mon, 17 Apr 2023 16:36:55 -0700 Message-ID: Subject: Re: another crash and going forward with zfs To: Cy Schubert Cc: Pawel Jakub Dawidek , Mateusz Guzik , freebsd-current@freebsd.org, Glen Barber Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Rspamd-Queue-Id: 4Q0k3m0Ht7z3FBH X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; TAGGED_FROM(0.00)[] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Mon, Apr 17, 2023 at 4:29=E2=80=AFPM Cy Schubert wrote: > > In message , Pawel Jaku= b > Dawi > dek writes: > > On 4/18/23 05:14, Mateusz Guzik wrote: > > > On 4/17/23, Pawel Jakub Dawidek wrote: > > >> Correct me if I'm wrong, but from my understanding there were zero > > >> problems with block cloning when it wasn't in use or now disabled. > > >> > > >> The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to exa= ctly > > >> avoid mess like this and give us more time to sort all the problems = out > > >> while making it easy for people to try it. > > >> > > >> If there is no plan to revert the whole import, I don't see what val= ue > > >> removing just block cloning will bring if it is now disabled by defa= ult > > >> and didn't cause any problems when disabled. > > >> > > > > > > The feature definitely was not properly stress tested and what not an= d > > > trying to do it keeps running into panics. Given the complexity of th= e > > > feature I would expect there are many bug lurking, some of which > > > possibly related to the on disk format. Not having to deal with any o= f > > > this is can be arranged as described above and is imo the most > > > sensible route given the timeline for 14.0 > > > > Block cloning doesn't create, remove or modify any on-disk data until i= t > > is in use. > > > > Again, if we are not going to revert the whole merge, I see no point in > > reverting block cloning as until it is enabled, its code is not > > executed. This allow people who upgraded the pools to do nothing specia= l > > and it will allow people to test it easily. > > In this case zpool upgrade and zpool status should return no feature > upgrades are available instead of enticing users to zpool upgrade. The > userland zpool command should test for this sysctl and print nothing > regarding block_cloning. I can see a scenario when a user zpool upgrades > their pools, notices the sysctl and does the unthinkable. Not only would > this fill the mailing lists with angry chatter but it would spawn a numbe= r > of PRs plus give us a lot of bad press for data loss. > > Should we keep the new ZFS in 14, we should: > > 1. Make sure that zpool(8) does not mention or offer block_cloning in any > way if the sysctl is disabled. > > 2. Print a cautionary note in release notes advising people not to enable > this experimental sysctl. Maybe even have it print "(experimental)" to wa= rn > users that it will hurt. > > 3. Update the man pages to caution that block_cloning is experimental and > unstable. I would suggest going a step further and making the sysctl RO for FreeBSD14= . (This could be changed for FreeBSD14.n if/when block_cloning is believed to be debugged.) I would apply all 3 of the above to "main", since some that install "main" will not know how "bleeding edge" this is unless the above is done. (Yes, I know "main" is "bleeding edge", but some still expect a stable test system will result from installing it.) Thanks go to all that tracked this problem down, rick > > It's not enough to have a sysctl without hiding block_cloning completely > from view. Only expose it in zpool(8) when the sysctl is enabled. Let's > avoid people mistakenly enabling it. > > > -- > Cheers, > Cy Schubert > FreeBSD UNIX: Web: https://FreeBSD.org > NTP: Web: https://nwtime.org > > e^(i*pi)+1=3D0 > > > From nobody Tue Apr 18 01:16:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0mG956fTz45TGf for ; Tue, 18 Apr 2023 01:16:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0mG80vLfz3wpZ for ; Tue, 18 Apr 2023 01:16:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20221208.gappssmtp.com header.s=20221208 header.b=Ty8fOYHb; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::635) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ej1-x635.google.com with SMTP id kt6so30879491ejb.0 for ; Mon, 17 Apr 2023 18:16:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1681780573; x=1684372573; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=yUY7Abn9c9PaqNNRkIxr+v6jaT7Z+RgW2tnfRAp36IA=; b=Ty8fOYHbBYSKSvzEsxrA4Uh/NmrVkJuWwBoxHh70wt77i0gOrCBWntnNs0iOsL5iK2 H0EWY7/j9VuczqkljY9/i7VF5Et/axOfhm0bmhk8eS4oMtjNI+95H/rUgmxBRYSNRj5B KaFGOsUKbjdvxJoM1ADcRqTpP+mZOT1aFuMXClSKBHvq6foq05GLuSbDxI/nTIUrR83W +6i8OBIzXnvVSjQWbtV6bRMB2pbPWCZ8u9wF5qrTVF+P8bc3bTIw1HDkrPP4tY2kz68G V545D2rRTWQZc01wjmCEghdjcd5Js+nLpKOzFxPBS0VWip5NOwzhzwVA2cLWFvrcDdDW Y3zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681780573; x=1684372573; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yUY7Abn9c9PaqNNRkIxr+v6jaT7Z+RgW2tnfRAp36IA=; b=ecyQr6EcfkblVOXHI/1Xv3GZwvDs/7MrVFuOj9d9S+QGZkCjjMxvJzdnI6Icam13x5 c+tW8BiM6N8LuSQ0rWxL2Cj5ErGhioFz1Uj09xpTcH+A03dKrenGT1fCPaiM/ekdt6HE birnj5Bp9LHnT1FlNJSxGUSs7uvNwIJfONDPaiDuGWHmCskHrQ8TX7HP38XPzjEO56Fp uyOVWl8ZCI1c/b6geGxw/4bX6EuZhal+h2fGfMPpH3tdOv0GZ7ICq8IdtATcDMVvqrzR ZQnAqRjG0HpM+vVeySXzs1E3uwHclpwi+Y4W27pRgc7OPa4Sr0lE+R34dwAWaYBUPVXj TdrQ== X-Gm-Message-State: AAQBX9djRqj8OQiSynchxDan8pDrIWTpClw8sQRLHouslOJCxumRdrRc uiVSM+oGp4eDFmZe976EMMq55kIQyAZj8waTVwLWgA== X-Google-Smtp-Source: AKy350Yi0XGLzSsdbcY+qkeNHQsz7l4CM+D4UyR54lFlZi22QJkdQo0Rl8U6zRT0Ovt5brvvIJPc3cKrc4pwSu2K694= X-Received: by 2002:a17:906:4757:b0:94e:7ce:4d1f with SMTP id j23-20020a170906475700b0094e07ce4d1fmr4587243ejs.2.1681780573309; Mon, 17 Apr 2023 18:16:13 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> <20230417232859.18262E2@slippy.cwsent.com> In-Reply-To: From: Warner Losh Date: Mon, 17 Apr 2023 19:16:01 -0600 Message-ID: Subject: Re: another crash and going forward with zfs To: Rick Macklem Cc: Cy Schubert , Pawel Jakub Dawidek , Mateusz Guzik , FreeBSD Current , Glen Barber Content-Type: multipart/alternative; boundary="000000000000a9a06505f9920d11" X-Spamd-Result: default: False [-1.50 / 15.00]; SUSPICIOUS_RECIPS(1.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20221208.gappssmtp.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[bsdimp.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::635:from]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; TAGGED_RCPT(0.00)[]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; DKIM_TRACE(0.00)[bsdimp-com.20221208.gappssmtp.com:+]; ARC_NA(0.00)[]; FREEMAIL_CC(0.00)[cschubert.com,freebsd.org,gmail.com]; RCPT_COUNT_FIVE(0.00)[6] X-Rspamd-Queue-Id: 4Q0mG80vLfz3wpZ X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N --000000000000a9a06505f9920d11 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Apr 17, 2023, 5:37 PM Rick Macklem wrote: > On Mon, Apr 17, 2023 at 4:29=E2=80=AFPM Cy Schubert > wrote: > > > > In message , Pawel > Jakub > > Dawi > > dek writes: > > > On 4/18/23 05:14, Mateusz Guzik wrote: > > > > On 4/17/23, Pawel Jakub Dawidek wrote: > > > >> Correct me if I'm wrong, but from my understanding there were zero > > > >> problems with block cloning when it wasn't in use or now disabled. > > > >> > > > >> The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to > exactly > > > >> avoid mess like this and give us more time to sort all the problem= s > out > > > >> while making it easy for people to try it. > > > >> > > > >> If there is no plan to revert the whole import, I don't see what > value > > > >> removing just block cloning will bring if it is now disabled by > default > > > >> and didn't cause any problems when disabled. > > > >> > > > > > > > > The feature definitely was not properly stress tested and what not > and > > > > trying to do it keeps running into panics. Given the complexity of > the > > > > feature I would expect there are many bug lurking, some of which > > > > possibly related to the on disk format. Not having to deal with any > of > > > > this is can be arranged as described above and is imo the most > > > > sensible route given the timeline for 14.0 > > > > > > Block cloning doesn't create, remove or modify any on-disk data until > it > > > is in use. > > > > > > Again, if we are not going to revert the whole merge, I see no point = in > > > reverting block cloning as until it is enabled, its code is not > > > executed. This allow people who upgraded the pools to do nothing > special > > > and it will allow people to test it easily. > > > > In this case zpool upgrade and zpool status should return no feature > > upgrades are available instead of enticing users to zpool upgrade. The > > userland zpool command should test for this sysctl and print nothing > > regarding block_cloning. I can see a scenario when a user zpool upgrade= s > > their pools, notices the sysctl and does the unthinkable. Not only woul= d > > this fill the mailing lists with angry chatter but it would spawn a > number > > of PRs plus give us a lot of bad press for data loss. > > > > Should we keep the new ZFS in 14, we should: > > > > 1. Make sure that zpool(8) does not mention or offer block_cloning in a= ny > > way if the sysctl is disabled. > > > > 2. Print a cautionary note in release notes advising people not to enab= le > > this experimental sysctl. Maybe even have it print "(experimental)" to > warn > > users that it will hurt. > > > > 3. Update the man pages to caution that block_cloning is experimental a= nd > > unstable. > I would suggest going a step further and making the sysctl RO for > FreeBSD14. > (This could be changed for FreeBSD14.n if/when block_cloning is believed = to > be debugged.) > > I would apply all 3 of the above to "main", since some that install "main= " > will not know how "bleeding edge" this is unless the above is done. > (Yes, I know "main" is "bleeding edge", but some still expect a stable > test system will result from installing it.) > > Thanks go to all that tracked this problem down, rick > Related question: what zfs branch is stable/14 going to track? With 13 it was whatever the next stable branch was. Warner > > > It's not enough to have a sysctl without hiding block_cloning completel= y > > from view. Only expose it in zpool(8) when the sysctl is enabled. Let's > > avoid people mistakenly enabling it. > > > > > > -- > > Cheers, > > Cy Schubert > > FreeBSD UNIX: Web: https://FreeBSD.org > > NTP: Web: https://nwtime.org > > > > e^(i*pi)+1=3D0 > > > > > > > > --000000000000a9a06505f9920d11 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Mon, Apr 17, 2023, 5:37 PM Rick Macklem <rick.macklem@gmail.com> wrote:
On Mon, Apr 17, 2023 at 4:29=E2=80=AF= PM Cy Schubert <Cy.Schubert@cschubert.com> wrote:
>
> In message <b57b06bd-7e73-ae2d-2fba-b= d226883ff34@dawidek.net>, Pawel Jakub
> Dawi
> dek writes:
> > On 4/18/23 05:14, Mateusz Guzik wrote:
> > > On 4/17/23, Pawel Jakub Dawidek <pjd@freebsd.org> wro= te:
> > >> Correct me if I'm wrong, but from my understanding t= here were zero
> > >> problems with block cloning when it wasn't in use or= now disabled.
> > >>
> > >> The reason I've introduced vfs.zfs.bclone_enabled sy= sctl, was to exactly
> > >> avoid mess like this and give us more time to sort all t= he problems out
> > >> while making it easy for people to try it.
> > >>
> > >> If there is no plan to revert the whole import, I don= 9;t see what value
> > >> removing just block cloning will bring if it is now disa= bled by default
> > >> and didn't cause any problems when disabled.
> > >>
> > >
> > > The feature definitely was not properly stress tested and wh= at not and
> > > trying to do it keeps running into panics. Given the complex= ity of the
> > > feature I would expect there are many bug lurking, some of w= hich
> > > possibly related to the on disk format. Not having to deal w= ith any of
> > > this is can be arranged as described above and is imo the mo= st
> > > sensible route given the timeline for 14.0
> >
> > Block cloning doesn't create, remove or modify any on-disk da= ta until it
> > is in use.
> >
> > Again, if we are not going to revert the whole merge, I see no po= int in
> > reverting block cloning as until it is enabled, its code is not > > executed. This allow people who upgraded the pools to do nothing = special
> > and it will allow people to test it easily.
>
> In this case zpool upgrade and zpool status should return no feature > upgrades are available instead of enticing users to zpool upgrade. The=
> userland zpool command should test for this sysctl and print nothing > regarding block_cloning. I can see a scenario when a user zpool upgrad= es
> their pools, notices the sysctl and does the unthinkable. Not only wou= ld
> this fill the mailing lists with angry chatter but it would spawn a nu= mber
> of PRs plus give us a lot of bad press for data loss.
>
> Should we keep the new ZFS in 14, we should:
>
> 1. Make sure that zpool(8) does not mention or offer block_cloning in = any
> way if the sysctl is disabled.
>
> 2. Print a cautionary note in release notes advising people not to ena= ble
> this experimental sysctl. Maybe even have it print "(experimental= )" to warn
> users that it will hurt.
>
> 3. Update the man pages to caution that block_cloning is experimental = and
> unstable.
I would suggest going a step further and making the sysctl RO for FreeBSD14= .
(This could be changed for FreeBSD14.n if/when block_cloning is believed to=
=C2=A0be debugged.)

I would apply all 3 of the above to "main", since some that insta= ll "main"
will not know how "bleeding edge" this is unless the above is don= e.
(Yes, I know "main" is "bleeding edge", but some still = expect a stable
=C2=A0test system will result from installing it.)

Thanks go to all that tracked this problem down, rick

Related question: what= zfs branch is stable/14 going to track? With 13 it was whatever the next s= table branch was.

Warner=


>
> It's not enough to have a sysctl without hiding block_cloning comp= letely
> from view. Only expose it in zpool(8) when the sysctl is enabled. Let&= #39;s
> avoid people mistakenly enabling it.
>
>
> --
> Cheers,
> Cy Schubert <Cy.Schubert@cschubert.com>
> FreeBSD UNIX:=C2=A0 <cy@FreeBSD.org>=C2=A0 =C2=A0Web:=C2=A0 https://FreeBSD.org
> NTP:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<cy@nwtime.org>=C2= =A0 =C2=A0 Web:=C2=A0 https://nwtime.org
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0e^(i*pi)+1=3D0
>
>
>

--000000000000a9a06505f9920d11-- From nobody Tue Apr 18 04:12:40 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0rB31f1qz45g12 for ; Tue, 18 Apr 2023 04:12:59 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic309-22.consmr.mail.gq1.yahoo.com (sonic309-22.consmr.mail.gq1.yahoo.com [98.137.65.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0rB15GrFz4MyV for ; Tue, 18 Apr 2023 04:12:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="cXGsiiW/"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681791175; bh=PZYHTqa7VsXdAdT8BxowVFVvsVCQPGlpBqQWzfznyeQ=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=cXGsiiW/EC/p/vnUj7QJoT+VeB0NKcIFPrd2fwuLvGtmSj/un6KNglU5n8EHpcCgsQDBD6pQ91tI1d2MdmxwV8UNCMMpOEJzDmg1rJKHTpE8Aw8Jzslo1kRj6fGeNI3dDLFVXFXBSfKIzp7d2c9nX6yvkbrMixDlTrSde6hNPLWfvHyHjEt8UeJMLOYtpPC4a+keMFHCBhuq78aQ+CzXyprbriQckaIwOBDInvuZVX5Ev8PlFhd0QmoemydHVFTm3x7EcEkhuEn+PUptOXzbpn9FlwoGeIydmK1lVI2H9O9oLGPZNrnJwP/YmrJmGHxr/I/4RSByOesHt7DsIv/yhw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681791175; bh=eTewRCMda1P6yeTN4+xMFI7PVnhSaTx/g5g0NGE+Jny=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=kWJ4B7zUx2TPppZ0F5EBHYlVJ0UL8RPGTN8vCXXT5Na+beAhZ6xsih00p/wPff3t4MfXRA+mxbkKr4QPVy8lvBkRsef7XpjxVwM3GJ/Y8WYgh1oPovaV6rgRT862GzHQ+/OP3A/VZyTtpGPNvLXI19rhinLD1Bbii3NELB+2zdKxP2oq5KOFAfobRnl7wB6x9l1vzISbAws7eFM8x2PCRZ8Mw4GDF48CqqSISAt32v5KobHQhzqkebzhG0vz8cj+JVdyzfLMAhrOmSsqWOMUf+ALJEykbSc7ckq3slIJvqfCGN+nG6o0mLgONBmXkBusHnfcV1acoF8ZqaYE+N2r+g== X-YMail-OSG: CciBIFcVM1lECnHJ8i7vc8DW6wsth6DpKbIjO7qqlPU_yfk_rEj7o06JbnW0NZS lmNgVtzjnzjj4kXkHrPUr6jkl0X7TSP.oQIjJ.fWAKpt9AISlVGXyfZjX_hVYM4.PrGdPrOrzcPK DhxOjGR2qnCwnvd2XUMkyqc6EbL6uhwoLOqvag_OkYLC0hwFlmA_1sSCs.NRWT_a8YxSUoJZrKTg RHvOGtpQ_qkoAzJAu119DT6Z3aUMxWwbBqkIqHKFvYLvtQISwKh.hPOVsv2cv3eBTGAIR9R9jlZL 9AzhKsaxfXJIkwkj6uKqaZjxOjp9xeOKJyXVk_V9AiG3jktzCtLt6Z_T9qsDsB23EqhuT42C7ixW Lz3qRE5DRf2iG34qnb.HQ4V_3WnjchIEIkxr6h.jTuVFqoMiFQFsWZFcwttg5sy9dITJUxcyLrPu XyegezopxhFFHI_oxqGL42At525sdF1fG4qrzvjUrbzFs2MZBKb4DpV_.havRKWsm_MNmny0afFN Ap7wJEP9977hSGBPLuR6yxE35QKP11dvv1tKSBd8K5YMZqheqeaaBVDOQWmh2Hg5FvHHSaCUmwIU MWYCTJ5o7TNBEjFJNKhI.mFTMKdlP3oD8Dh6QWHi3C38tLN_9pusOvfDFg8y7yRbzBIvFKcErp1m 2uU93Jddqsqm0LCNZ6vGT9w0WSr._c4Jmb932xt6U8FaKeUKAEmj8v61kwq2yhKjKoaNvVmZfjDA nLVOJevvcx9qepZBm_wpk.SgZMcbRReU.cLGN7vsm6jtC7QI.oDt9PRJxOUUI7g44stUJIAt0xjY FzQEwXX1nygepYlecEf7CyKOgEoKgnl802xUojKP.wy_LAuuXBXRS.CWKeHd0mR95Xe4qqjTIw7M BxmaRK9IxIiH0khH79_t3zCEw3rUhOTC.AhjQKKiUqK3dAkJwU.6jpLCrc9qMa2n.HnL0_4FJQ_A mDqSHHmkZEtAmztPVfQ1XrKOCejTqoJzb6MFg3P4qixcGJK5Ilusu6.RNtyuCl.cRSCUza2BqTmG HhEwyqaWTqvRbNQlRiQDEXRB1qN7lohCqzXBYoNi5Xh8qdcf4ps4.z9i37zVecV3UNRgWFse5InM tvXe2hL6RgYPXaK7uNkQ5DnCnWnz7TVC4Rdkbj8ZR25tEEXdG5UVRn1hXYDHNTdXFefxNDwElWpW F8fmNSsp5d87zQQzkeTn4Ev_zKsvxqjYrJflLO5xVAW2kOBpSN14yL9xcvTNMifsa5qxk7QulNIr eugCS5PdyU.Dot7VLDbDcm23MLJe9a4j9TJ9gMH9p.ROwprwhv9B9atwfjaiKyH1rN8cMmVAmnCq 0gFAml.Afcjhnq2dr2z2F_Hxu.T8TWnQBXKhuvDjahmVBGVw5cZiWyxJtJkhikmNHIXSWy4lSk6g IrMriDaDqoIe5_rkDJVZEoAkWKuqTiDl34.1011V9CELsUQ1JMzpY14xvkUoVC4f4VBn1IFbH3tr en57ns6TqdjOkengTULTqFzf71EoDsl9PUChVOSA4sWWHZbeaiCINn3sOrfq__fCTxKC5bdsqmJ6 bS7h5EGCvJKuS8i3Bh2QWYsKzEXSg_R6WAQex12PvOCZJoz4.gkECei8.WR41x54CEwpXg6P_NpY MNCbiiI4sVmP3diafiUW_yOAzt7RSSrDaWo96CZ_lHeJmNOMWyftiTpUfZezt4Fq_TWoh.MirDN9 rF.qlDtSHnysinwVEEzZvlKjdfJvcku9jEUJledA8Rm29i0II3LsWeCrDBmVIew3FsyyC31ChRFu sz4UrPJuOQ7TWGFBFfrmvY85fbV233UtO6IdE_xgZSXbOK_DKUwuL4r_1wxZqr_qXHOhJ7hKEp4N INYtTLOJdeNt1i3nEbgY0o.cFtRElT3BKMVWboBy2Eqokl2CCLwyLmHhwsxm4e7nZCSeFyZl4pZO Xlg039ZZnXGgRod6QsaJsrdGZQkLISe4oMJ6B4QRfVqGAXLrKsECGKYYpKYa31shSRVLFBNIPPTD pn.U9Rt.rWGaZiIb5aRkc5QdUvpL85rCFvL2OC8bPoquWaMYqTuVgoqb88kxAZlTVZHskYv_wwp1 ZVLTtLUlFBxImpsFVIPtRzTCB4.iDda_xNfWVL3SAiqz1WC3oDXbiJ4ahwhw_Z0Pthj6tdZN1koK oduSjPP0OQpNatMBUZLuDiwdhgbUS9SROjsx_YkUC_oU_tDuEnfnhZm6KFYQmMudhDmV1R_jsXMZ k9NnqSU2Ccl7wMA3_bVUu4Tr8VfMlqbx9sGCZLNRxOksenDbXtrNEn6zedtKks_9e5HgKbUPiCp0 1Lx9yvg-- X-Sonic-MF: X-Sonic-ID: 8c312869-422b-4a08-8976-d58259c87964 Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.gq1.yahoo.com with HTTP; Tue, 18 Apr 2023 04:12:55 +0000 Received: by hermes--production-gq1-546798879c-8jjxz (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 560d7c7a28d23d822fc272f0dac9dc1a; Tue, 18 Apr 2023 04:12:51 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: FreeBSD ??-?-RELEASE matching vs. zpool create -o compatibility= use Message-Id: <7997A6FF-10BA-49D2-B09E-43A4590DA700@yahoo.com> Date: Mon, 17 Apr 2023 21:12:40 -0700 To: Warner Losh , Current FreeBSD X-Mailer: Apple Mail (2.3731.400.51.1.1) References: <7997A6FF-10BA-49D2-B09E-43A4590DA700.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.148:from]; SUBJECT_HAS_QUESTION(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.65.148:from] X-Rspamd-Queue-Id: 4Q0rB15GrFz4MyV X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Warner Losh wrote on Date: Tue, 18 Apr 2023 01:16:01 UTC : [For a different subject.] > . . . > > Related question: what zfs branch is stable/14 going to track? With 13 it > was whatever the next stable branch was. I've a somewhat related question, using 13.2-RELEASE as an example of my general question. FreeBSD 13.2-RELEASE releng/13.2-n254617-525ecfdad597 GENERIC generates: # zpool version ZFS filesystem version: 5 ZFS storage pool version: features support (5000) zfs-2.1.9-FreeBSD_g92e0d9d18 zfs-kmod-2.1.9-FreeBSD_g92e0d9d18 but there is only: # ls -C1 /usr/share/zfs/compatibility.d/openzfs*freebsd /usr/share/zfs/compatibility.d/openzfs-2.0-freebsd /usr/share/zfs/compatibility.d/openzfs-2.1-freebsd No openzfs-2.1.9-freebsd file is available for use with the likes of the notation: -o compatibility=openzfs-2.1.9-freebsd Such presumably would also enable (based on an what is reported for an existing openzfs-2.1-freebsd style pool): # zpool get all | grep feature@ | grep disabled zoptb feature@edonr disabled local zoptb feature@zilsaxattr disabled local zoptb feature@head_errlog disabled local zoptb feature@blake3 disabled local but there would be a named compatibility assignment available for that. It is normal for me to hold compatibility at what some FreeBSD ??.?-RELEASE (and later) supports, even for main based systems (my normal context). (I do not know how common that personal policy is in the world.) I stick to the most recent that is official in the ??.?-RELEASE's /usr/share/zfs/compatibility.d/ Would it be appropriate for 13.2-RELEASE (e.g.) to have such a: /usr/share/zfs/compatibility.d/openzfs-2.1.9-freebsd that would match the FreeBSD release but that eventually would not list everything stable/13 or main would eventually support? ( stable/13 and main would then also end up with the file being present as well. But I'm working backwards from the end result to how to get there. ) Note: One could imagine a openzfs-2.1.10-freebsd in releng/14.0 that did not list block_cloning, even if there was (only) an odd way to get block_cloning enabled for testing purposes. === Mark Millard marklmi at yahoo.com From nobody Tue Apr 18 07:00:23 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0wVH6MfHz45tqw for ; Tue, 18 Apr 2023 07:27:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0wVH10gMz3sHC; Tue, 18 Apr 2023 07:27:19 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy01.your-server.de ([78.46.139.224]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pofKC-0006f5-S3; Tue, 18 Apr 2023 09:00:24 +0200 Received: from [188.167.171.2] (helo=[10.0.9.128]) by sslproxy01.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pofKC-0001aK-Gz; Tue, 18 Apr 2023 09:00:24 +0200 Message-ID: <88d3fa2a-a5fc-5298-f102-10c4f0287e20@FreeBSD.org> Date: Tue, 18 Apr 2023 09:00:23 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: another crash and going forward with zfs Content-Language: en-US To: Warner Losh , Rick Macklem Cc: Cy Schubert , Pawel Jakub Dawidek , Mateusz Guzik , FreeBSD Current , Glen Barber References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> <20230417232859.18262E2@slippy.cwsent.com> From: Martin Matuska In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.8/26878/Mon Apr 17 09:23:32 2023) X-Rspamd-Queue-Id: 4Q0wVH10gMz3sHC X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_RCPT(0.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 18. 4. 2023 3:16, Warner Losh wrote: > > Related question: what zfs branch is stable/14 going to track? With 13 > it was whatever the next stable branch was. > > Warner FreeBSD 14.0 is about to track soon-to-be-branched OpenZFS 2.2 From nobody Tue Apr 18 07:46:41 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0wwg4w4nz45w3d for ; Tue, 18 Apr 2023 07:46:43 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0wwg49ZYz4JdR; Tue, 18 Apr 2023 07:46:43 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy04.your-server.de ([78.46.152.42]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pog30-000C06-DE; Tue, 18 Apr 2023 09:46:42 +0200 Received: from [188.167.171.2] (helo=[10.0.9.128]) by sslproxy04.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1pog30-000Eki-5j; Tue, 18 Apr 2023 09:46:42 +0200 Message-ID: <8ea96ab9-89c5-d05a-95d1-ef71663f28bb@FreeBSD.org> Date: Tue, 18 Apr 2023 09:46:41 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: another crash and going forward with zfs Content-Language: en-US To: Mateusz Guzik , Pawel Jakub Dawidek Cc: freebsd-current@freebsd.org, Glen Barber References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> From: Martin Matuska In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.8/26878/Mon Apr 17 09:23:32 2023) X-Rspamd-Queue-Id: 4Q0wwg49ZYz4JdR X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Btw. I am open for setting up a pre-merge stress testing I will check out if I can use the hourly-billed amd64 and arm64 cloud boxes at Hetzner with FreeBSD. Otherwise there are monthly-billed as well. Cheers, mm On 17. 4. 2023 22:14, Mateusz Guzik wrote: > On 4/17/23, Pawel Jakub Dawidek wrote: >> On 4/18/23 03:51, Mateusz Guzik wrote: >>> After bugfixes got committed I decided to zpool upgrade and sysctl >>> vfs.zfs.bclone_enabled=1 vs poudriere for testing purposes. I very >>> quickly got a new crash: >>> >>> panic: VERIFY(arc_released(db->db_buf)) failed >>> >>> cpuid = 9 >>> time = 1681755046 >>> KDB: stack backtrace: >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame >>> 0xfffffe0a90b8e5f0 >>> vpanic() at vpanic+0x152/frame 0xfffffe0a90b8e640 >>> spl_panic() at spl_panic+0x3a/frame 0xfffffe0a90b8e6a0 >>> dbuf_redirty() at dbuf_redirty+0xbd/frame 0xfffffe0a90b8e6c0 >>> dmu_buf_will_dirty_impl() at dmu_buf_will_dirty_impl+0xa2/frame >>> 0xfffffe0a90b8e700 >>> dmu_write_uio_dnode() at dmu_write_uio_dnode+0xe9/frame >>> 0xfffffe0a90b8e780 >>> dmu_write_uio_dbuf() at dmu_write_uio_dbuf+0x42/frame 0xfffffe0a90b8e7b0 >>> zfs_write() at zfs_write+0x672/frame 0xfffffe0a90b8e960 >>> zfs_freebsd_write() at zfs_freebsd_write+0x39/frame 0xfffffe0a90b8e980 >>> VOP_WRITE_APV() at VOP_WRITE_APV+0xdb/frame 0xfffffe0a90b8ea90 >>> vn_write() at vn_write+0x325/frame 0xfffffe0a90b8eb20 >>> vn_io_fault_doio() at vn_io_fault_doio+0x43/frame 0xfffffe0a90b8eb80 >>> vn_io_fault1() at vn_io_fault1+0x161/frame 0xfffffe0a90b8ecc0 >>> vn_io_fault() at vn_io_fault+0x1b5/frame 0xfffffe0a90b8ed40 >>> dofilewrite() at dofilewrite+0x81/frame 0xfffffe0a90b8ed90 >>> sys_write() at sys_write+0xc0/frame 0xfffffe0a90b8ee00 >>> amd64_syscall() at amd64_syscall+0x157/frame 0xfffffe0a90b8ef30 >>> fast_syscall_common() at fast_syscall_common+0xf8/frame >>> 0xfffffe0a90b8ef30 >>> --- syscall (4, FreeBSD ELF64, write), rip = 0x103cddf7949a, rsp = >>> 0x103cdc85dd48, rbp = 0x103cdc85dd80 --- >>> KDB: enter: panic >>> [ thread pid 95000 tid 135035 ] >>> Stopped at kdb_enter+0x32: movq $0,0x9e4153(%rip) >>> >>> The posted 14.0 schedule which plans to branch stable/14 on May 12 and >>> one cannot bet on the feature getting beaten up into production shape >>> by that time. Given whatever non-block_clonning and not even zfs bugs >>> which are likely to come out I think this makes the feature a >>> non-starter for said release. >>> >>> I note: >>> 1. the current problems did not make it into stable branches. >>> 2. there was block_cloning-related data corruption (fixed) and there may >>> be more >>> 3. there was unrelated data corruption (see >>> https://github.com/openzfs/zfs/issues/14753), sorted out by reverting >>> the problematic commit in FreeBSD, not yet sorted out upstream >>> >>> As such people's data may be partially hosed as is. >>> >>> Consequently the proposed plan is as follows: >>> 1. whack the block cloning feature for the time being, but make sure >>> pools which upgraded to it can be mounted read-only >>> 2. run ztest and whatever other stress testing on FreeBSD, along with >>> restoring openzfs CI -- I can do the first part, I'm sure pho will not >>> mind to run some tests of his own >>> 3. recommend people create new pools and restore data from backup. if >>> restoring from backup is not an option, tar or cp (not zfs send) from >>> the read-only mount >>> >>> block cloning beaten into shape would use block_cloning_v2 or whatever >>> else, key point that the current feature name would be considered >>> bogus (not blocking RO import though) to prevent RW usage of the >>> current pools with it enabled. >>> >>> Comments? >> Correct me if I'm wrong, but from my understanding there were zero >> problems with block cloning when it wasn't in use or now disabled. >> >> The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to exactly >> avoid mess like this and give us more time to sort all the problems out >> while making it easy for people to try it. >> >> If there is no plan to revert the whole import, I don't see what value >> removing just block cloning will bring if it is now disabled by default >> and didn't cause any problems when disabled. >> > The feature definitely was not properly stress tested and what not and > trying to do it keeps running into panics. Given the complexity of the > feature I would expect there are many bug lurking, some of which > possibly related to the on disk format. Not having to deal with any of > this is can be arranged as described above and is imo the most > sensible route given the timeline for 14.0 > From nobody Tue Apr 18 08:12:57 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0xW369bDz44jVr for ; Tue, 18 Apr 2023 08:13:03 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic313-10.consmr.mail.ne1.yahoo.com (sonic313-10.consmr.mail.ne1.yahoo.com [66.163.185.33]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0xW26ZVPz3x3W for ; Tue, 18 Apr 2023 08:13:02 +0000 (UTC) (envelope-from filippomore@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=RqdWUQlm; spf=pass (mx1.freebsd.org: domain of filippomore@yahoo.com designates 66.163.185.33 as permitted sender) smtp.mailfrom=filippomore@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681805580; bh=WERZV/I0NN6YA9bXpBp2mjXXHV/pdzfx117VSJW/niE=; h=Date:From:To:Subject:References:From:Subject:Reply-To; b=RqdWUQlmzafif2+dvCY/OJK7x1rzWFpngRETYNzxtjn4PugQsKLzhdEnVvjKlFEUA9I8zywkmTV/cMoIFBb2b26OdrbdsmVPO09ZQoxSGeGtIfXE+XcogBBZ9vyW0JTamSmLtm0N3r5STcDwRTc+jkd6Tq9yayxJZNu2z4SV4gPtNt0kWVqtzbvv7yp0J5Z3B9yz1Ja9tQc6WO2nXHl/nwPUNccWHntiqtTL5VFKGaNaXZVwiV7XsmSUFqAufFu7XeaTjvQdksPMVDa7hWJrKX4OKpLOde9uMMLRMiRQBnYhaT9sdFPuymDcVe3c9TJ7ctApArv6vAXdLTQ62FwkIw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681805580; bh=wQ5kLeXsjGVtnoEV4Cgb4VDLwh/8B5yPirlWg30dmbU=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=q74D05SyusITXWo+PYDgiN2k/xhE7OXva5bWCyr3xrfyQSF8d7fLBqZDQXtpgRuZFXRucJYnNUZEs5bx8+ydYBuMFtLeUFAro292ozGYISi3VLINIQagiF7VrpZlVtK0MYUeGa4vb0sFznFEPXlFVchgysceGoQQR7M2RxojOJGYNlvqjyqqoNbpsoM3JKGdMn78ObhjoWNAfXtUvcUm0vO+irvrLGaIK87KryteaHHpgJ+RwvUa4QTsa6y4taVK8W8/oIOb4TOEm+ooWpKxyScrCohxBmJW+sTVxjk14qCzUr4CEF43UHwpNJn3q6EsHcUeKxe9EnfEnAF31BLEEA== X-YMail-OSG: xsSAhnsVM1nmojMbLfkOMWF9x7pGBIr4f.WShKWVoqEmw.7zikFmI5zdiMu7dkD 5D6Jw93WsbnKbKNYsFxZh.78bHTfHk2MgSDidvY.zUxJrcoelri2c3acdBEVFY8KRuNpkvY6NhmD rCu65WpfOuOEQ39ldSV39VVE28U7onqD_7cSrhcfh5rwdkD4QMuV20CswZZK7B2gsTDj_SzrICez VaYcSLsNPJGjnCVoa0LPwqxFWpI_hyFFUYBfTnd1qqlSUOJl_QQMD5A7p32SnePWC4I3zQRrQyK. eVFRo5sPLfQn6favhe1croZDdCHbDilbDVdE4A3M9sA.Lhs3HZ7JbUdFtp4RjgMky76pI1Sz_ZVb MwRkc6dHPtHNRo__t5RZMbkBmEXs_7OT61.KPxuI5VsL5KrF.X74nuMiGvkO1WLLQqjk52Hu5kLe YMMbgHxZw5eZNs6uxNSy_bg2QNd5nV64jdnEgwo.K5_FyoKF2KSXfoPFKyI7PhJcBZmIZHzqf8BM IGZvArgMGI2is.1BGk52lCHxFjFRNT1xKAoyEJ5NuKLHuC4QcUi9O49pLgc8euEySLDu2Inm43Ww FjFdPisV3ThnG2Td8nDpyiun09BKSAc_Rfv9IGFZO9e6E0HuOmr_pO_bVbMZ9lhJ1vZA950BXXJI RTgyWcR.i5P6E_ir7dqvCpSe4poiBhA2h_E131beTUpuhup7TY99AlMnalYchR5H.7zaH2Ykpydo LpUGmzmgs49c9xH2FMOLL_mDBtnB2OZUbdITZfv7SkKgzKGVOL850.HHywUezQdjaZIT0pj9Dg6C cxzYLViQG5MhuTQgwSgAVPRZHfSfBETxPQRQJa8nX00b.oyNn1XvjC4X.K4vFKS6sQfEfpWiEeSx 37leIgkmkEmLQvM0BBBz8VM4xMCG0UY1281PlnXmFbPLNrG6BkGgngu2plMOMcddcEWAOgITah7L T7wluMrENgUBhF8PzMp.LzoF3WQsOhzxO19xRysdosnYFe7CNVubZl1ij4fpRjUDojNxhvzoENTV S7boQDd86zFOii3PP6rxzcFuBOQjQ.hEtBt8CPI1TycUH7Aw2zxH4CSU6BCZHQDrUgwsxvfm6fpS Wm2dSRRUAWCHbHnEDIX9gPVAn577v6M4r76_uY06ZWGutsX4E1d_MxJP7rB5OCsR.x0TM3VDYDKC Cg4tDPp1tfcpRRt6oNfUlGKdtA581Komi4LN7rKVgpDAYBwRWlYld3I2NTuTRPF4JX870vyYqlfX LqnRjg91rsu.kZ3BQbDH.v.5OGh_9B930uheJ8FGS0_cs8i3lfjD84K9b_ZrXSMkl6.WKUaJRS97 onXSYDza3pCOhgVE8QHeY640nYNi74Lo9RJ..ff.fCu7yXfD44Y0BE.hEAKc14fWrD.SYa3wNZnO _UHvUnguWq.mlk2cwMAou8a.p464dkG61hfruMTGNWaim5GNxomeN4rl.UGd1Fk86nVT20oMp7pH 0c1DWF.FhWN.HRCAXpuZ0.MRY5Ro84zGGtKlPurC9vkHeH8Igj1lzR2HirgOaE1WGpyQRANzjK2n QK.MNn8d8H4AewQoTjzi3R_0xVHgdwjC.Kl8uB53.hKRiqTIc3sUdvuoBRUCCpfl0ELVhpm7_KSv SGEjZU_NhSqNuizHb0smlNVZHulue.ELy422fp3RzuSVut8Gf96RAmj11oDPhNrcS.EOe8gWA8jk 631JteuZmaVBkOllA3R7aUStlhssiGOHYcmg5xWj4SF5zOYjJzvFNSdBrs8JwrLQlHIVjsoCRsd9 oNyyDf1dju0wZYIfgEwjYhMYE0gSZltKL8c7UeDQi3OFaLFphfIecbMQ9d0BHpPmXKBsZHDVn.1Z .3RqgqaKhSgjK18qIw6FQ_PplBVYBpAP0P4WVOuUjLW36S2U9w2CVzkB0kvAVMZ9.IOnAD2K9o9h BPnlAdiD9DVElUHG9SeZ_z9RvhZhpO7yzeOoFPVxTBGWbxnc5NYvis_gXEU.00uefxhK244e1V5S Itr0WSv90ZPPeIxHghRTCgGkBKDfuzCDizjnL4nCCJDGfIFTto3z12Eq6PKDluTh_pbkKAk2plve nxC5BLwTnHk.ylS3p_0IHUAfSQ6MCWuuy3NyG5LgaH9dQuyIPJH2cUhGb2tYQwEXg4w59L7FMJXS NNqxlvVK43YFQKRtvpDBuFQ.C.GlG X-Sonic-MF: X-Sonic-ID: 4a0fc8f0-485e-4dc6-ae7d-0c1e7e7988d7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic313.consmr.mail.ne1.yahoo.com with HTTP; Tue, 18 Apr 2023 08:13:00 +0000 Date: Tue, 18 Apr 2023 08:12:57 +0000 (UTC) From: Filippo Moretti To: FreeBSD Current Message-ID: <214790055.5338048.1681805577606@mail.yahoo.com> Subject: Problem compiling py-* ports List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5338047_741423702.1681805577604" References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> X-Mailer: WebService/1.1.21365 YMailNorrin X-Spamd-Result: default: False [-3.87 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_LONG(-1.00)[-0.998]; NEURAL_HAM_SHORT(-0.88)[-0.877]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; RCVD_IN_DNSWL_NONE(0.00)[66.163.185.33:from]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.163.185.33:from]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4Q0xW26ZVPz3x3W X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N ------=_Part_5338047_741423702.1681805577604 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Good morning,=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 I = run this versione of Frrebsd and al py-* ports fail with the following mess= age.sincerelyFilippo FreeBSD STING 14.0-CURRENT FreeBSD 14.0-CURRENT #6 main-n261981-63b113af570= 6: Tue Apr=C2=A0 4 16:57:47 CEST 2023=C2=A0=C2=A0=C2=A0=C2=A0 filippo@STING= :/usr/obj/usr/src/amd64.amd64/sys/STING amd64 =C2=A0=C2=A0 return _bootstrap._gcd_import(name[level:], package, level)=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 =C2=A0=C2=A0 File "", line 1030, in _gcd_i= mport=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 =C2=A0 File "", line 1007, in _find_and_load= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 =C2=A0 =C2=A0 File "", line 972, in _find_and_load_un= locked=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 =C2=A0 =C2=A0 File "", line 228, in _call_with_frames= _removed =C2=A0 File "", line 1030, in _gcd_import =C2=A0 File "", line 1007, in _find_and_load =C2=A0 File "", line 986, in _find_and_load_un= locked =C2=A0 File "", line 680, in _load_unlocked =C2=A0 File "", line 850, in exec_mod= ule =C2=A0 File "", line 228, in _call_with_frames= _removed =C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py"= , line 18, in =C2=A0=C2=A0=C2=A0 from setuptools.dist import Distribution =C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", li= ne 34, in =C2=A0=C2=A0=C2=A0 from ._importlib import metadata =C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.p= y", line 39, in =C2=A0=C2=A0=C2=A0 disable_importlib_metadata_finder(metadata) =C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.p= y", line 28, in disable_importlib_metadata_finder =C2=A0=C2=A0=C2=A0 to_remove =3D [ =C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.p= y", line 31, in =C2=A0=C2=A0=C2=A0 if isinstance(ob, importlib_metadata.MetadataPathFinder) AttributeError: module 'importlib_metadata' has no attribute 'MetadataPathF= inder' ERROR Backend subprocess exited when trying to invoke get_requires_for_buil= d_wheel *** Error code 1 Stop. make: stopped in /usr/ports/textproc/py-pygments =3D=3D=3D>>> make build failed for textproc/py-pygments@py39 =3D=3D=3D>>> Aborting update =3D=3D=3D>>> Update for textproc/py-pygments@py39 failed =3D=3D=3D>>> Aborting update =3D=3D=3D>>> You can restart from the point of failure with this command li= ne: ------=_Part_5338047_741423702.1681805577604 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
=
Good morning,
        =             &nb= sp;  I run this versione of Frrebsd and al py-* ports fail with the fo= llowing message.
sincerely
Filippo

FreeBSD STING 14.0-CURRENT FreeBSD 14.0-CURRENT #6 main-n261981-63b113af5= 706: Tue Apr  4 16:57:47 CEST 2023     filippo@STI= NG:/usr/obj/usr/src/amd64.amd64/sys/STING amd64






&= nbsp;  return _bootstrap._gcd_import(name[level:], package, level)&nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;            &= nbsp;           &nbs= p;  
  File "<frozen importlib._bootstrap>", line 1030= , in _gcd_import          = ;            &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;   
  File "<frozen importlib._bootstrap>", lin= e 1007, in _find_and_load        &n= bsp;            = ;            &n= bsp;            = ;            &n= bsp;            = ;  
  File "<frozen importlib._bootstrap>", line 972, in= _find_and_load_unlocked        &nb= sp;            =             &nb= sp;            =             &nb= sp;      
  File "<frozen importlib._bo= otstrap>", line 228, in _call_with_frames_removed
  File "<fr= ozen importlib._bootstrap>", line 1030, in _gcd_import
  File "&= lt;frozen importlib._bootstrap>", line 1007, in _find_and_load
 = File "<frozen importlib._bootstrap>", line 986, in _find_and_load_un= locked
  File "<frozen importlib._bootstrap>", line 680, in _= load_unlocked
  File "<frozen importlib._bootstrap_external>"= , line 850, in exec_module
  File "<frozen importlib._bootstrap&= gt;", line 228, in _call_with_frames_removed
  File "/usr/local/lib= /python3.9/site-packages/setuptools/__init__.py", line 18, in <module>= ;
    from setuptools.dist import Distribution
  = File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 34, = in <module>
    from ._importlib import metadata  File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.= py", line 39, in <module>
    disable_importlib_met= adata_finder(metadata)
  File "/usr/local/lib/python3.9/site-packag= es/setuptools/_importlib.py", line 28, in disable_importlib_metadata_finder=
    to_remove =3D [
  File "/usr/local/lib/pytho= n3.9/site-packages/setuptools/_importlib.py", line 31, in <listcomp><= br>    if isinstance(ob, importlib_metadata.MetadataPathFind= er)
AttributeError: module 'importlib_metadata' has no attribute 'Metada= taPathFinder'

ERROR Backend subprocess exited when trying to invoke = get_requires_for_build_wheel
*** Error code 1

Stop.
make: stop= ped in /usr/ports/textproc/py-pygments

=3D=3D=3D>>> make bu= ild failed for textproc/py-pygments@py39
=3D=3D=3D>>> Aborting = update

=3D=3D=3D>>> Update for textproc/py-pygments@py39 fa= iled
=3D=3D=3D>>> Aborting update


=3D=3D=3D>>&= gt; You can restart from the point of failure with this command line:


------=_Part_5338047_741423702.1681805577604-- From nobody Tue Apr 18 08:46:44 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0yG05qYgz44lSr for ; Tue, 18 Apr 2023 08:46:48 +0000 (UTC) (envelope-from zirias@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0yG03yxpz3P4K for ; Tue, 18 Apr 2023 08:46:48 +0000 (UTC) (envelope-from zirias@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681807608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cHz8fHGFh51epg+0+1u0JIBEg95F6HPKKeO9KB+rc5w=; b=GJgQT81NSf6M6tOREEtMLKjM3kC3khEERt15AiwbDSMk7qdTQseRXNQ/EJ1JLcFBlVqg07 jH0xS6zVtUmHrE7akCXkzHPXzqV1ltkDi9B4wmal93yNNfWFUeW68P4+9CYoMVPKfzkD5a qlCwHuGX8mB3iTgL8/FjHAUFgNGaNl/a8MKrcUudeDT3BkzoF9KUWLCybMZiv8OoUj8SyB DH6kYXg/NbpIcBR/tJj7Cr/2+S6g1NOeae1iItJnulwBD7E2OBTEyhHKZC7qgePMa+VVE8 AVKREI6ROjZPhB/cuaefa2jd4VxukONMa0g5wvAbllSkFPnlZvBEvzqFiWN80w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681807608; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cHz8fHGFh51epg+0+1u0JIBEg95F6HPKKeO9KB+rc5w=; b=itukV2p+tIwP37zQEWtC/Ypu1hAEDadAxjbJv1Q/eoyJJfj5mKYKkrJV60KP14OJWgoLL7 MEhxtk7fU6NrjMP6zmaOEj/9NbormZ/gUDXsyojC33wEZjg2eY36ex2EZI4kKlywJQCY/C GauaNXi1e335FfWEOwV8B8UhHA/gC2+wtNgI9VyIZvW8q71fbSJOkUbwoLly5O5+xt+Iq0 hThwpyINGv5rmJ/al4Vqfm7klKGHSXLe9sIiwXJfYKXzgXOeSaMOECfgebNkK76/3Li32L dPpP4EuUG6LtPPQPNBgWdwCuCGAmu1o134tC9I9dfAl3xmsinz93HdaFVYsqAg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681807608; a=rsa-sha256; cv=none; b=xQazAfVC5/AzoQHauBZjY4CFvxUjt+Gc3DxxLlmKvhgkSPCDBame3fAZE2AsYx7Up9riUp hmuwWx6iUkC6D2xKueLsgUXeN87GEoiBe5l0AfsFnyCaPzGA5N7NkpJYh+mrM4JiGUxV6Y M4uSa4PuybRN7lO+cSF4AL/OfhirjoNNQIlvfUu6MA37VMeHnbJzS35bCgkeNEelBZ3b7Q dWdirv93GPM+Y54Qiwn6q8sdHblNdDYoB5KQ+qf4J562UUFWN9/T1Bmi4CMfQJiFkPwbhx XMGIbA9vF8Gm3iTz++3jekIb8OfuPG6tlwiquhMKgtYHu5bpm0fNgnjb8w2jHQ== Received: from stef.palmen-it.de (stef.palmen-it.de [IPv6:2001:470:1f0b:bbb:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: zirias/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q0yG02pSzz10Jv for ; Tue, 18 Apr 2023 08:46:48 +0000 (UTC) (envelope-from zirias@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=palmen-it.de; s=20200414; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:To:From:Date:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=cHz8fHGFh51epg+0+1u0JIBEg95F6HPKKeO9KB+rc5w=; b=aAv9IDIx81py1p7CZQvDIpK3Aw NMKNEhknKJOxBZzi/DwVqSOrgIlg8mn6OvyqmeyGBkQ9QCrAx6d1Yi0ogFYQumQNaQj5T51594ToI IPpO2pXoOGDf8dbaf6LWC+jPRefI6MaxoL41NP6Ak8PbE9qfKjQ5Jm3+C9D1A4CMa/vjL3k7u8wmZ MOjKoIFmaD29/mdmaKCFHuKVipAc/BMmnas7mQcKQ/PGlLZdwxD/Eau6HZKIcJ0+UNV57zvFfmowT 8ZCMwmdhEKvg2QCGzqjzWWY6duPtrdCI00/lK9wy0mI+kH9j/8ToKGrx+14FHL1VoAJmKbH5ppPvl 1LT8En8A==; Received: from [192.168.71.101] (helo=mail.home.palmen-it.de) by stef.palmen-it.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1pogz7-005NxB-Jd for freebsd-current@freebsd.org; Tue, 18 Apr 2023 10:46:45 +0200 Received: from nexus.home.palmen-it.de ([192.168.99.2]) by mail.home.palmen-it.de with esmtpsa (TLS1.3) tls TLS_CHACHA20_POLY1305_SHA256 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1pogz6-0006uD-Ug for freebsd-current@freebsd.org; Tue, 18 Apr 2023 08:46:45 +0000 Date: Tue, 18 Apr 2023 10:46:44 +0200 From: Felix Palmen To: freebsd-current@freebsd.org Subject: Re: Problem compiling py-* ports Message-ID: Mail-Followup-To: freebsd-current@freebsd.org X-Face: /1K@t"h.}e~pR@]c7HorQ!T`F^RJCa'BCr#e>IKA{>C/9OTGB4|xh"y2{?1Z5M i2w"AH^pN_LlHR^{+f',_Np~;.B;!M/bL}*qk]p5*r7F5vW};{:@4u5S?T&f0$7BJ-71Q5SV]:v$`5 A0[DZ:=?S52x8HJ~5@^P_\T@MsjG{R( Organization: FreeBSD.org References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="bxqszyolvkwhctcl" Content-Disposition: inline In-Reply-To: <214790055.5338048.1681805577606@mail.yahoo.com> User-Agent: NeoMutt/20230322 X-ThisMailContainsUnwantedMimeParts: N --bxqszyolvkwhctcl Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Filippo Moretti [20230418 08:12]: > =A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.py= ", line 31, in > =A0=A0=A0 if isinstance(ob, importlib_metadata.MetadataPathFinder) > AttributeError: module 'importlib_metadata' has no attribute 'MetadataPat= hFinder' Looks like you're building directly on your host (not in clean jails) and the build picks up "something" that's already installed. I'd probably try to uninstall all python packages, check whether there are leftovers in LOCALBASE/lib/python3.9 (and remove them if necessary) and then do a rebuild. You can avoid this class of issues using poudriere btw... --=20 Felix Palmen {private} felix@palmen-it.de -- ports committer (mentee) -- {web} http://palmen-it.de {pgp public key} http://palmen-it.de/pub.txt {pgp fingerprint} 6936 13D5 5BBF 4837 B212 3ACC 54AD E006 9879 F231 --bxqszyolvkwhctcl Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iNUEABYKAH0WIQRpNhPVW79IN7ISOsxUreAGmHnyMQUCZD5Y7F8UgAAAAAAuAChp c3N1ZXItZnByQG5vdGF0aW9ucy5vcGVucGdwLmZpZnRoaG9yc2VtYW4ubmV0Njkz NjEzRDU1QkJGNDgzN0IyMTIzQUNDNTRBREUwMDY5ODc5RjIzMQAKCRBUreAGmHny MQSRAQC12NYyXt4IxBth7V8KyYlxtvY1LbFJKzWJwDuRu9aASAEA+d7vkBTIo0G7 RITHmuHxE6AsEiC5DtBR5Ltara/6oQs= =VTrY -----END PGP SIGNATURE----- --bxqszyolvkwhctcl-- From nobody Tue Apr 18 09:41:07 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0zT05zdPz44qSX for ; Tue, 18 Apr 2023 09:41:24 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Received: from plan-b.pwste.edu.pl (plan-b.pwste.edu.pl [IPv6:2001:678:618::40]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "plan-b.pwste.edu.pl", Issuer "GEANT OV RSA CA 4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0zSy2pBKz4DRx for ; Tue, 18 Apr 2023 09:41:22 +0000 (UTC) (envelope-from zarychtam@plan-b.pwste.edu.pl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=plan-b.pwste.edu.pl header.s=plan-b-mailer header.b=GQCsKA7M; spf=none (mx1.freebsd.org: domain of zarychtam@plan-b.pwste.edu.pl has no SPF policy when checking 2001:678:618::40) smtp.mailfrom=zarychtam@plan-b.pwste.edu.pl; dmarc=pass (policy=none) header.from=plan-b.pwste.edu.pl Received: from [IPV6:2a02:22e0:cf00:1ff:9c57:baef:f047:f325] (mzar@[IPv6:2a02:22e0:cf00:1ff:9c57:baef:f047:f325]) (authenticated bits=0) by plan-b.pwste.edu.pl (8.17.1/8.17.1) with ESMTPSA id 33I9f9jf042241 (version=TLSv1.3 cipher=TLS_AES_128_GCM_SHA256 bits=128 verify=NO) for ; Tue, 18 Apr 2023 11:41:10 +0200 (CEST) (envelope-from zarychtam@plan-b.pwste.edu.pl) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=plan-b.pwste.edu.pl; s=plan-b-mailer; t=1681810870; bh=ypCl2pIeSTjod3NnJwjdhS/LsFXb8te4EHM0rwpbzAc=; h=Date:To:References:From:Subject:In-Reply-To; b=GQCsKA7MSLB7w+/TQ50thLW89WgClW8n/KBoStY0IOXevgbhkkyr+ZVK4YTnxsofa kYATchKmwIfpSQgdlw1ASVFoqSX4T2FigH/CuyEje4Di0WCpKeo/YFMAl67me2I0bI uMhVdTe7lG11lnT4YOpzBPyyFSvvMUdqsWL6LJwzPk8xmF3Cgp6UqN9ZGPBjmRcSA8 p9Io6kOelCuKrqqkVrfIM4x9TQ5ll+pGAC4esX/ps0oFAlyxTVCsk+/yUKuud7jTl9 wEuVog5idqkxyDLi+wDchvFzjpHo6z7zEsyyf3vuJA8dNJSZz95nHhsgSdh2SrG5Me L7VQ/As/X7EXg== Message-ID: <3cdd0420-7f42-9477-f4ac-0bf3a9b15181@plan-b.pwste.edu.pl> Date: Tue, 18 Apr 2023 11:41:07 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:102.0) Gecko/20100101 Thunderbird/102.9.1 Content-Language: en-US To: freebsd-current@freebsd.org References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> From: Marek Zarychta Subject: Re: Problem compiling py-* ports In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-3.80 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[plan-b.pwste.edu.pl,none]; R_DKIM_ALLOW(-0.20)[plan-b.pwste.edu.pl:s=plan-b-mailer]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_SPF_NA(0.00)[no SPF record]; ASN(0.00)[asn:206006, ipnet:2001:678:618::/48, country:PL]; MIME_TRACE(0.00)[0:+]; DKIM_TRACE(0.00)[plan-b.pwste.edu.pl:+]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Q0zSy2pBKz4DRx X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N W dniu 18.04.2023 o 10:46, Felix Palmen pisze: > * Filippo Moretti [20230418 08:12]: >>   File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.py", line 31, in >>     if isinstance(ob, importlib_metadata.MetadataPathFinder) >> AttributeError: module 'importlib_metadata' has no attribute 'MetadataPathFinder' I am also hitting this bug with Python 3.8, so I recently considered an upgrade to 3.10, but it comes out that flaw is not version dependent. > Looks like you're building directly on your host (not in clean jails) > and the build picks up "something" that's already installed. What is the culprit? Removing blindly all py- packages for all affected hosts with dependent software is not the best solution. > > I'd probably try to uninstall all python packages, check whether there > are leftovers in LOCALBASE/lib/python3.9 (and remove them if necessary) > and then do a rebuild. > > You can avoid this class of issues using poudriere btw... > -- Marek Zarychta From nobody Tue Apr 18 09:44:41 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0zXv4Xbpz44qjh for ; Tue, 18 Apr 2023 09:44:47 +0000 (UTC) (envelope-from chris@cretaforce.gr) Received: from relay3.cretaforce.gr (relay3.cretaforce.gr [195.201.253.216]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "*.cretaforce.gr", Issuer "RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0zXr4XVQz4Jd4 for ; Tue, 18 Apr 2023 09:44:44 +0000 (UTC) (envelope-from chris@cretaforce.gr) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=cretaforce.gr header.s=cretaforce header.b=KmFZJJHl; spf=pass (mx1.freebsd.org: domain of chris@cretaforce.gr designates 195.201.253.216 as permitted sender) smtp.mailfrom=chris@cretaforce.gr; dmarc=pass (policy=none) header.from=cretaforce.gr Received: from server1.cretaforce.gr (server1.cretaforce.gr [94.130.217.104]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) client-signature RSA-PSS (2048 bits)) (Client CN "*.cretaforce.gr", Issuer "RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1" (verified OK)) by smtp1.cretaforce.gr (Postfix) with ESMTPS id 6D00F1FF62 for ; Tue, 18 Apr 2023 12:44:42 +0300 (EEST) Received: from smtpclient.apple (ppp-94-65-240-220.home.otenet.gr [94.65.240.220]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: chris@cretaforce.gr) by server1.cretaforce.gr (Postfix) with ESMTPSA id 39E5211F25 for ; Tue, 18 Apr 2023 12:44:42 +0300 (EEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cretaforce.gr; s=cretaforce; t=1681811082; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oLhck9kx3JOLRo7CcKTvugKn3BNvld2lzHn2bsO+YjI=; b=KmFZJJHl1YkZ6B/QYb43F5j2M7+nwpSJo/+lcnNGmYb4K6OWxwL9lIOM1qr4I/iezP6A88 2eFaZmuh8gbXVzxE6Mckj4qBvlFL2VZu/x8UKrmsu0L9QVqwB7KBkuSn6rXK+/guPoJH7n t2WecnfNSbvcCiGdXed8QBhoUSGzIQo= From: Christos Chatzaras Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: Problem compiling py-* ports Date: Tue, 18 Apr 2023 12:44:41 +0300 References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> To: FreeBSD Current In-Reply-To: <214790055.5338048.1681805577606@mail.yahoo.com> Message-Id: <81A4E134-807C-43A5-998E-E08C6FFCF303@cretaforce.gr> X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-4.60 / 15.00]; DWL_DNSWL_LOW(-1.00)[cretaforce.gr:dkim]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[cretaforce.gr,none]; R_DKIM_ALLOW(-0.20)[cretaforce.gr:s=cretaforce]; R_SPF_ALLOW(-0.20)[+ip4:195.201.253.216]; MIME_GOOD(-0.10)[text/plain]; RCVD_IN_DNSWL_LOW(-0.10)[195.201.253.216:from]; TO_MATCH_ENVRCPT_ALL(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; FREEFALL_USER(0.00)[chris]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_VIA_SMTP_AUTH(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; DKIM_TRACE(0.00)[cretaforce.gr:+]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; ARC_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:24940, ipnet:195.201.0.0/16, country:DE]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4Q0zXr4XVQz4Jd4 X-Spamd-Bar: ---- X-ThisMailContainsUnwantedMimeParts: N > On 18 Apr 2023, at 11:12, Filippo Moretti = wrote: >=20 > Good morning, > I run this versione of Frrebsd and al py-* = ports fail with the following message. > sincerely > Filippo Maybe this helps: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D269545#c6= From nobody Tue Apr 18 09:57:07 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0zqD5Qxcz44rNl for ; Tue, 18 Apr 2023 09:57:12 +0000 (UTC) (envelope-from zirias@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0zqD4yp1z3Pmb for ; Tue, 18 Apr 2023 09:57:12 +0000 (UTC) (envelope-from zirias@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681811832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=SDnWHgfzTvP2uk0rGnUSJcT/H9tCfc5qt52yJpk4fj4=; b=fPCE7tNH4ojeMZX2po9A5USD5x0GNjYDpQjeLXdkQ/+GeRFw+AbEX26kwoR4KVSatWtFLv 0/r7pxqMZaNBxsgtrnSgXCdSkPJWEWpumLstQMdMZ5BNgTUUyrYzlMUl+SHjQjYbA2aCMJ 6m/ZknQ8QVCaVw7uzKIKYA7aNYY0r3MggQdJWL4hDrF4FsVKBUfQ2nLeU57FHq/pwTmzYH 68VSNnAjo9sIg6cSLdwCiD+4j87gA0mXsSnTH002p4L+9jl51UKf5UZNG2W7zl0URp+5oW 8rXVNTdZ8Uxh4aLdogkcbJ6HR483k99QlHq3iPG/7GdzMzdK6bbXkUvxvOP8Tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681811832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=SDnWHgfzTvP2uk0rGnUSJcT/H9tCfc5qt52yJpk4fj4=; b=MPlDOT3A7t2a9N6fmqXdSDyhDu27opYcvIYppmauyttoRcezzrGNHAQBzL3FoSOb8LQfCH o8JA9BmLlL7cY5PsF/SGNRrd1FQBu5GwXHEqWR0Nui3kS9jEmCqrmUgcWOy/ZDPRQ1XZhx 0sR2zawwfhKVXr3t+kdXZjISvYlvauqXuc4ccsqXt/mG6YKuDEZTgppT+6IHsSH2qfzxBp uKbM9xLrl0v4YIHw1+xT2na1763QFMYtLwCkhXzn/9/TyVetQzmm8sUdihyQQLkvmdAq7W k2Q4dxPMjk7TaaZEGpG3rh01jo/tC4RgPDd1+PHG4GZO/T9SBmbcx+OvQX+8Kg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681811832; a=rsa-sha256; cv=none; b=fLCuDK5qSE2TLx+aYdzXrf6Saox1hc1GSwdF/tAvFtuDSb/OqiqiLaTklO6+Y1gz0+mkb8 A2OEZ8Y6bzpM9Q3dFHwLxoojZFgJwR1av11ST8/Op4hqa3Txf3Mts/1UTCyPpg86jp8jRA gOAk0kft2070+VVV4CNArN1t5gZ2oXSTMSoXhYXQl7szB2d8V4Un2A6GFSJp48Mg/OyYuQ A3F1iMppyv3t2ih7inKcrAqddqx7KWWqCqNSEHfRMF2gDpYcYBpj/OU3ajyTdOruv7v2XN ogWwAW0dmB1DgIeVqDJ3uE/i8oxVwiAnclLtypjnaEt/zxiElfextg6GLZFZ7Q== Received: from stef.palmen-it.de (stef.palmen-it.de [IPv6:2001:470:1f0b:bbb:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: zirias/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q0zqD42Xsz1185 for ; Tue, 18 Apr 2023 09:57:12 +0000 (UTC) (envelope-from zirias@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=palmen-it.de; s=20200414; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:To:From:Date:Sender:Reply-To:Cc: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=SDnWHgfzTvP2uk0rGnUSJcT/H9tCfc5qt52yJpk4fj4=; b=p+9sQgNuMqUhrPtmD+kyAjK7hL TCWETr8HkMZqkEGBGjszsDg0j4eXESsJir5L5I3BYV2o9+dRk7VB186doWFOy4MdSb3dUapu+s5sZ KDCVJJbtft0TEmVAL8SN0fFelgCa+oZW5TxkI7KKEnkwMU22EaWRyOstSKQwhLcUshrSiGQqkdaQC WvEXVmYm/EoS8T6w+nWzSlZ0ba5ScRVks+PmAgDLQoXsWD6nTQ7DlaLRcv23nsk0neLa+cqEBGhuP CxXvZ1gZxkLJ1Jx5636cl8L4ptvEmRQg75i/3ERI3fOdJVHCFCcfaFEZeyFyHwt2x/NW6WR5dgL6Q a0/5jRvA==; Received: from [192.168.71.101] (helo=mail.home.palmen-it.de) by stef.palmen-it.de with esmtps (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1poi5H-005OA9-6Z for freebsd-current@freebsd.org; Tue, 18 Apr 2023 11:57:11 +0200 Received: from nexus.home.palmen-it.de ([192.168.99.2]) by mail.home.palmen-it.de with esmtpsa (TLS1.3) tls TLS_CHACHA20_POLY1305_SHA256 (Exim 4.95 (FreeBSD)) (envelope-from ) id 1poi5F-000Jav-GW for freebsd-current@freebsd.org; Tue, 18 Apr 2023 09:57:09 +0000 Date: Tue, 18 Apr 2023 11:57:07 +0200 From: Felix Palmen To: freebsd-current@freebsd.org Subject: Re: Problem compiling py-* ports Message-ID: Mail-Followup-To: freebsd-current@freebsd.org X-Face: /1K@t"h.}e~pR@]c7HorQ!T`F^RJCa'BCr#e>IKA{>C/9OTGB4|xh"y2{?1Z5M i2w"AH^pN_LlHR^{+f',_Np~;.B;!M/bL}*qk]p5*r7F5vW};{:@4u5S?T&f0$7BJ-71Q5SV]:v$`5 A0[DZ:=?S52x8HJ~5@^P_\T@MsjG{R( Organization: FreeBSD.org References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> <3cdd0420-7f42-9477-f4ac-0bf3a9b15181@plan-b.pwste.edu.pl> List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="iyybidjp2oiqywta" Content-Disposition: inline In-Reply-To: <3cdd0420-7f42-9477-f4ac-0bf3a9b15181@plan-b.pwste.edu.pl> User-Agent: NeoMutt/20230322 X-ThisMailContainsUnwantedMimeParts: N --iyybidjp2oiqywta Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable * Marek Zarychta [20230418 11:41]: > What is the culprit? Most likely some leftovers from older package versions (so it's only a problem when building in the live system. A clean jail won't have the offending files). > Removing blindly all py- packages for all affected > hosts with dependent software is not the best solution. Depending on your situation, it might or might not be faster than trying to identify the exact offending files and just removing them ;) --=20 Felix Palmen {private} felix@palmen-it.de -- ports committer (mentee) -- {web} http://palmen-it.de {pgp public key} http://palmen-it.de/pub.txt {pgp fingerprint} 6936 13D5 5BBF 4837 B212 3ACC 54AD E006 9879 F231 --iyybidjp2oiqywta Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iNUEABYKAH0WIQRpNhPVW79IN7ISOsxUreAGmHnyMQUCZD5pc18UgAAAAAAuAChp c3N1ZXItZnByQG5vdGF0aW9ucy5vcGVucGdwLmZpZnRoaG9yc2VtYW4ubmV0Njkz NjEzRDU1QkJGNDgzN0IyMTIzQUNDNTRBREUwMDY5ODc5RjIzMQAKCRBUreAGmHny MTT8AQCK3l34liLMVhWll86bX1Xz6IZ7WzZfRIzEoaY0cJJvNgD5AULkDOkZiIqm 6gChA2cW/4uIBHOJRfOsSbtnXAk2vAk= =R/sv -----END PGP SIGNATURE----- --iyybidjp2oiqywta-- From nobody Tue Apr 18 10:35:57 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q10hC1VCGz44tZC for ; Tue, 18 Apr 2023 10:36:11 +0000 (UTC) (envelope-from freebsd@grem.de) Received: from mail.evolve.de (mail.evolve.de [213.239.217.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mail.evolve.de", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q10h96Vbjz3KNR; Tue, 18 Apr 2023 10:36:09 +0000 (UTC) (envelope-from freebsd@grem.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=grem.de header.s=20180501 header.b=ROuB1I0+; spf=pass (mx1.freebsd.org: domain of freebsd@grem.de designates 213.239.217.29 as permitted sender) smtp.mailfrom=freebsd@grem.de; dmarc=none Received: by mail.evolve.de (OpenSMTPD) with ESMTP id cb0a146d; Tue, 18 Apr 2023 10:36:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=grem.de; h=date:from:to:cc :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=20180501; bh=hmlDaJOc XugcwXOrL7fIcFiUBD8=; b=ROuB1I0+0w/TNVq6Wy8ArE0XSveMCS2M6kEKu9l9 FxX5bboUNled7miH8XvvM9VQ+1iTa4MvUrEahfiQ4Z5QwrMc/PPbvATSOUv37IB3 GjPzjpSey0Xsl493Jyn7XWGXAQ92q4hsjo93N/+nSmTV2qYxTZ7pWDYAXvMnVQrR L9bo/GbksnmGWmzYvVpIZSt6kz/gv6aZDWs1DaSuA7dVuu1ZUyvTttvq6llG/QNm nXPQ2m1GQqkjbEwK1QeqxIQh7VCSoyWibRI9p6WNM5ONvaqU6xp6eM2CDXXbAsaR aGPZWjqTEWCjgu6RKev9PR8Geto9Yalz0mAHHwWyp/vH5g== DomainKey-Signature: a=rsa-sha1; c=nofws; d=grem.de; h=date:from:to:cc :subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; q=dns; s=20180501; b=s+ YTzwdJNvXdWEkzttiCrdPEmInnYWXwb5yIeJH3yPqK544fB5DZqeyP1U/6/gt+Ep DFSnhcc5okZaJvmjUeflB9BUTh7L1MP2QNvIt5sH3rQBSaJhjnGh372Qy9qC3iYl HXHnFt6pIMLIt/ucC7GbHOZuMx4W1fV8t8E5OyptOVvWZke+03i0v19Deza009v1 ZXuYwIzkYElI+NjEIrm6a3acXSB5SjHngBJtuIXcWFiqd9WImCjXxXnGlGIy5pIL gVeamz/DxLZVrnx9ir+hsEiWIytDTZfXpizF5CJPpBPBeDBPJni/IP734XD7Mr9N fTJmVIU6mDYsZ07gm8Dw== Received: by mail.evolve.de (OpenSMTPD) with ESMTPSA id aa93a051 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Tue, 18 Apr 2023 10:36:01 +0000 (UTC) Date: Tue, 18 Apr 2023 12:35:57 +0200 From: Michael Gmelin To: Warner Losh Cc: Michael Gmelin , Ceri Davies , Conrad Meyer , "freebsd-current@freebsd.org" Subject: Re: Reducing SIGINFO verbosity Message-ID: <20230418123557.443cd189@bsd64.grem.de> In-Reply-To: References: <20210520180155.3e23500e@bsd64.grem.de> <20210520185917.GL14975@funkthat.com> <20220619140624.21263e46@bsd64.grem.de> X-Face: $wrgCtfdVw_H9WAY?S&9+/F"!41z'L$uo*WzT8miX?kZ~W~Lr5W7v?j0Sde\mwB&/ypo^}> +a'4xMc^^KroE~+v^&^#[B">soBo1y6(TW6#UZiC]o>C6`ej+i Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwBAMAAAClLOS0AAAAJFBMVEWJBwe5BQDl LASZU0/LTEWEfHbyj0Txi32+sKrp1Mv944X8/fm1rS+cAAAACXBIWXMAAAsTAAAL EwEAmpwYAAAAB3RJTUUH3wESCxwC7OBhbgAAACFpVFh0Q29tbWVudAAAAAAAQ3Jl YXRlZCB3aXRoIFRoZSBHSU1QbbCXAAAAAghJREFUOMu11DFvEzEUAGCfEhBVFzuq AKkLd0O6VrIQsLXVSZXoWE5N1K3DobBBA9fQpRWc8OkWouaIjedWKiyREOKs+3PY fvalCNjgLVHeF7/3bMtBzV8C/VsQ8tecEgCcDgrzjekwKZ7TwsJZd/ywEKwwP+ZM 8P3drTsAwWn2mpWuDDuYiK1bFs6De0KUUFw0tWxm+D4AIhuuvZqtyWYeO7jQ4Aea 7jUqI+ixhQoHex4WshEvSXdood7stlv4oSuFOC4tqGcr0NjEqXgV4mMJO38nld4+ xKNxRDon7khyKVqY7YR4d+Cg0OMrkWXZOM7YDkEfKiilCn1qYv4mighZiynuHHOA Wq9QJq+BIES7lMFUtcikMnkDGHUoncA+uHgrP0ctIEqfwLHzeSo+eUA66AqzwN6n 2ZHJhw6Qh/PoyC/QENyEyC/AyNjq74Bs+3UH0xYwzDUC4B97HgLocg1QLYgDDO1v f3UX9Y307Ew4AHh67YAFFsxEpkXwpXY3eIgMhAAE3R19L919nNnuD2wlPcDE3UeT L2ytEICQib9BXgS2fU8PrD82ToYO1OEmMSnYTjSqSv9wdC0tPYC+rQRQD9ESnldF CyqfmiYW+tlALt8gH2xrMdC/youbjzPXEun+/ReXsMCDyve3dZc09fn2Oas8oXGc Jj6/fOeK5UmSMPmf/jL+GD8BEj0k/Fn6IO4AAAAASUVORK5CYII= List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; R_SPF_ALLOW(-0.20)[+ip4:213.239.217.29/32]; R_DKIM_ALLOW(-0.20)[grem.de:s=20180501]; MIME_GOOD(-0.10)[text/plain]; DKIM_TRACE(0.00)[grem.de:+]; MID_RHS_MATCH_FROMTLD(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_EQ_ADDR_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCPT_COUNT_FIVE(0.00)[5]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[grem.de]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:24940, ipnet:213.239.192.0/18, country:DE] X-Rspamd-Queue-Id: 4Q10h96Vbjz3KNR X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Thu, 23 Jun 2022 11:15:55 -0600 Warner Losh wrote: > On Sun, Jun 19, 2022 at 6:06 AM Michael Gmelin > wrote: >=20 > > > > > > On Fri, 21 May 2021 08:36:49 -0600 > > Warner Losh wrote: > > =20 > > > On Fri, May 21, 2021 at 7:38 AM Ceri Davies > > > wrote: > > > =20 > > > > On Thu, May 20, 2021 at 03:57:17PM -0700, Conrad Meyer wrote: =20 > > > > > No, I don=E2=80=99t think there=E2=80=99s any reason to default it > > > > > differently on stable =20 > > > > vs =20 > > > > > current. I think it=E2=80=99s useful (and I prefer the more verbo= se > > > > > form, which isn=E2=80=99t the default). =20 > > > > > > > > I agree that there's no reason for the default to be different, > > > > but I would say that it is much easier for someone who knows > > > > that there is a default to be changed to change it, than it is > > > > for someone who does not. Therefore, if the information is not > > > > useful to someone who does not know how to get rid of it, then > > > > it should default to not being displayed, IMHO. > > > > =20 > > > > > > I plan on changing the default for non-INVARIANT kernels back to > > > the old behavior. > > > > > > INVARIANT kernels will keep this behavior because it's a debugging > > > kernel and this is one more thing to help debugging problems. > > > =20 > > > > Did this ever happen? I just installed a fresh 13.1-RELEASE > > production system (non-INVARIANT kernel) and it seems like SIGINFO > > still outputs kernel stack information. > > =20 >=20 > https://reviews.freebsd.org/D35576 for those who wish to weigh in. >=20 > Warner >=20 >=20 Hi Warner, I just installed 13.2-RELEASE, seems like this was never MFCd (it is in main, but not in stable/13 or releng/13.2). TBH, I could've checked myself back then (it's so easy to forget to MFC). Cheers Michael p.s. Learned about it by hitting ctrl-t to check if freebsd-update on my slow test machine is actually alive :D --=20 Michael Gmelin From nobody Tue Apr 18 11:49:06 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q12Jg5kK6z44yXj for ; Tue, 18 Apr 2023 11:49:23 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q12Jg5CmVz3RKt; Tue, 18 Apr 2023 11:49:23 +0000 (UTC) (envelope-from otis@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681818563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1rEoO7aA0c6DApOAXXgK4vnJp/BRUMCMuWuQl8oGpKU=; b=yiQKjLIJ8actSMkg0oU0R36kE5T1KJmncmhvFbBwXeh5GFhU+EKLpNVACZgShRYxWz4trS BuvCatcTKIBzdSyKlH57qXwpotOnXpv4DhdDWPNR6xK2B8zW/rQRfDh+j1qHUft60qVdLb jeNBNCWJa9vAs7oiz1plqaD1Kc3Rg7Wbb4rfTkTHpqesu2m/nEYsDurnuu47l43QIErhpD RNW0+cQH0THk1RLdtjcinxaEJhvw6P/eeD7n9Bm9X5wEPR36o71VXEXbqydS3Q0YYWZuRU bv4qh/WaD15lMV1x+Q5y0vkvhTmaDd6Up52C1wlexGdAGMbVZxrdbHPup8A/Cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681818563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1rEoO7aA0c6DApOAXXgK4vnJp/BRUMCMuWuQl8oGpKU=; b=fPWwseWN0hE+Q5nCes4XHWc15c5wxGbKCFphcZCKzqwGvXyq79XgwtTvotfPbpcl3CL5yO OHvidBmRhvRmoHnr9I96COuvMkzwi4+DcUUej/tGATDr9q9eU2926ohVCNMGZb0RYl7htY px+wGedOOiYnHfK16TVmXqiZCiUV9Kfk71063tV0/epiUp+Wp1cn7gB5MJb1/dCAeTLdFH 8TNj/Qb7DMRPPSNqoln6C7uQ77+0c4QWcKukcMF2VtRFk/TDdVrHT+JxuxdfTuY9rQXA8M RXrVmI3Eq1Ri/HMGcOd7INfSuMf35NcFstGvTr7g22p0xtrdz0GYK+EIygmlVA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681818563; a=rsa-sha256; cv=none; b=qvUr/C5HyUaQuosJQUgSY1EISdxkdjeSECkj0TSSHYuNBumxKkGOmIpelrbSPOuJ8wtk+s VynPPwH0TJKtFmRtwZeOeMODzmjrV6L3Av5oGF9DKdWv/bi9F2taBceqTLj8NxfMy+pYzC 6dAaTa0WS1ou5PMnNdqLjBsYjaV0E88e5Vl8Y8YKwSpTA989hhZj8M7/kp5qhDNS4JyFML rcd0RsRetQ9N56id6/jGxhO5kWUziPbOnWeG0Bhk29iWuo+wn8i0wmOKygrMQ5gjtW1+Qk pMM8Kc37HaqdXqZNc2dSYC3zs+J3DMKkCD9N+R2mCgXedldzJhS04q/jjJ4VmQ== Received: from ns2.wilbury.net (ns2.wilbury.net [92.60.51.55]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature ECDSA (P-256) client-digest SHA256) (Client CN "svc.wilbury.net", Issuer "R3" (verified OK)) (Authenticated sender: otis) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q12Jg3WxVz12kp; Tue, 18 Apr 2023 11:49:23 +0000 (UTC) (envelope-from otis@FreeBSD.org) Received: from smtpclient.apple (unknown [217.73.28.193]) (Authenticated sender: juraj@lutter.sk) by svc.wilbury.net (Postfix) with ESMTPSA id 97D4045CD86; Tue, 18 Apr 2023 13:49:18 +0200 (CEST) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: Re: another crash and going forward with zfs From: Juraj Lutter In-Reply-To: <8ea96ab9-89c5-d05a-95d1-ef71663f28bb@FreeBSD.org> Date: Tue, 18 Apr 2023 13:49:06 +0200 Cc: Mateusz Guzik , Pawel Jakub Dawidek , FreeBSD CURRENT , Glen Barber Content-Transfer-Encoding: quoted-printable Message-Id: <8D520540-8C29-47D3-9E4F-4F8067F3E6BB@FreeBSD.org> References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> <8ea96ab9-89c5-d05a-95d1-ef71663f28bb@FreeBSD.org> To: Martin Matuska X-Mailer: Apple Mail (2.3731.500.231) X-Spam-Status: No, score=-2.8 required=5.0 tests=ALL_TRUSTED,BAYES_00,TW_ZF autolearn=ham autolearn_force=no version=4.0.0 X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-14) on ns2.wilbury.net X-ThisMailContainsUnwantedMimeParts: N > On 18 Apr 2023, at 09:46, Martin Matuska wrote: >=20 > Btw. I am open for setting up a pre-merge stress testing >=20 > I will check out if I can use the hourly-billed amd64 and arm64 cloud = boxes at Hetzner with FreeBSD. > Otherwise there are monthly-billed as well. I can provide a bhyve VM on some of my hosts. We can discuss it = off-list. jl =E2=80=94 Juraj Lutter otis@FreeBSD.org From nobody Tue Apr 18 12:41:06 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q13SR5Skgz453BZ for ; Tue, 18 Apr 2023 12:41:11 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic320-22.consmr.mail.ne1.yahoo.com (sonic320-22.consmr.mail.ne1.yahoo.com [66.163.191.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q13SQ4KmZz3pV6 for ; Tue, 18 Apr 2023 12:41:10 +0000 (UTC) (envelope-from filippomore@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=N0JBnjl0; spf=pass (mx1.freebsd.org: domain of filippomore@yahoo.com designates 66.163.191.84 as permitted sender) smtp.mailfrom=filippomore@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681821669; bh=sLLf/yPvxx+uFJzkv35YigzB/cK98OG+12mw39s8hw4=; h=Date:From:To:In-Reply-To:References:Subject:From:Subject:Reply-To; b=N0JBnjl03pA/kT10xYrnaL6RCxz2ixwiWUOICDomXIAZrn5yS7eWqL6r//tNndou3t8qbaBEMPUMwl0ysUvRaXbmkqJwsYuwMSUWxaDYNKBQURIp4ZDlERf281UJEVQpd5iz7WRQVcQjszu32kSagG3WcXgaQEaqRj5EqcAQikhYb8jyGx6rGFkftnmqXFVjtQ7QSDg3BlczVHssL+2WM2iWw0R2rXToOO0/uQwe5lcbNvfTKNS4yNDlf70kXTy5PlIyvtlBD0Fux2KmS5LpMsu/znGrGcxGltsTGo8MaFPdeU/Cl3rjcsNvO0A81odCdjIHH2Vy4AGTsNG40myR3w== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681821669; bh=/bS+COYVFhp7zHJIgD7zBFPKBv9TcSoqhDXLnaszmpc=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=cYC1RSu2tk2TnSzZWM5rtZulwsxl6/b6vdtPsj/OfJtbfS6c5+ulQ9C8W5SqJai84qywbQxN2jwfSUNIXtw7o9ref20stHTZNp3CTRBvw5vo8I5Lk8Jj3ww7fn7fXMcnC63Q4BbUNBTlHtHkMnlPpAEeUCZfYIaObvhmiysP/AaitF8P2FouF0hOfgfoMWLVBrw1BDEALKif0FZarfmN2WjJpBU+iomqQQI28ULu7YAxZd1YrLVflDd2VxAv/PnGpcVWjN+Trg5qodjhkOKuxKgJYjLTfIE//xyuwnz5xG3BXvOEBgoB5e8/vx+y3PcAXd/tIA4Wl4FqQMvGLyRtFw== X-YMail-OSG: rq_VYGIVM1lKXUHJfNAxxk1nkKa7Ymb4TC6cnx_FuBiNJUm.k54.ubwYqiIXncY YyKiuWFteiKfr877v_cedT3utQt515mQWimB6lHME05RzdlPcXdQZK7SlEpJawvl3UReZjBeg0hl hQ.Giyql7lch9cNXhQFnHo3mDNq69vqGR7udnpZPpMXEHn40gDwBq21qBq67SWkSfPad83ronIk8 FXpdcNqolgBAh69Cu_OxxTODKBmvm25RgXaq4VjQgV.JsRzhiHq82Ur_b5uKPWMeX.ub1tzzLqJh IWHMip3a5q_M5ULzC5Rt2H3EZheoDQQttDfGt2SzjSwJZDAHCek45hRsOccugOfjFiUhOa2WXFZF KsGt82WSbi4KAW0CBQatORuc5wmyHxsMZrD8KX0ZPd9ZWaPJYOr2s63bNlYWTF9g4pYwfP2vIxOD DdxOMgaInLR_6N3EoE4fcRlMkvW6Pg3D2Mz9LOWptZj8yp1RbE1GQMLxJl8IwUHfWME598kzQ5C2 zxXqpYApFqS1vfPHaXCKV432W0SjIPVv68vdSRLBrYJbTXpAejwuso0Z_1dyh2fRG3XDZ3LpTbCi 68Sm16ksEkzlvb2kV_GFQFOMcsdcxWdCEU6Sb_Bc22ajfPA_uvvOPqTVlbbubKM2p82.mgT1c6fs ghuWtjkplJE96QHOwCEAT2bu8GGt7bTu1agxzBI7tgkHxgUKv_6s41rtPqAyw9iF3PUJVqWFJzkx SVkbQyJMwDXIILFJSj01sJLb.0C2R8sNoTmR7cMSQDWqpKtOYyCCXLK_XYpISkfTe1Lpoxp_c.LU RghOZixrWYMcPPchVYG30N98o9xIpzuA4KLFhLLVJN8MuiNtgJvubg92YQSYZLaxmM1dH8qJjXPS LPQ2oXBhFVokQKeGWqG2SbFZcah0IuO7aOroL4Cj_tphoGeTYPPrWjhaZJ1o6tfL8I7kUeNYijOL GOWGXUigtgFnQaBv1n_6Nzkdh5sffQkMOeUAGYhfhmncn46fh.jNBqMxjQDHu8XyrwxgcwajzAJB 9pdIWaU0LNOai3BPCYMtsliPk30ERnxLxoaF.ZKNZMMk8E7n_5vlSim7lEOJxbz9tha4hQfvTsCP _yOdtTgddUItKXvJPvLpwiK.NGM4sGwrNvQ15QAW9n9OGgsWtRuT3PCrZn2zwWebsLSVgypHo8AK nzQeoghiuWul3pmpPHKz9ovqluA0duRIu9DDa22gg1ZRZrtuSvocgUqn4w0UxiEpzA3.FthJ4zIF Tb1hakSAKg5eNsHcHIu3sGxy_YuCo0v5vo80D5sUfdEybf1XTQitOyC_eZ5sokUJPS5wCxOPVdXL ClPKC7_JoEB526OR3KUcaapOb8hydYmu9T3WcBylcsQOP3uPmEJFQw2z.LuZA0Pkf4c9vp4qwNUj DKlXKxfocp2kgkmLxXAKCT2OWZz_ohnSv8b0rTKLI8XdQHHP5BOJxnB0PA8NO_bKBDTxv4zyW2YR 2zo3X7DELzr8Q_PpuWRsX.S7FnsKkbigaYzHT.RGF8rKyoTQ.WLrdI74mf7Dqkp4WFxWcawpEN4f AKkbGItLIHcr41GMYIzCJGzYq2_VuNGJgBnpW6i1vujwK1bZWeq2o3.EzdnQRhZg_PN.fves4KzT eZOahhzzW21ZTVdUmJHrP7IkOwFPtW1wTz2vAeL5QahrhNTPZxc2M61CMsig9bqRqJLj367CKK4x fEm9h4Ovnu4KVFFvUxPb3fPullC3S98izGXS3_Qs.q9TtLSEoSm9LxXLFqGGF6Kx5TVA8_43kAir Cm_TJ1.PKx8y_xtQgn0K2HaTjRBPPsIMw0RJMNz7yZ5.3ac8_Exyjf3pSBqKIITycQBpfY7m87Vf n5Ugy8vLIVjBAoTPCc1JvACFYgu3vIC2uKBfOh41FIkAl6517xzeXkma.hT5dt.C.1GcQF8zVX4s sF4uri52Ba3YsZhshCInzgta0YJTmqFoK77bIHAp3c6b7Api4MIXmWgYLz46rd6QAYByjBMV8fNd trmlCjKOQZ_2kApfTHMQ0QCZQelnOEUsHN8kotBNXY.BULvTkm96S2hdnIZpIGDcbYows_X4E7V0 9QI5phTHDWBqeM9q.WQSjkHW5XzG5P3hzXk7bv8Xjhj7Ga0fhC95dRyeHSxc7VQxOmSj8r42C9Ws EBmJ9vSKaUMqH1bhRgTKj2WApDyvSZoPNIPvR.AYu5eH_970rpoqp0v_n3.Cosjkp4OtJZnw- X-Sonic-MF: X-Sonic-ID: 9031c59f-42fd-42d1-af9e-24494f43583e Received: from sonic.gate.mail.ne1.yahoo.com by sonic320.consmr.mail.ne1.yahoo.com with HTTP; Tue, 18 Apr 2023 12:41:09 +0000 Date: Tue, 18 Apr 2023 12:41:06 +0000 (UTC) From: Filippo Moretti To: "freebsd-current@freebsd.org" Message-ID: <1408490282.5393219.1681821667051@mail.yahoo.com> In-Reply-To: References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> <3cdd0420-7f42-9477-f4ac-0bf3a9b15181@plan-b.pwste.edu.pl> Subject: Re: Problem compiling py-* ports List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5393218_1420843073.1681821667049" X-Mailer: WebService/1.1.21365 YMailNorrin X-Spamd-Result: default: False [-2.94 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; NEURAL_SPAM_SHORT(0.06)[0.058]; TO_DN_EQ_ADDR_ALL(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; RCVD_TLS_LAST(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; FREEMAIL_FROM(0.00)[yahoo.com]; RCPT_COUNT_ONE(0.00)[1]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4Q13SQ4KmZz3pV6 X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N ------=_Part_5393218_1420843073.1681821667049 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable =20 After following the instructions provided I get the following:=3D=3D>>> All= >> py39-pygments-2.14.0 (1/9) =3D=3D=3D>=C2=A0 Building for py39-pygments-2.15.0 /usr/local/bin/python3.9: No module named build *** Error code 1 Stop. make: stopped in /usr/ports/textproc/py-pygments =3D=3D=3D>>> make build failed for textproc/py-pygments@py39 =3D=3D=3D>>> Aborting update =3D=3D=3D>>> Update for textproc/py-pygments@py39 failed =3D=3D=3D>>> Aborting update On Tuesday, April 18, 2023 at 09:57:31=E2=80=AFAM UTC, Felix Palmen wrote: =20 =20 * Marek Zarychta [20230418 11:41]: > What is the culprit? Most likely some leftovers from older package versions (so it's only a problem when building in the live system. A clean jail won't have the offending files). >=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 Removing blindly all py- packages for all affected > hosts with dependent software is not the best solution. Depending on your situation, it might or might not be faster than trying to identify the exact offending files and just removing them ;) --=20 Felix Palmen =C2=A0 =C2=A0 {private}=C2=A0 felix@palme= n-it.de -- ports committer (mentee) --=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 {w= eb}=C2=A0 http://palmen-it.de {pgp public key}=C2=A0 http://palmen-it.de/pub.txt {pgp fingerprint} 6936 13D5 5BBF 4837 B212=C2=A0 3ACC 54AD E006 9879 F231 =20 ------=_Part_5393218_1420843073.1681821667049 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

After followi= ng the instructions provided I get the following:
=3D=3D>>> All >> py39-pygments-2.14.= 0 (1/9)

=3D=3D=3D>  Building for py39-pygments-2.15.0
/us= r/local/bin/python3.9: No module named build
*** Error code 1

Sto= p.
make: stopped in /usr/ports/textproc/py-pygments

=3D=3D=3D>= >> make build failed for textproc/py-pygments@py39
=3D=3D=3D>&g= t;> Aborting update

=3D=3D=3D>>> Update for textproc/py-= pygments@py39 failed
=3D=3D=3D>>> Aborting update



=20
=20
On Tuesday, April 18, 2023 at 09:57:31=E2=80=AFAM UTC, = Felix Palmen <zirias@freebsd.org> wrote:


* Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> [20230418 11:41]:
> What is the culprit?

Most lik= ely some leftovers from older package versions (so it's only a
problem when building in the live system. A clean jail won't have the=
offending files).

&= gt;                    &n= bsp; Removing blindly all py- packages for all affected
&= gt; hosts with dependent software is not the best solution.

Depending on your situation, it might or might not be= faster than trying
to identify the exact offending files= and just removing them ;)

--
Felix Palmen <= ;
zirias@FreeBSD.org>    {private}&n= bsp; felix@palmen-it.de
-- ports co= mmitter (mentee) --            {web}  http://palme= n-it.de
{pgp public key}  http://palmen-it.de/pub= .txt
{pgp fingerprint} 6936 13D5 5BBF 4837 B212 = ; 3ACC 54AD E006 9879 F231
------=_Part_5393218_1420843073.1681821667049-- From nobody Tue Apr 18 12:53:59 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q13lF5YwBz453td for ; Tue, 18 Apr 2023 12:54:01 +0000 (UTC) (envelope-from mjguzik@gmail.com) Received: from mail-oa1-x36.google.com (mail-oa1-x36.google.com [IPv6:2001:4860:4864:20::36]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q13lF2lVvz47Xl for ; Tue, 18 Apr 2023 12:54:01 +0000 (UTC) (envelope-from mjguzik@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-oa1-x36.google.com with SMTP id 586e51a60fabf-187b70ab997so3392095fac.0 for ; Tue, 18 Apr 2023 05:54:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1681822440; x=1684414440; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=AeMi6+P59eKpp+j+ZzR21smbCDf41eh7u3nGgYCqpIk=; b=VYP6ugZ62iXWMz5/tMu5MCBZt9YWkHe95PXKIwDLuWDzVJF1QP8J/vvVYkOLhewDck BM7ikDAWnvqbVQMViIwYzf/wZvYe+ozvqI7UDLyUO2/V5Rs7WHyzNtRGI6PRbYclkPN6 NPdvRFrNRMh6octdYSs6CMGGtGJpuIA2j69pexPdTRKaIAdJ8LJr/8R4sRGwNGwSf15n Y64JENZQeajPEFf6V610yMsgqPJelV1Qag7Z8uuFxKHRn2CDm4Cpi/fJWAGQsNQzs5YJ 4aHcQ1sFQwx9HgAxkW5YUyFFAZ2+O+YjWvv+KN0Nme089GA28LK5KxJ+GTywQU4ffd5M TTKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681822440; x=1684414440; h=cc:to:subject:message-id:date:from:references:in-reply-to :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=AeMi6+P59eKpp+j+ZzR21smbCDf41eh7u3nGgYCqpIk=; b=gBPIjiLxFakW8xEZdBpDRzm1imEE7wYrFlQb+zatt3vaXgsjEZYmPu1UkyiRGVtSLi z5I07wBiwnVqMLAAYunQlnU7ySTlmavzzBX1jdZ+LrNJKAOUn89vG1si+PzuKAK3aCcz eAhzY1MFSpYAagj33MpeXeReaZWckLQjdY0LG/pCAvx1hggcMX5TnkdfazG0IJ9Fuiw/ R+cihVh9fPKPENZpz66U14VGVjXU/pjXunlCcD4EbJ7pXvSWAX28dDbtBSEm6SI6E9JQ FVafQNQOny6vVsqVf+BJKEwm0j3dyinT5O0BLqURgm4RNbd4xRXr8cs1H7KiI5IxVxrc dNXg== X-Gm-Message-State: AAQBX9dm3ken7jYRNZMU0I7d7YtPBwA6HxS2NWMEJCGdU5E4Z/B62XOL dImyF/vHDhAHBtShybxuEITn+FkY+4lJWLJe6qM= X-Google-Smtp-Source: AKy350bSpTFf+ViUiCnBkVTqyijA87KjVnsxw/m28Tp49B/V6ZNt4NjR+ZYTngIVBNKJKzJO+PzkGf40AAaJj/et7P4= X-Received: by 2002:a05:6870:82a3:b0:187:7f29:c1 with SMTP id q35-20020a05687082a300b001877f2900c1mr934184oae.0.1681822440331; Tue, 18 Apr 2023 05:54:00 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Received: by 2002:a8a:11cf:0:b0:49c:b071:b1e3 with HTTP; Tue, 18 Apr 2023 05:53:59 -0700 (PDT) In-Reply-To: <214790055.5338048.1681805577606@mail.yahoo.com> References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> From: Mateusz Guzik Date: Tue, 18 Apr 2023 14:53:59 +0200 Message-ID: Subject: Re: Problem compiling py-* ports To: Filippo Moretti Cc: FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Rspamd-Queue-Id: 4Q13lF2lVvz47Xl X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2001:4860:4864::/48, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On 4/18/23, Filippo Moretti wrote: > Good morning, I run this versione of Frrebsd and al > py-* ports fail with the following message.sincerelyFilippo > > FreeBSD STING 14.0-CURRENT FreeBSD 14.0-CURRENT #6 > main-n261981-63b113af5706: Tue Apr 4 16:57:47 CEST 2023 > filippo@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64 > > you are on a zfs commit with known data corruption (in fact, 2) bare minimum you need to update to fresh main and reinstall all python stuff > > > > return _bootstrap._gcd_import(name[level:], package, > level) > File "", line 1030, in > _gcd_import > > File "", line 1007, in > _find_and_load > > File "", line 972, in > _find_and_load_unlocked > > File "", line 228, in > _call_with_frames_removed > File "", line 1030, in _gcd_import > File "", line 1007, in _find_and_load > File "", line 986, in > _find_and_load_unlocked > File "", line 680, in _load_unlocked > File "", line 850, in exec_module > File "", line 228, in > _call_with_frames_removed > File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line > 18, in > from setuptools.dist import Distribution > File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 34, > in > from ._importlib import metadata > File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.py", > line 39, in > disable_importlib_metadata_finder(metadata) > File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.py", > line 28, in disable_importlib_metadata_finder > to_remove = [ > File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.py", > line 31, in > if isinstance(ob, importlib_metadata.MetadataPathFinder) > AttributeError: module 'importlib_metadata' has no attribute > 'MetadataPathFinder' > > ERROR Backend subprocess exited when trying to invoke > get_requires_for_build_wheel > *** Error code 1 > > Stop. > make: stopped in /usr/ports/textproc/py-pygments > > ===>>> make build failed for textproc/py-pygments@py39 > ===>>> Aborting update > > ===>>> Update for textproc/py-pygments@py39 failed > ===>>> Aborting update > > > ===>>> You can restart from the point of failure with this command line: > > > > -- Mateusz Guzik From nobody Tue Apr 18 16:51:44 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q191d0V6zz45M4d for ; Tue, 18 Apr 2023 16:51:49 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic315-20.consmr.mail.ne1.yahoo.com (sonic315-20.consmr.mail.ne1.yahoo.com [66.163.190.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q191c57vjz4FjQ for ; Tue, 18 Apr 2023 16:51:48 +0000 (UTC) (envelope-from filippomore@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681836706; bh=LQ9SnyxzjkeX8hELQwmKlDJ51YUGcqIod2sIcQ7f31c=; h=Date:From:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=AKqME5ri7NDbVEZdY3PJ9F8jaqNT1pi0hK8q32DBUp3YzJGuYigyYz9vzYfhvJn5/EzWaYw34Uw6AUaz7FcJbVow+yi/uSptKoJpv/R0WASlKxgD6/62AN9wtpqdGSVpXni4SoE9E0y9TLN7AmFrXRgLub5T4KoRg+N/hx3KvsXVdnSuVZoJPWf61KCq2Wp8fbQ7NB2FUxRUePA1/1v9dpItTuhl+7U7fvn1r0n0n3emJz/uYA5BdIQa7ggAOxnxWtrmOpXC65mtfpJxgsx4JYInTZDHcetBB1skBPOgURtEgDm6uHR2209kE3rmHZDIicprea9HIZ/DAPV3MBOAdQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681836706; bh=t/Pd+gMfFcgaR3UDcMKnHvMpbCN0W1Nt7fwgAenjCSR=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=fGTz6L8AxSo8ql7mlXMP2IsLwbVNJG0hDXcfsqnFinuR1uhbQ+L0p4qny3pGkKSa5yU0e27kGaVdlZGuUlcMC3W6ugWfOsrDnK4Bd7FTjwOh6I8NLpHKf3tOa/8O1/lc7v/zog8n5VvhmU6SKQ5yHjOji0SnaMzfrXTI+eIKA2T2zkaKQOOr33CqwIR8OSn28c1BY0GOdS93Dr9aI9mbqpn0auXgH4bl5+3eT/kkVqzlKJkLbS0vcfqc6N2CXAx+LSrQauLhzVFsBlkfC1AvdwHHLOY19YhM79FfhXUyLM909kTpPVjG1Wu5r58dSW2OEmjR4vFpQoHhK3NZ1GODiA== X-YMail-OSG: 29Ql5M8VM1m6NgP6FLio922In_fIZQXxjJSp5Gat8fyaOXtNPQA9P7mCkAv1jTg L3.K.UIE_rU8Tm3vEJcVHdQP.2teqBFbSlluZmgP1vi2ZvV06n6MUFlHf3hid_PsuJOpkjKJRmu7 1cuU2whFqkBUk9RvTQyVCSj8IqcvGytdNYVumsWtBSKmu93Q7J9MlYa5n8zOLKraWBHdhUTN5te5 M4WRH1v4If2vqRMmxvTl40Y95AHBaUHQEG5QK4Z0j0OLZw4PvgMaioeVfD03Yz3Kwubzykz6XbNV TX9rf90Sti8rkm5zxePOiBkyC2mnsSBL_uCv9_Yh3ZnF4zgHY5erh44onlWSg9dr4T1P3sF.at.Y RvNdgap47syuh29bWhF.TKCxcpLiTWsBGf75j7tzNMGFJoM39IPnFifgVBBrBIJBOWdG3NwthxiP WpIwriE2dgx_0ax2Px2Zb2vZ5ONxkub_yAf_xKIXkrEu.pM6FS4I2qNhsQ8BzLF7UIeiJlrramyb U4m684_7T.x1_TDY.0CgaVJ9.xn8H6__tc3jS04dMOqDInUfEVO8Cy0a2lGqT6cjOy3dx1JZOVub 9t8kkUvRd4Z0X3ypysWZpy4vKAAB0JwQJNrXUIk_dVYf2Ytm6o.xEb6z5Xtb_ei2q4Ijsv36a2_N CN2mVnHxDKRFJkoCiAU1mzZHUKgZjuA1W81a0K.rvMWFSSJOrBA9lSBF7lR91H2wrR4aXYwHvyLi ca0qe5EfjnKwsuMRky6wWTWvfVU2T1PLc4l3vCRllyNAc01GV2Yjrx.t6uLMy3Edr2xYoYEz0FNl 4bE_SGkXlqYeADYkwvOPAdZRka483pLxPEIDEm3LAGtyvtNUuzPkJKYwdOfeSuJoQ6z4PZAHQIGj 3jzI.Q.0tHRLszbC4yFNLdfkXjuVtSqNSf7kFFfzvW_CGEeBXuWb97TnHbZnZiQNR.ciPRMpMN9u ZuNSK8iQYrYh8Fct9RYoa5qfzcnSmjWRWvmACQdwUAO95.cT6BNgY31u.QByS8ngk6OwCRceqKa2 J6BOQdSq7POUpfHWoC_FDxwN1D4plGob0EA7.QkmNHmxC0g3hqT6LYPI0Ai5PQFv7IVmdEKXlie5 dgLWeqGnhAlvnglgI4iFuAIG7rmI0rW2em2wc5e0pe_SgT6UAINFWQMGwn1OewNKTG9gSrWTeay8 LLGJFlbI.MOh_TcSKpt.MStb0lb0Jhg0q_zND6bi916CGEyUibSAg_ubtt1Gia9RtWQFD82kD18K FsOr7HAiqI8lvebzjsIujN1WuqmKIgf1gx9raUz8PlE5vNDMWdamiHHBKdT22x3pf0Y2Q36zTiyM 8ub0QANHhdI4fzrb6JPfG_ix5sZ5NrTKBq5xbg4sS56I9jBsNqobs0idiCpNndfwuBlw78hzyjaM i8Z0BQ4DSeIEkVgFu3XbLqvYs4qgcfKKD59VbpDKRvnwTHtLAYWHs.EN8N_0VyeukUTgar35v75z 3M11kZ.hYrYetz5T8vSBzq5fRePF1e32w242oiegD1NIFs4hUcbr2D3FLtZ6kj.0usRvwE10Scqy 9UBsOX5MMvAGH8BZnrMxYnLY9qo0vQPwIqZvPSeYZR.DsNQhltX1A8xLzxYjn3LQtDF8XXPmF15C 8hSEOOKw3blMjp.pcnflbD74O.WgoDT5XS.QLIfl6P.aNGfKt3d1.mdWxtv5l1aOORaORe07sWX4 lJ9fjE3Oto5YLH3_OQ0NQf7PmBWjq1ntRpogvR.NQpuVeiyxoUH9rb8LaPg1AokebZ.oGloSZkfi mFKoHa.YMMPFEnCLcuGKmGc4oYmLhiFir7HcSIXwnv08eAwH.2BXxEvH42Js2oKLnMal6uhi664v Veth0pTLwsgp1Ukf3jVAi_2fRGveGp1_3AufEFuR8mAqAHVId9bmnmEibHup_lQJ0KVmHT1UdQw. wdYA4oRHpgln.zAUuE0yJGnn_VWATsQ0bh56SwlZp.kUSvh3fRApLgnvL_4LofwP2rYnXSuO4JoJ Iipo0bZFhDIchWmwKTVbpS.wQF37sVHrrDNvWpLqA.tyXeE4O2_DNeWrz_.XWf7ipifWfhS7EAvT 1Y42GxQK6l0nhfMzmB8z_ppW6wkD_I85kRJGEBnytQ.byThpX0EZtbewSY5JFbm32uKcTlMU5g6i PQPwFMxHWSfd1Um1XRDeBpBq5gmE9Qt5PJ_V2SkA4wR96USSyJR4rfdbZpaVoJsgt4QnGL.5BY3G BJpj. X-Sonic-MF: X-Sonic-ID: 3bd941a4-e490-4aba-b15e-544a1a5736e7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic315.consmr.mail.ne1.yahoo.com with HTTP; Tue, 18 Apr 2023 16:51:46 +0000 Date: Tue, 18 Apr 2023 16:51:44 +0000 (UTC) From: Filippo Moretti To: Mateusz Guzik Cc: FreeBSD Current Message-ID: <93857963.5550998.1681836704930@mail.yahoo.com> In-Reply-To: References: <214790055.5338048.1681805577606.ref@mail.yahoo.com> <214790055.5338048.1681805577606@mail.yahoo.com> Subject: Re: Problem compiling py-* ports List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_5550997_817047533.1681836704927" X-Mailer: WebService/1.1.21365 YMailNorrin X-Rspamd-Queue-Id: 4Q191c57vjz4FjQ X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N ------=_Part_5550997_817047533.1681836704927 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable After completing installworld and rebuilding python39 I do get the same er= ror as reported previouslyFilippo On Tuesday, April 18, 2023 at 12:54:33=E2=80=AFPM UTC, Mateusz Guzik wrote: =20 =20 On 4/18/23, Filippo Moretti wrote: > Good morning,=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2= =A0 =C2=A0 =C2=A0 I run this versione of Frrebsd and al > py-* ports fail with the following message.sincerelyFilippo > > FreeBSD STING 14.0-CURRENT FreeBSD 14.0-CURRENT #6 > main-n261981-63b113af5706: Tue Apr=C2=A0 4 16:57:47 CEST 2023 > filippo@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64 > > you are on a zfs commit with known data corruption (in fact, 2) bare minimum you need to update to fresh main and reinstall all python stuf= f > > > >=C2=A0 =C2=A0 return _bootstrap._gcd_import(name[level:], package, > level) >=C2=A0 =C2=A0 File "", line 1030, in > _gcd_import > >=C2=A0 File "", line 1007, in > _find_and_load > >=C2=A0 File "", line 972, in > _find_and_load_unlocked > >=C2=A0 File "", line 228, in > _call_with_frames_removed >=C2=A0 File "", line 1030, in _gcd_import >=C2=A0 File "", line 1007, in _find_and_load >=C2=A0 File "", line 986, in > _find_and_load_unlocked >=C2=A0 File "", line 680, in _load_unlocked >=C2=A0 File "", line 850, in exec_mo= dule >=C2=A0 File "", line 228, in > _call_with_frames_removed >=C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py= ", line > 18, in >=C2=A0 =C2=A0 from setuptools.dist import Distribution >=C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", l= ine 34, > in >=C2=A0 =C2=A0 from ._importlib import metadata >=C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.= py", > line 39, in >=C2=A0 =C2=A0 disable_importlib_metadata_finder(metadata) >=C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.= py", > line 28, in disable_importlib_metadata_finder >=C2=A0 =C2=A0 to_remove =3D [ >=C2=A0 File "/usr/local/lib/python3.9/site-packages/setuptools/_importlib.= py", > line 31, in >=C2=A0 =C2=A0 if isinstance(ob, importlib_metadata.MetadataPathFinder) > AttributeError: module 'importlib_metadata' has no attribute > 'MetadataPathFinder' > > ERROR Backend subprocess exited when trying to invoke > get_requires_for_build_wheel > *** Error code 1 > > Stop. > make: stopped in /usr/ports/textproc/py-pygments > > =3D=3D=3D>>> make build failed for textproc/py-pygments@py39 > =3D=3D=3D>>> Aborting update > > =3D=3D=3D>>> Update for textproc/py-pygments@py39 failed > =3D=3D=3D>>> Aborting update > > > =3D=3D=3D>>> You can restart from the point of failure with this command = line: > > > > --=20 Mateusz Guzik =20 ------=_Part_5550997_817047533.1681836704927 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
After completing installworl= d and rebuilding python39 I do get the same error as reported previously
Filippo

=20
=20
On Tuesday, April 18, 2023 at 12:54:33=E2=80=AFPM UTC, = Mateusz Guzik <mjguzik@gmail.com> wrote:


On 4/18/23, Filippo Moretti <filippomore@yahoo.com> wrote:
> Good = morning,                  &nbs= p;   I run this versione of Frrebsd and al
= > py-* ports fail with the following message.sincerelyFilippo
<= div dir=3D"ltr">>
> FreeBSD STING 14.0-CURR= ENT FreeBSD 14.0-CURRENT #6
> main-n261981-63b= 113af5706: Tue Apr  4 16:57:47 CEST 2023
>= ; filippo@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64
>
>
<= br>
you are on a zfs commit with known data corruptio= n (in fact, 2)

bare mi= nimum you need to update to fresh main and reinstall all python stuff

>
>
>
> = ;   return _bootstrap._gcd_import(name[level:], package,
> level)
>    File "= <frozen importlib._bootstrap>", line 1030, in
> _gcd_import
>
>  File "<frozen importlib._bootstrap>", line 1007, in
=
> _find_and_load
>
>  File "<frozen importlib._bootstrap&= gt;", line 972, in
> _find_and_load_unlocked
>
>  File "&= lt;frozen importlib._bootstrap>", line 228, in
> _call_with_frames_removed
>  File = "<frozen importlib._bootstrap>", line 1030, in _gcd_import
<= div dir=3D"ltr">>  File "<frozen importlib._bootstrap>", lin= e 1007, in _find_and_load
>  File "<f= rozen importlib._bootstrap>", line 986, in
>= ; _find_and_load_unlocked
>  File "<f= rozen importlib._bootstrap>", line 680, in _load_unlocked
>  File "<frozen importlib._bootstrap_external>"= , line 850, in exec_module
>  File "<= frozen importlib._bootstrap>", line 228, in
&g= t; _call_with_frames_removed
>  File "/u= sr/local/lib/python3.9/site-packages/setuptools/__init__.py", line
> 18, in <module>
>=     from setuptools.dist import Distribution
>  File "/usr/local/lib/python3.9/site-packages/setuptool= s/dist.py", line 34,
> in <module>
>    from ._importlib import metadata
>  File "/usr/local/lib/python3.9/site-pac= kages/setuptools/_importlib.py",
> line 39, in= <module>
>    disable_import= lib_metadata_finder(metadata)
>  File "/= usr/local/lib/python3.9/site-packages/setuptools/_importlib.py",
<= div dir=3D"ltr">> line 28, in disable_importlib_metadata_finder
>    to_remove =3D [
>  File "/usr/local/lib/python3.9/site-packages/setuptools/_im= portlib.py",
> line 31, in <listcomp>
>    if isinstance(ob, importlib_meta= data.MetadataPathFinder)
> AttributeError: mod= ule 'importlib_metadata' has no attribute
> 'M= etadataPathFinder'
>
> ERROR Backend subprocess exited when trying to invoke
> get_requires_for_build_wheel
>= *** Error code 1
>
= > Stop.
> make: stopped in /usr/ports/textp= roc/py-pygments
>
&g= t; =3D=3D=3D>>> make build failed for textproc/py-pygments@py39
> =3D=3D=3D>>> Aborting update
>
> =3D=3D=3D>>>= Update for textproc/py-pygments@py39 failed
>= =3D=3D=3D>>> Aborting update
>
>
> =3D=3D=3D>>&= gt; You can restart from the point of failure with this command line:
>
>
>
>

=

--
Mateusz Guzik <mjguzik gmail.com>

=
------=_Part_5550997_817047533.1681836704927-- From nobody Tue Apr 18 16:59:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q19BG0pHRz45MLP for ; Tue, 18 Apr 2023 16:59:18 +0000 (UTC) (envelope-from fbl@aoek.com) Received: from mail.yourbox.net (mail.yourbox.net [IPv6:2001:41d0:1:767d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.yourbox.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q19BF5HyQz3Ddd; Tue, 18 Apr 2023 16:59:17 +0000 (UTC) (envelope-from fbl@aoek.com) Authentication-Results: mx1.freebsd.org; none Received: from mail.yourbox.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail.yourbox.net (8.17.1/8.17.1) with ESMTP id 33IGx8av084746; Tue, 18 Apr 2023 18:59:08 +0200 (CEST) (envelope-from fbl@aoek.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=aoek.com; s=mailbox; t=1681837148; bh=JsfkixIelPeb+g8s7GG+at0QAnkbuW26OmTXLSmZbfs=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=QgtHDbWn52EYcfZ7gf2cd0U/7+tqWGklu2Pd6e6VjXq+BadG+qC2BN4+lyVWdHb4d Se1AA3rGDaKtSfR/Whr50JJZTpdVELfBKKY3vUrhLyrSsJcwP8bIjoI17jwZrwMYIr sH3DWvJIAGjvyIPgcGxO/s9iQU3rnL/L7QLmFSckflhvz8ClnCjMVEGrpiVXy74lK/ mXT4SSLwn/JPKVM6Q5ytZalu17sWGlk1Jfho7w5JnLuTPtGvg60J9z7OU3ysS+Sozc gZPUYwww41JjFnTzCcMEO6d2GLwD0VazXdPvLaJwi/0Hg0bgqEl6vOej8O3r54ram+ 6SVzGmSFl31GA== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 18 Apr 2023 18:59:03 +0200 From: =?UTF-8?Q?Jos=C3=A9_P=C3=A9rez?= To: Pawel Jakub Dawidek Cc: freebsd-current@freebsd.org Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 In-Reply-To: References: <20230413071032.18BFF31F@slippy.cwsent.com> <20230413135635.6B62F354@slippy.cwsent.com> <319a267e-3f76-3647-954a-02178c260cea@dawidek.net> <441db213-2abb-b37e-e5b3-481ed3e00f96@dawidek.net> <5ce72375-90db-6d30-9f3b-a741c320b1bf@freebsd.org> <99382FF7-765C-455F-A082-C47DB4D5E2C1@yahoo.com> <32cad878-726c-4562-0971-20d5049c28ad@freebsd.org> <20230415115452.08911bb7@thor.intern.walstatt.dynvpn.de> <20230415143625.99388387@slippy.cwsent.com> <20230415175218.777d0a97@thor.intern.walstatt.dynvpn.de> <6792aded-6e2e-a118-259d-0df0f80c361c@smeets.xyz> <80ea8a67-9b64-c723-6d97-21cfa127ae43@dawidek.net> <01430095-33a3-a949-3772-2ec90b4c3fe6@dawidek.net> <0164e42a-e7cd-a1e8-295c-21f414edf67b@dawidek.net> Message-ID: <4ab79579555b34317e9210d5e9f52832@mail.yourbox.net> X-Sender: fbl@aoek.com User-Agent: Roundcube Webmail/1.2.0 X-Rspamd-Queue-Id: 4Q19BF5HyQz3Ddd X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16276, ipnet:2001:41d0::/32, country:FR] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N El 2023-04-17 21:59, Pawel Jakub Dawidek escribió: > José, > > I can only speak of block cloning in details, but I'll try to address > everything. > > The easiest way to avoid block_cloning-related corruption on the > kernel after the last OpenZFS merge, but before e0bb199925 is to set > the compress property to 'off' and the sync property to something > other than 'disabled'. This will avoid the block_cloning-related > corruption and zil_replaying() panic. > > As for the other corruption, unfortunately I don't know the details, > but my understanding is that it is happening under higher load. Not > sure I'd trust a kernel built on a machine with this bug present. What > I would do is to compile the kernel as of 068913e4ba somewhere else, > boot the problematic machine in single-user mode and install the newly > built kernel. > > As far as I can tell, contrary to some initial reports, none of the > problems introduced by the recent OpenZFS merge corrupt the pool > metadata, only file's data. You can locate the files modified with the > bogus kernel using find(1) with a proper modification time, but you > have to decide what to do with them (either throw them away, restore > them from backup or inspect them). Sharing my experience on how to get out of the worst case scenario with a building machine that is affected by the bug. CAVEAT: this is my experience, take it at your own risk. It worked for me, there is no guarantee that it will work for your. You may create corrupted files and make your system harder to recover or definitely brick it. Don't blame me, you have been warned. YMMV. Boot in single user mode and check if your pool has block cloning in use: # zpool get feature@block_cloning zroot NAME PROPERTY VALUE SOURCE zroot feature@block_cloning active local In this case it does because the value is "active". If it's "enabled" you do not need to do anything. 1) When in single user mode set compression property to "off" on any zfs active dataset that has compression other than "off" and the sync property to something other than "disabled". 2) Boot multiuser and update your current sources, e.g. git update --rebase 3) Build and install a new kernel without too much pressure (e.g. with -j 1): make -j 1 kernel 4) Reboot with the new kernel 5) Now you have to reinstall the kernel with make installkernel This is because the new kernel files were written by the old kernel and need to be removed. 6) Find out when the pool was upgraded (I used command history) and create a file with that date, in my case: touch -t 2304161957 /tmp/from 7) Find out when you booted the new kernel (I used fgrep Copyright /var/log/messages | tail -n 1) and create a file with that date, in my case: touch -t 2304172142 /tmp/to 8) Find the files/firs created between the two dates: find / -newerBm /tmp/from -and -not -newerBm /tmp/to > /tmp/filelist.txt 9) Inspect /tmp/filelist.txt and save any important items. If the important files are not corrupted you can do: cp important_file new; mv new important_file NOTA BENE: "touch important_file" would not work, you do need to re-create the file. 10) Delete the remaining files/dirs in /tmp/filelist.txt. If you did 5) you will remove /boot/kernel.old files, but not /boot/kernel files. 11) Restore your compression and sync properties where appropiate. BR, -- José Pérez From nobody Tue Apr 18 19:37:30 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1Dj864zqz45ZBD for ; Tue, 18 Apr 2023 19:37:48 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic311-24.consmr.mail.gq1.yahoo.com (sonic311-24.consmr.mail.gq1.yahoo.com [98.137.65.205]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1Dj71kRQz3Jg0 for ; Tue, 18 Apr 2023 19:37:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=NH4v4ZWf; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.65.205 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681846665; bh=HJ7yIqeyX5QICTOi3zRCcXtQr2ktDEiMkP7LgjKK1Bo=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=NH4v4ZWfD3W0wCDiUwIow6vy8X4XZWpeUfC06c/Lq8x2+gnlfkQq17qleN/zTffhjJKRPB+TyPXjaTNrT4C/4T9v160DX/EGO9UuzBQN/IPdhhYVkeewNILBsD19Yj4+iY1vFhYUPwp1HMXY2CDKjHmktITdESRPDHrItPUIlYaBACtIuIrNYePeKxLQMaZ2QRJ0g5kIWA3XXL8t2A1YiR+gWIFK5FGk2cBuIlOQiDobRXRmlVna9zUjfG5lqhPznSkCzpet2fJfxuL53q3d0jEm1lM+3hbr/15HhG/RW114K5ZQHPKS0kn3HAa9tgjDAZ2RiI+NuJfu/0Nh/p0wkg== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681846665; bh=fwjDajeFbXwFi6nvBDj8RcY7c2DHi0hTdAbRa4RI5IH=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=L3o7HtmNd6oZHjrrVNNqkApC3Kr1GI2Y/wcDO6YCkns/yeP+IyxAPln+w0qLxca0wyYxfJhMplxWJffEVf0lUwxE+VI8O2IheTcLhBZzKgQJyrjvaVBWX48fFeFLJ7FM6pN9e+FRzGP3LoJRSU5C3ZhzfghAUdEkT0bm5XWim+O/vqgpEYnR0attTQylIwzxaSRyoGd39XRvw3NCtRyykQe/DURvVUZ9fArY2031cdV71VbJfuMIMf/FOx1rddsLkIUhMFkvMErkFiC1OqBDIC4/a/E/qqcCLD7gU6utT+jW/ICpbqiGvjJkqfdue0wC/cPVLTFpIEhkXxC00gmv9Q== X-YMail-OSG: Gw2U1bIVM1lZQ3A_cQFQQhVsgzH54d2WNdDTAs.UXE8EYGY9H1vvYeHWsvMUUHT 08Lgy9SQXKzlj7VOxfePHkC6pcluVXIEjvB8d6xSrk8.YC_7.Md3AtxaL3tjjcQPQjkHlsrUWG2D o3m0vk94Y7rq4FEuSn.hkd0EB04PDpS2U7c4RCgY1_FepezLKAyjwefBIUIPV3l1SvrhDDD7Dopp hQXUu_Cets._aFDrv5gDm6kFcxiRrBI94Hydm5dXzIUtbw2ld8IcjGt6CVqyo0jbqW2cVwPwXuAo cqoy8H182OvQqVnJF8_PAtl28gLauwyEatG4VOrDSMfFi5P0Ah2p7gBrqPqrfLe8Agw_x.NxxXxj agcg0Sq8JKRWbs_jVoJ6FC84jV6gO9ygkij7T5N_vYeoWkurDVFbE_6UXpbfw6KBRllXzF2yWiix oygaYoBD.sk_rEXnUWAJFhXBOqCGMxW9CiyqyC0LNH71ijuCVfUkj0XNYUBhWAx7xGunAuAUlQ6b K.qgxric4m1dziiTEK6LLn4uakAu_ATM8LjMlj_wXPvXrni6ZSBuMnA118Lw57G6p3D7lOExQy8m WjgV.eC5thDkBzrgGuK78mOCt1qdxYINcHxI7RVKoy7W_IaE0sxAhLN6OnpygfF96II0diDO3zPp 0X_49ihr5OLBRN0VEMjoUxr7vW1kSkxCDxvWweN2c_AxhxrB74YLgfLb_n_KbSr27w30JY_YTReC eXihKc3zsQAO9lsxWNJ2furkvCDoNxpZ6fCEW_FmO9p_dGBy7P37FOmdNumJt86CUY61jtNaNsgZ WbOVMYsXN8.85PRnzNoTfYWnzCx6cq2n.3ehYYvP7a9_M7p1GnsP0E926KjWz_H2i5AhEIqfMCUl 4n.Qwam6Lal98djxijS9If9Ww0VpHS6Y6uZFEhKHu7ZCH_Pd071P1XK8d4vXZOm8l.dWSxqIGeBj .xIZMOlipH3pqLWEwyf65minf17JyTsqh5ufGc.tFkojHxrvlWDrpGjfz2mrh_ktZqayw8LXAaDS paCF3nv9nuYN6oXSpOs895bmhEgvrMw44NT7W0GCRYEWMB6i8EtMJSzMJhy92Ei3ihc2Wt9rASNk tu3r5Qs6gyqAxXMxz7BNzSovoKdd9tPgr_Rem3ccGa0FMMzS._EqLJJp2W_Rp9scRGNIwgT7adkL B.NUbFJ84n2m1TjDoUibMMuitEBpSYL_.GK0DKtzJ748vq_A8ChCgCYRUiL9R2EoXJM2m4bbCFXy .XMJwyM2mTpUV7Y.tS5M2kR_6nnvXc5P_49kUpDTMS4PB0dN30aLaClykGlVvdFyMm0WvfUne0PG sQ4flWre5Morg64p_Qp5BxdWmGlahl_4166ngqatTIAyifVakXbbEjtCmHyQto.OQyYGLO9vxeO8 kUQBVhpc8DGqKxgAcSWnfDIIjX.LqkD.PK0qr.JIC.Yl1b5SZs920IeDIHtAVZF_Eib8xmP.boJS ofiMlHorImfI1ouOk0Tfhoa.y24xzNf3cYvoEJ9ZkB0AGEqE2UXTGmZLd6tSSdklXzSI174RnzeW fJDbaUWtJr7LaU4DtrMHuAou26AyMqhwvUvBbVEpIf16.21PiGK2zksy8HUqK1DFzHR4.VtJ3bNM FbEZwIScjCoNBZbKKoCX3kFmzUbA6kIrSenbHBSyYxm9PJlkMLfv1u3amxhW5mQRqHOFRFaCOsqs 7iBYeLaXNt0D0wOL9g5LrdJ0aT6CeTQIa3tE2jndOFHsMN7Q2EmyAZYAsPyE3nlRPKPJkK9xjo6b k_bb82Vd9mcbWLJZYxoM0LwrwNfbpTgeQ6PSTET2BykJJhZZ.ezfsL0iFfngLv_.3maZcIcOcoNL W.TVL0OcRzCoQOzxisQclhYEGOrPljwXcwuoq03z4mYRLmFEdxCwN.pGkKI06UVewHUsO6wMIt4_ amzfkSmnBjuYpV_DZdkjinNmEKPpnVX9axh1um3gIf764QWwfBMQyUXySpywddQu2pd7vKUfyt3P RyAg7QGy_cjNHP3eUTPeHjlSfQSq7QqfqOqqkz065fy44cSsTglKFvKLyHwolXr15HTLs6hCJTiW O.qSq1JNZY7HShDo1UnjRlbX7hSpqwa1mkWV6AWlF3doCIYaW3OzbumOg69Wtk8eMAkyKx64_qO1 XtwQ5j8nziCKb.QL7YmBkB0lZDJr0fgkfV5uScpP4mzLdQmuMYRLIuIlWzC_nHCIqaaZM9J9Wd5H e_v7guyGuBPDyheebRanTXqfJI1p.B.qnSLf3O8UueAND3mOJUSHkUHHLPwZWfOsiTaSwSdxglBk - X-Sonic-MF: X-Sonic-ID: 84b69ef5-9fc8-491a-a4f0-c5f86270fe0c Received: from sonic.gate.mail.ne1.yahoo.com by sonic311.consmr.mail.gq1.yahoo.com with HTTP; Tue, 18 Apr 2023 19:37:45 +0000 Received: by hermes--production-gq1-546798879c-l2qgj (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID dc2424b58e19021d7f37588319f5e9ba; Tue, 18 Apr 2023 19:37:40 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 Message-Id: <963B0620-5342-4EC3-AA54-52DDD70D9E3D@yahoo.com> Date: Tue, 18 Apr 2023 12:37:30 -0700 To: =?utf-8?B?Sm9zw6kgUMOpcmV6?= , Current FreeBSD X-Mailer: Apple Mail (2.3731.400.51.1.1) References: <963B0620-5342-4EC3-AA54-52DDD70D9E3D.ref@yahoo.com> X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.65.205:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; MIME_TRACE(0.00)[0:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4Q1Dj71kRQz3Jg0 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Jos=C3=A9_P=C3=A9rez wrote on Date: Tue, 18 Apr 2023 16:59:03 UTC : > El 2023-04-17 21:59, Pawel Jakub Dawidek escribi=C3=B3: > > Jos=C3=A9, > >=20 > > I can only speak of block cloning in details, but I'll try to = address > > everything. > >=20 > > The easiest way to avoid block_cloning-related corruption on the > > kernel after the last OpenZFS merge, but before e0bb199925 is to set > > the compress property to 'off' and the sync property to something > > other than 'disabled'. This will avoid the block_cloning-related > > corruption and zil_replaying() panic. > >=20 > > As for the other corruption, unfortunately I don't know the details, > > but my understanding is that it is happening under higher load. Not > > sure I'd trust a kernel built on a machine with this bug present. = What > > I would do is to compile the kernel as of 068913e4ba somewhere else, > > boot the problematic machine in single-user mode and install the = newly > > built kernel. > >=20 > > As far as I can tell, contrary to some initial reports, none of the > > problems introduced by the recent OpenZFS merge corrupt the pool > > metadata, only file's data. You can locate the files modified with = the > > bogus kernel using find(1) with a proper modification time, but you > > have to decide what to do with them (either throw them away, restore > > them from backup or inspect them). >=20 > Sharing my experience on how to get out of the worst case scenario = with=20 > a building machine that is affected by the bug. >=20 > CAVEAT: this is my experience, take it at your own risk. It worked for=20= > me, there is no guarantee that it will work for your. You may create=20= > corrupted files and make your system harder to recover or definitely=20= > brick it. Don't blame me, you have been warned. YMMV. >=20 > Boot in single user mode and check if your pool has block cloning in=20= > use: > # zpool get feature@block_cloning zroot > NAME PROPERTY VALUE SOURCE > zroot feature@block_cloning active local >=20 > In this case it does because the value is "active". If it's "enabled"=20= > you do not need to do anything. Well, if block_cloning is disabled it would not become active. But, if it is enabled, it can automatically become active by creating a first entry in the involved Block Reference Table during any activity meets the criteria for such. If the FreeBSD vintage in place is one that corrupts zfs data for any reason, one would still want to progress to a vintage that does not corrupt zfs data, even if block_cloning is enabled but not active just before starting such an update sequence. So, in progressing past the vintage that corrupt zfs data, one could end up with block_cloning enabled in the process. At least, that is my understanding of the issue. May be only a subset of the "causes data corruption" range of vintages would have to worry about block_cloning becoming active during the effort to get past all the sources of corruptions. (If so, I've no clue what range that would be.) I expect that the "you do not need to do anything" for block_cloning being "enabled" instead of "active" may be too strong of a claim, depending on the specific starting-vintage inside the range with zfs data corruption problems. (=46rom what I've read, when the last Block Reference Table entry is removed for any reason, the matching block_cloning changes back from being indicated as active to being indicated as enabled.) > 1) When in single user mode set compression property to "off" on any = zfs=20 > active dataset that has compression other than "off" and the sync=20 > property to something other than "disabled". > 2) Boot multiuser and update your current sources, e.g. > git update --rebase > 3) Build and install a new kernel without too much pressure (e.g. with=20= > -j 1): > make -j 1 kernel > 4) Reboot with the new kernel > 5) Now you have to reinstall the kernel with > make installkernel > This is because the new kernel files were written by the old kernel=20 > and need to be removed. > 6) Find out when the pool was upgraded (I used command history) and=20 > create a file with that date, in my case: > touch -t 2304161957 /tmp/from > 7) Find out when you booted the new kernel (I used fgrep Copyright=20 > /var/log/messages | tail -n 1) and create a file with that date, in my=20= > case: > touch -t 2304172142 /tmp/to > 8) Find the files/firs created between the two dates: > find / -newerBm /tmp/from -and -not -newerBm /tmp/to >=20 > /tmp/filelist.txt > 9) Inspect /tmp/filelist.txt and save any important items. If the=20 > important files are not corrupted you can do: > cp important_file new; mv new important_file > NOTA BENE: "touch important_file" would not work, you do need to=20 > re-create the file. > 10) Delete the remaining files/dirs in /tmp/filelist.txt. If you did = 5)=20 > you will remove /boot/kernel.old files, but not /boot/kernel files. > 11) Restore your compression and sync properties where appropiate. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Apr 18 22:02:45 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1HwZ6y26z45l8t for ; Tue, 18 Apr 2023 22:02:54 +0000 (UTC) (envelope-from fbl@aoek.com) Received: from mail.yourbox.net (mail.yourbox.net [IPv6:2001:41d0:1:767d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "mail.yourbox.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1HwZ0jyZz3DBQ for ; Tue, 18 Apr 2023 22:02:54 +0000 (UTC) (envelope-from fbl@aoek.com) Authentication-Results: mx1.freebsd.org; none Received: from mail.yourbox.net (localhost [IPv6:0:0:0:0:0:0:0:1]) by mail.yourbox.net (8.17.1/8.17.1) with ESMTP id 33IM2oVa006351; Wed, 19 Apr 2023 00:02:50 +0200 (CEST) (envelope-from fbl@aoek.com) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=aoek.com; s=mailbox; t=1681855370; bh=WjwWZ6M07ljOOkOqLULXQ2XVMezkEQizb6QbEMBwICs=; h=Date:From:To:Cc:Subject:In-Reply-To:References; b=R+9ll0KjV1dodQ/ptkp8BGPeiFZbj2r4IHNa/0DdNaUzsSe0QbWNylDF1sKprqY5A NN+hde+7dXWqy2FrSD57AYC7i1YP0KRs+VioRdSDT1PfKO712FA6qF9lOz5e3wzlkt 04vmxhDToVbiSZbwn/UJGG1VmI5qI0l1YB4fl+CvFrKswSqxycL6AL4GkPJ+R9soMe xIwaW/xNb/RCCUvpsdxjYVdRd1kvhPRntRqN6OG3i4lcZJjQr4M41eSWCJegtz8o8z CGWUEcX40cqgtUw+7th79AgaDWyBKSZFNv+sjqYitPgOqELOsgg4UH5XGxmhOazyyr xJH+/UN3zUKUA== List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Wed, 19 Apr 2023 00:02:45 +0200 From: =?UTF-8?Q?Jos=C3=A9_P=C3=A9rez?= To: Mark Millard Cc: Current FreeBSD Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 In-Reply-To: <963B0620-5342-4EC3-AA54-52DDD70D9E3D@yahoo.com> References: <963B0620-5342-4EC3-AA54-52DDD70D9E3D.ref@yahoo.com> <963B0620-5342-4EC3-AA54-52DDD70D9E3D@yahoo.com> Message-ID: <8fecc2d8c2eb471ec70122a7d5fb249d@mail.yourbox.net> X-Sender: fbl@aoek.com User-Agent: Roundcube Webmail/1.2.0 X-Rspamd-Queue-Id: 4Q1HwZ0jyZz3DBQ X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16276, ipnet:2001:41d0::/32, country:FR] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N El 2023-04-18 21:37, Mark Millard escribió: >> In this case it does because the value is "active". If it's "enabled" >> you do not need to do anything. > > Well, if block_cloning is disabled it would not become active. [...] > So, in progressing past the vintage that corrupt zfs data, > one could end up with block_cloning enabled in the process. You still have to willingly issue the command zpool upgrade so you might not just end up with the feature enabled by running this or that kernel, that's why I suggested step 0: verify if you are the worst case scenario before you begin. >> Boot in single user mode and check if your pool has block cloning in >> use: >> # zpool get feature@block_cloning zroot >> NAME PROPERTY VALUE SOURCE >> zroot feature@block_cloning active local >> >> In this case it does because the value is "active". If it's "enabled" >> you do not need to do anything. If you did not upgrade the pool, the feature would just not be there and the pool is sane (*). unaffected_machine# zpool get feature@block_cloning zroot unaffected_machine# As said, if the feature has been enabled but no calls to copy_file_range() occurred, the pool is also sane. To summarize: no feature -> sane feature "enabled" -> sane feature "active" -> might not be sane BR, (*) as per this bug. -- José Pérez From nobody Tue Apr 18 22:44:58 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1JsQ1wSwz45nXy for ; Tue, 18 Apr 2023 22:45:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic301-22.consmr.mail.gq1.yahoo.com (sonic301-22.consmr.mail.gq1.yahoo.com [98.137.64.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1JsP1JpBz4PG0 for ; Tue, 18 Apr 2023 22:45:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="L/iZZka4"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681857911; bh=zb1CLX72W4BX7jXcqBrl90RouD4bA19KhOtfFiy8a7w=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=L/iZZka4KSXLHlAQd0Y+ONQuv9aVTcKE74pZFNumjJdK3EbKgo9kYcuE2jAL/oLLgn5VMsw1dZ705HNLTzxhD7ahsWhrKVpwjY6dh0MtSNFpO4XRs8CVMmW/E+Glnq/mOxIEhWUnOlr8//ycOY/73uuPQrrSeCahUTnWC88ORqw1wD2K3iUr3UPkgE41fOhPdtOfnFG1dzBdFPBD2lRMHYi39aRSChD6fGrmeRsXw5jVIWPS9rWBBTWx7uEIWGR6vqPAp6wt+k9X8WWRqa5avdz01ViyNFg+bW2EFf9QmX080gZQ7uW9WvOEMXlghYnfy5F1osdZvDiKDT82NHk7nA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681857911; bh=Xeb6I2R3aJ7qM6ocgPO6QL8A31r5G8lKs5pGfxlaDkX=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=VROwXCifTmb2Zh2XvByDc5eDn/fHl/ZI+7LDDGYg7GtoMxmxm8Bv9HNC8E+OiupW0StqUB/4WL1Q2yucOkDe//isyUv0tRRgcN/Ls5M2pVPSBFcrX7PIiovEzrWzpGkukRCVTDRDQBx+xcvZKWoRpPpgDZpNRgeq94UQzpjbkiWDjhQaWSRZx3aKFXMrhufcaPpyWc6aaeZhdC1Clq5ef/ZHa+/0db9O82dPtYfVYSXqwuPnziXXQE6A/Opit+TFj3D5VA3wTwvm+mZohbp4LNyIKE4FmyBrfgApLeqvUZvK89sI0y4Fn+Zv9H8Y0Orc+ggekuyLvPeI/qLeutvsbQ== X-YMail-OSG: AWPzeUMVM1lo1vi78NXOwpype3v_aQaoDgFoLiCSjXQF_glmOa_eEozn_kTkbqw O2aRf3SZrAzgUxWdbrgVWGkvDMGgwCNzlK1BHZcJ4I6tK8FDQMDvQDJSV.HIV2ku9VgRI9JSRT31 WTY3IoGd8Nqm3DHoGHRu3ur4xsAiz0HDUZpKkti0t6MUiDHoRA2tTBlrCShU0Vz9jM6eqmUpUMDZ CwNx6LKqqXComupzLkZl8wRWFAVW7Pzporkmi6KZzRJuSE1M1zI8XBa4NXdpgEbF5H6N37dPdP_0 7xBYf2L45Male_UwrYrCS6eog683oSub0gle1GrGnrmJTNVS.6a63ug.l4h3.pk51Nn7L3B3e7Mm 4iq_1UTAGldxXAXYMl0XQIo_J0UU8CL_en1LFq3xp8ooUwRGnWIohWRLYbg90t7fXAs2ke089XkS PArDWncViLNa8CCG0Qk65R9tfC9PvYt3eGFbqtrDZ2goMioFyvcUh2WXOLiW_mOHxt_4tjw30HyE OA7yOF1Xnp8RTNIc0XcZNJht6kGxadiMV5zxc7EDTEhmFBLCw.CY7_24waUqzf6NjCtJnTPXtsgT OTWsa2clzz8NTq_GLv4hjcEfKPJrjOm9khLlfwNTNforMPtdhLh6y08Ju.RtG0Ulc0mgpzmGCuSp lZ0LpdnCJFPW63V9bhEncPdQwVS8lHmykU6_QDwyO8U7jvZ3hzl8Vs_pRg9yWyvN5LisS5g7T7WG xmZkTgtvZrARKHoHKQf0m_gnvK2mG4kriddjwI_YFTUPUoIUPgKP.NgSobeyeICqkbiRDBWoxhL6 63ZC4e_D4ptDDHtNx8Kw3t7vXBnOmeW5dafugKiqxAY.auleQjJ2YRvPgzc91QWCaBsdR2BzDqaa Hw4M6aBzd7Ypw.ALx5aXpSPrK0l75A5_MXZbm7_QR8zduUO0pNGv96gZYIMdk3P_9NqXWkEsasWT qq8S9nOWld0okLKPbU84VZ0WwCFHJzUPEaXfvZvA87FTDw5.O4JS6JFEe9CKHNM7fuWKqJVWJW0. STxXX_sP5fEmivzlvHRMa3EBr7GeBKI3LjS.wIrzBq8rLo9A2mDHCneRQCgwO6S_CVTiDz0HVjeJ QKNOxMJm58A3x.p62FDHoyfxJlicFSvZYeOoXIt_mXDWRuclhNTgCkZu7j9IV_cTMXyzsSyySjcv SmfQR_d051TadMFA1JaHyUgB5w4_yqoHFRHXnQNYFkP2FoGJqxslXby8gZve86DlW8Ffkz3JEnE. OgxQYAs2Xqk8cUCN8rzQVvaKW5EqWn4QeYGaf49X8iBMLgAYdTkdYFkolNGsLWFezWthJSSi9Qbi _sAKFbQbgaZzYiCUOJYydYgbVxKbqJ6C4tzlRKRQuI2N8aAwyj6JTNjGrbgJc93SkTQh_WhxlFt7 OaJ_cjSKThwms_DKlwj_BRApDdHXth.tpr.LErnNHGc1r90X7rCiiJNVuiXBG4mKXcc7AsRvgUH7 yI.s8HYTF9rroX_h1gKeBQXOXFbOhY4U04dNhoy_muEnRnzpjiQHi.geangyI.VbMc2jbdzvQh7W Qi8gQNwk6.AIkMm7S_tAuo7wFOmd6WN2lylX1pyDEuNjAIlbQfrdnx7WjnU_.2eVbi3DFk8o0eg0 uEHg4IVmPLyiz8ib54j6fW5y942D4w23FxEIl0DGwUO.VQsVJWqCM84S8toG8HnPOrQ7NvlY5Qn0 9gar6nfrhv14HRDbFOlyxVz2Ft7f2givw.QGAb_kUCBcSfIkf4hzd1Pzzo_p.veUbfPuxBssy8e8 3F81yAO6OQqMAvM5SwgID.QcElDTKnuNWhTfmqBNGpvvsp9GmtpVk0bMiN04LLK1Sva52sZEQKIz PUVnYaKq0YQbjd9ZCvHGVul0EF3AlF15wkLars7..ppgyCBMa_9wNm8PbX_uCZ2Fr.gBylnmKliy Oxb8ZaCUid7KOQpNjxns5VeS705jdPunG1U9qLYNvC7oxnc8NPt.RTzHQmT1o_r.K2pQ0QduAy34 YfTP9g_zYjkoGcoEoeSYD.PTiCnYWCyeLUSM4cJ99.viAQIixTOffFmI1ODAaPCH8i_KaJntxI2d PomZyAi9ysZep7GzFe7L9rAvw6714FOlIoFDR4oSNn_sZml388o2114hDIGIQzAmSkhTwwxavYxJ Bs.sHW5N9qV5X4k0ILOrbhRh0eIF2evZKpbPZOUm3wH.VI0m0UpdWE2XbXIGxI51I3TDCONSao9f 7wvDqqxozyEQtTiix1p1Z_R6jVhNpkhpfbxt_kABcJwrPmYwmDdSnJS5F0REs1uIdE6YbSL9XUFS v9p0_iDoGyQ-- X-Sonic-MF: X-Sonic-ID: 0410807c-5217-49a2-9aa7-6cbb42df9b8a Received: from sonic.gate.mail.ne1.yahoo.com by sonic301.consmr.mail.gq1.yahoo.com with HTTP; Tue, 18 Apr 2023 22:45:11 +0000 Received: by hermes--production-bf1-5f9df5c5c4-84ds6 (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 50890fee8f614b27b29dded918684dd3; Tue, 18 Apr 2023 22:45:10 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: The import of openzfs vs. armv7: boot crashs Message-Id: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> Date: Tue, 18 Apr 2023 15:44:58 -0700 Cc: "mjg@freebsd.org" , "pjd@freebsd.org" , Kyle Evans To: Current FreeBSD , freebsd-arm X-Mailer: Apple Mail (2.3731.400.51.1.1) References: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F.ref@yahoo.com> X-Spamd-Result: default: False [-3.45 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.95)[-0.949]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.148:from]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4Q1JsP1JpBz4PG0 X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N h= ttps://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c7413a= ae9 does not seem to cover armv7, just aarch64. (FreeBSD disabled floating point for both armv7 and aarch64 but that is a different change than above.) I used: = FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img.= xz booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the "undefined floating point instruction" notice): # zpool import ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow = panic! Please consider adding 'options KSTACK_PAGES=3D4' to your kernel config panic: undefined floating point instruction in supervisor mode cpuid =3D 2 time =3D 1680784610 KDB: stack backtrace: db_trace_self() at db_trace_self pc =3D 0xc05eb154 lr =3D 0xc007a688 = (db_trace_self_wrapper+0x30) sp =3D 0xdd25c480 fp =3D 0xdd25c598 db_trace_self_wrapper() at db_trace_self_wrapper+0x30 pc =3D 0xc007a688 lr =3D 0xc02eb1b4 (vpanic+0x140) sp =3D 0xdd25c5a0 fp =3D 0xdd25c5c0 r4 =3D 0x00000100 r5 =3D 0x00000000 r6 =3D 0xc0736bfc r7 =3D 0xc0b1aea8 vpanic() at vpanic+0x140 pc =3D 0xc02eb1b4 lr =3D 0xc02eaf94 (doadump) sp =3D 0xdd25c5c8 fp =3D 0xdd25c5cc r4 =3D 0xc0b92210 r5 =3D 0x00000000 r6 =3D 0xc0610ca0 r7 =3D 0xf4210a0d r8 =3D 0xddf32e4c r9 =3D 0x00000013 r10 =3D 0xdd25c6c0 doadump() at doadump pc =3D 0xc02eaf94 lr =3D 0xc0610eb0 (vfp_new_thread) sp =3D 0xdd25c5d4 fp =3D 0xdd25c638 r4 =3D 0xdd25c6c0 r5 =3D 0xdd25c5cc r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4 vfp_new_thread() at vfp_new_thread pc =3D 0xc0610eb0 lr =3D 0xc060ff84 = (undefinedinstruction+0x178) sp =3D 0xdd25c640 fp =3D 0xdd25c6b8 undefinedinstruction() at undefinedinstruction+0x178 pc =3D 0xc060ff84 lr =3D 0xc05edaa8 (exception_exit) sp =3D 0xdd25c6c0 fp =3D 0xdd25c750 r4 =3D 0x20000013 r5 =3D 0xde45e000 r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 r8 =3D 0x00000000 r9 =3D 0x00000000 r10 =3D 0xdd25c8c0 exception_exit() at exception_exit pc =3D 0xc05edaa8 lr =3D 0xddf31f20 (K256) sp =3D 0xdd25c750 fp =3D 0xdd25c750 r0 =3D 0xdd25c890 r1 =3D 0xde45e000 r2 =3D 0xde45e400 r3 =3D 0xddf309fc r4 =3D 0x00000400 r5 =3D 0xde45e000 r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 r8 =3D 0x00000000 r9 =3D 0x00000000 r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0 zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c pc =3D 0xddf32e4c lr =3D 0xc0946e8c (pcpup) sp =3D 0xdd25c758 fp =3D 0xc0b0aeec r4 =3D 0xc0919610 r5 =3D 0xc0919630 r6 =3D 0xc0919618 r7 =3D 0x642ebce2 r8 =3D 0xc0b1b0ec r9 =3D 0xc0915e88 r10 =3D 0xc0b1b0dc Fatal kernel mode data abort: 'Translation Fault (L1)' on read trapframe: 0xdd25c330 FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3 r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555 r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0 r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308 panic: Fatal abort . . . (repeats over and over) . . . =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Apr 18 22:46:27 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1Jv36sMDz45nxQ for ; Tue, 18 Apr 2023 22:46:39 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1Jv36N6nz3CMy for ; Tue, 18 Apr 2023 22:46:39 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62b.google.com with SMTP id sz19so20043569ejc.2 for ; Tue, 18 Apr 2023 15:46:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1681857999; x=1684449999; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=ujarkVJtZON5/zCQzEjUOhn9K7Gppkzzs4kwx75CZxg=; b=z88fdFMC4WPjosZQyWcEhDC+VraGC8PUa2oFHLUL9QReSi4ZzAQCbA1e7Qfrpe1fUm DNxMTSqA/BiihT/EAcoPzJv1K4imYYOfv/Uww/fG8SgsRPbaaFfpBW6ELl5L2Uud7Z6u zT5m2pTGRAhzic6wN+2TnxyoTve0ylUPQvIx/avNGdNqIwkqw7ZW2hoolYwwe/DqiiLy ipbbvPwkiQVvoNBUvQ2ulInF+MznAaMuxYJ7Sc5aBL3HD+3aYnuisO7Yc9QGIFxW/iab Uez5ThnvAuVT0S8A+5isTsHu/lT6GWu9MWd3RTj6cK6yMTqBIMAERrLTBWRbKmtTmBHy upTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681857999; x=1684449999; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ujarkVJtZON5/zCQzEjUOhn9K7Gppkzzs4kwx75CZxg=; b=VJqdKK/LYMvRWFmuQCfruMWf2sOE1I1C9mph6UO0EQaaMLnPZMtldu3k1rGSPAteK3 AbVX5HYpN2Jom/4mFWuzvN/5wBkupsc+HdpJJFcdBRVuzJSOZGW3m7DB2ZpwFQCFD3/S 029ATVfsGh/2HrbSX2iJffYrADm8Xlk9Vb8LKYvYW4D9/iEz1/Z+Dx8uEF58HE6lB198 TFYsAjc7UkO3h+MECMf4oaWGNsmIcXIsb2U9PSrxqEq267SjeLQmS6sXsZOBe0Clrsit BBn6acFMwrs51Jix7fWSWGlPP5m+s6UmzGna097voOagK8wgLnkkhRo9FLJWU8362oWp eGfQ== X-Gm-Message-State: AAQBX9evZsz45Kuk/gptAl2BrPdd69sAMhlOI4NJO+Gmh3EaJemArZkj 9W5C/Xi+A6Sz9qIeHF0Qe96Hw+d0qXGgGyTYeYGhgA== X-Google-Smtp-Source: AKy350YJQh/eVsrs26iYNxeUR9WXZOmO40tOmZ396WaA/d1cUU9XngTwB1H3KlsU1RV1iLFuueshFv82JfqVMBHQ88Q= X-Received: by 2002:a17:906:cc48:b0:94e:bc04:c6f6 with SMTP id mm8-20020a170906cc4800b0094ebc04c6f6mr11809779ejb.9.1681857998634; Tue, 18 Apr 2023 15:46:38 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F.ref@yahoo.com> <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> In-Reply-To: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> From: Warner Losh Date: Tue, 18 Apr 2023 16:46:27 -0600 Message-ID: Subject: Re: The import of openzfs vs. armv7: boot crashs To: Mark Millard Cc: Current FreeBSD , freebsd-arm , Mateusz Guzik , Pawel Jakub Dawidek , Kyle Evans Content-Type: multipart/alternative; boundary="0000000000009253ac05f9a414a4" X-Rspamd-Queue-Id: 4Q1Jv36N6nz3CMy X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --0000000000009253ac05f9a414a4 Content-Type: text/plain; charset="UTF-8" Fun... I'm also fighting aarch64 issues... Warner On Tue, Apr 18, 2023, 4:45 PM Mark Millard wrote: > > https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c7413aae9 > > does not seem to cover armv7, just aarch64. (FreeBSD disabled > floating point for both armv7 and aarch64 but that is a > different change than above.) > > I used: > > > FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img.xz > > booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the > "undefined floating point instruction" notice): > > # zpool import > ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow panic! > Please consider adding 'options KSTACK_PAGES=4' to your kernel config > panic: undefined floating point instruction in supervisor mode > cpuid = 2 > time = 1680784610 > KDB: stack backtrace: > db_trace_self() at db_trace_self > pc = 0xc05eb154 lr = 0xc007a688 (db_trace_self_wrapper+0x30) > sp = 0xdd25c480 fp = 0xdd25c598 > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > pc = 0xc007a688 lr = 0xc02eb1b4 (vpanic+0x140) > sp = 0xdd25c5a0 fp = 0xdd25c5c0 > r4 = 0x00000100 r5 = 0x00000000 > r6 = 0xc0736bfc r7 = 0xc0b1aea8 > vpanic() at vpanic+0x140 > pc = 0xc02eb1b4 lr = 0xc02eaf94 (doadump) > sp = 0xdd25c5c8 fp = 0xdd25c5cc > r4 = 0xc0b92210 r5 = 0x00000000 > r6 = 0xc0610ca0 r7 = 0xf4210a0d > r8 = 0xddf32e4c r9 = 0x00000013 > r10 = 0xdd25c6c0 > doadump() at doadump > pc = 0xc02eaf94 lr = 0xc0610eb0 (vfp_new_thread) > sp = 0xdd25c5d4 fp = 0xdd25c638 > r4 = 0xdd25c6c0 r5 = 0xdd25c5cc > r6 = 0xc02eaf94 r10 = 0xdd25c5d4 > vfp_new_thread() at vfp_new_thread > pc = 0xc0610eb0 lr = 0xc060ff84 (undefinedinstruction+0x178) > sp = 0xdd25c640 fp = 0xdd25c6b8 > undefinedinstruction() at undefinedinstruction+0x178 > pc = 0xc060ff84 lr = 0xc05edaa8 (exception_exit) > sp = 0xdd25c6c0 fp = 0xdd25c750 > r4 = 0x20000013 r5 = 0xde45e000 > r6 = 0xdd25c890 r7 = 0xdd25c8b0 > r8 = 0x00000000 r9 = 0x00000000 > r10 = 0xdd25c8c0 > exception_exit() at exception_exit > pc = 0xc05edaa8 lr = 0xddf31f20 (K256) > sp = 0xdd25c750 fp = 0xdd25c750 > r0 = 0xdd25c890 r1 = 0xde45e000 > r2 = 0xde45e400 r3 = 0xddf309fc > r4 = 0x00000400 r5 = 0xde45e000 > r6 = 0xdd25c890 r7 = 0xdd25c8b0 > r8 = 0x00000000 r9 = 0x00000000 > r10 = 0xdd25c8c0 r12 = 0xdd25c7a0 > zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c > pc = 0xddf32e4c lr = 0xc0946e8c (pcpup) > sp = 0xdd25c758 fp = 0xc0b0aeec > r4 = 0xc0919610 r5 = 0xc0919630 > r6 = 0xc0919618 r7 = 0x642ebce2 > r8 = 0xc0b1b0ec r9 = 0xc0915e88 > r10 = 0xc0b1b0dc > Fatal kernel mode data abort: 'Translation Fault (L1)' on read > trapframe: 0xdd25c330 > FSR=00000005, FAR=95e29398, spsr=200000d3 > r0 =dd25c424, r1 =81000000, r2 =95e29395, r3 =55555555 > r4 =c08ae93c, r5 =00004aa0, r6 =00004aa0, r7 =c08d3e3c > r8 =00000001, r9 =c079567a, r10=0000000b, r11=dd25c3e0 > r12=00000000, ssp=dd25c3c4, slr=00000001, pc =c0610308 > > panic: Fatal abort > . . . (repeats over and over) . . . > > === > Mark Millard > marklmi at yahoo.com > > > --0000000000009253ac05f9a414a4 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Fun...

I'= ;m also fighting aarch64 issues...

Warner

On Tue, Apr 18, 2023, 4:45 PM Mark Millard <marklmi@yahoo.com> wrote:
<= blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px= #ccc solid;padding-left:1ex">https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f= 2e679256c71e0c7413aae9

does not seem to cover armv7, just aarch64. (FreeBSD disabled
floating point for both armv7 and aarch64 but that is a
different change than above.)

I used:

FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img.x= z

booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the
"undefined floating point instruction" notice):

# zpool import
ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow panic! Please consider adding 'options KSTACK_PAGES=3D4' to your kernel co= nfig
panic: undefined floating point instruction in supervisor mode
cpuid =3D 2
time =3D 1680784610
KDB: stack backtrace:
db_trace_self() at db_trace_self
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc05eb154=C2=A0 lr =3D 0xc007a688= (db_trace_self_wrapper+0x30)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c480=C2=A0 fp =3D 0xdd25c598=
db_trace_self_wrapper() at db_trace_self_wrapper+0x30
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc007a688=C2=A0 lr =3D 0xc02eb1b4= (vpanic+0x140)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c5a0=C2=A0 fp =3D 0xdd25c5c0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r4 =3D 0x00000100=C2=A0 r5 =3D 0x00000000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r6 =3D 0xc0736bfc=C2=A0 r7 =3D 0xc0b1aea8=
vpanic() at vpanic+0x140
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc02eb1b4=C2=A0 lr =3D 0xc02eaf94= (doadump)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c5c8=C2=A0 fp =3D 0xdd25c5cc=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r4 =3D 0xc0b92210=C2=A0 r5 =3D 0x00000000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r6 =3D 0xc0610ca0=C2=A0 r7 =3D 0xf4210a0d=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r8 =3D 0xddf32e4c=C2=A0 r9 =3D 0x00000013=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 r10 =3D 0xdd25c6c0
doadump() at doadump
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc02eaf94=C2=A0 lr =3D 0xc0610eb0= (vfp_new_thread)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c5d4=C2=A0 fp =3D 0xdd25c638=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r4 =3D 0xdd25c6c0=C2=A0 r5 =3D 0xdd25c5cc=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4
vfp_new_thread() at vfp_new_thread
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc0610eb0=C2=A0 lr =3D 0xc060ff84= (undefinedinstruction+0x178)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c640=C2=A0 fp =3D 0xdd25c6b8=
undefinedinstruction() at undefinedinstruction+0x178
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc060ff84=C2=A0 lr =3D 0xc05edaa8= (exception_exit)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c6c0=C2=A0 fp =3D 0xdd25c750=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r4 =3D 0x20000013=C2=A0 r5 =3D 0xde45e000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r6 =3D 0xdd25c890=C2=A0 r7 =3D 0xdd25c8b0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r8 =3D 0x00000000=C2=A0 r9 =3D 0x00000000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 r10 =3D 0xdd25c8c0
exception_exit() at exception_exit
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xc05edaa8=C2=A0 lr =3D 0xddf31f20= (K256)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c750=C2=A0 fp =3D 0xdd25c750=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r0 =3D 0xdd25c890=C2=A0 r1 =3D 0xde45e000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r2 =3D 0xde45e400=C2=A0 r3 =3D 0xddf309fc=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r4 =3D 0x00000400=C2=A0 r5 =3D 0xde45e000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r6 =3D 0xdd25c890=C2=A0 r7 =3D 0xdd25c8b0=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r8 =3D 0x00000000=C2=A0 r9 =3D 0x00000000=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0
zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0pc =3D 0xddf32e4c=C2=A0 lr =3D 0xc0946e8c= (pcpup)
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0sp =3D 0xdd25c758=C2=A0 fp =3D 0xc0b0aeec=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r4 =3D 0xc0919610=C2=A0 r5 =3D 0xc0919630=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r6 =3D 0xc0919618=C2=A0 r7 =3D 0x642ebce2=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r8 =3D 0xc0b1b0ec=C2=A0 r9 =3D 0xc0915e88=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 r10 =3D 0xc0b1b0dc
Fatal kernel mode data abort: 'Translation Fault (L1)' on read
trapframe: 0xdd25c330
FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3
r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555
r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c
r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0
r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308

panic: Fatal abort
. . . (repeats over and over) . . .

=3D=3D=3D
Mark Millard
marklmi at yahoo.com


--0000000000009253ac05f9a414a4-- From nobody Tue Apr 18 23:04:35 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1KJ76Hdjz45q1H for ; Tue, 18 Apr 2023 23:04:55 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic308-8.consmr.mail.gq1.yahoo.com (sonic308-8.consmr.mail.gq1.yahoo.com [98.137.68.32]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1KJ70RYvz3vHC for ; Tue, 18 Apr 2023 23:04:54 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681859093; bh=W6bin266JNCR2kcYB2hBkbnRf29W2fbg5xiA5aDeF8s=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=fCf8Z9wGwbmBM9FPlqsyuuhoPDzI664lizU/Lc5XMyOYoUCQPfzPs/oPBUuE1Ww6glJ8nXx43P6VWZPLBvlXvlmMHX5Bn/940OJx7DONFSDZpeOgVDJoqHxLE9C5RM7PEIy8LWXcUzDIIj0KIfkeda1NFbWXYXGibQldQUWU45nkorYKfr5v2NqBarjjWhmxwdAmxiypqVWF4RpSXOrDie6EIbDxLOVKKrR1IejK4vODm4GNXriIu1m94hw/G1VKEaKjIxdSqoA00faMwmdFCR4fyzAKFxvE637U2jXc4hxvms+c4tMxwBM+KDHnEVcxMsZn060BoO22S1I74wdKoQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681859093; bh=7DA4d9p/ECW7enUF2jzPyLxq2tzQ41FrT54hLR+z48d=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=dA2EDDrg+zg84xQolLssrKkNwr7+PfKDeq8FCmMVVY+bw3SScr84T1iBr3I0gFj+oFC1F/OeXBzkRlRkEglXNgzU7PycQbQAfJ46AGTbVK1XoxAsK7noS5smjq8bOG8cGRnciiS0lp6tBpfoRRchnnB2btsojRYvfy9feMG5Tlv0C5Gfjy9pJNcPgab0+mrkMdl3Q/HqAOdb8lncQaFBiDGfVdE0bcEa2k/rPAhfQsoABnhkLnNvJgxgE5cAqRYGCRbC10G2kZWPKS+Qonx8aT/qHav32tTYY+ddd9n9xkETuoId2i6wHK0OMgTY7lCqFjI8XNpbDuZ2/3e9eYDRHA== X-YMail-OSG: aXKbECcVM1lUzADXt0zW_UAJvCi03PIXzJh1skW8tk377cllDV73RjmEhjjKGCt FUaWGxGeIXfi7FaZ7DjyJ8BR5HBwcotHujrD4_yG0xCmDYxFmg.MHu65vvHPY9_D_ef6wITe_ROd mZ7OIgKpcg3UEfTWUn4ju4OLI7MiUd.Wxo0Fye7VQSGInBzpNCay.aYqgTUzG_sYjAU7.DS8UaAh bIMhojVybzORE32SSdGUXwC5bsBBlL9KWC5H9nbYVZfAnemkjHjqZB8un25FxV0594yZ8dBbejfg PXCKlBjE_wDMQPCYuvdJ54tKtugLjI0TdqfTziTEzD7_hraBC9wT2n4jaIuKO5naWhoD8csOjG9l fOmY2i9wuVcKTU2s_wMetkO_WXS8r2HYDJR2eZy42rBskHCmixU0K9GUF3DOo4GNajEhsM53uS.B lUcnRuN031Xpa5aSb_9yQ1_79hYdCm9xmO2nn_H9trVPtOY4X2aNC9oNZ.e2oJPd6PUnz_5Jo98J Y5KbJHdx135598btz.LwdtKWWOGziWHVnfgzV_gMTC6SCBZLX75mqFmSizvnPpRgV_nyrwcXnCYi A6zkG0MTvKmVxRWnv37PP40x_Hg.x1OKiNTiTfxvFnqlpzuyNQKAnnjiqawenKWElTJt_hUSUg7K pDZ3GrGhXM8HbwRxxcyswMR4pkMSDGRIZkdtvkDY9pUmhohSkYmx8iWY78YG9w5MihRH7uS6lEiI I2Yh2ixkHMXV1OnjjdPix1N2o963g3BozyNwo16IG8Sg30hn6vUs5NalS0hWs5nnYIWaJFJ2cvU9 gwqcGykYFs0BZSVbxPVNE1po15zkPw6QFTu6F1OSXsEbRF.3JzxaqjrIF3_aZhdmRFKTa88UvGhD pKmip5G2sOjDxnAjHHodXT9ZYigtaGQtVUeyoxDHq3On_NK825F9s3TBlVuBKGu8YFD0OtexZNxr O2r2PjuwzaPQyhw2V.unNMZoEE8hTjxUZ0zdvf6kJ5PcZwMbdRZhTRsDiry2D1NWspEWcYNy13.l 2GkNmQ9MRb7xCNBDHwgBGROBpJ1PgkHM3na03zpToJhnpDLAXXcbOSMDfErNDe8DaVLJ78rvUG2K raduJMwFmqTZwfIXOTP3e2LOzTA2UtKDOSrTjmhaZfbrq7ddkzdVwEI5UuxKZbK.VJIei4wsw.kb qTdh9YjhNBJ2NdKZyqObW.4si4Jh5jDy4m8nGUCcgbQtyNYnIl0d3j9ftbbtujsRA5Xas0H7zK61 HApxmWU_ETo4KgymKmIW9po9e3sqcKzBpHVEo.Da.pEedDO665FOL2QeChX.YGqApR.zXv1D3cMm scUYkzf6dxZdu5ZgMSh8YIjfvJfzHUmxRE5V2pmAhb1813_ZzG1nGa82tcgfEFyYX7QTJiANN_oZ e75594hLWaBaBrETzPMh4FsGdTgRxabc5zx7FSi9uFdFav6GNNZ7C.eWMZnNiy.ozy76EPRcFB_K WxgyeM0Tnk3iwvbU_pAXMa3SJfGEbTPXMOpFdblxUqS.cYkXs4CtEPvUhNtKReDuaeH2F08pPx6f CoZh911Ggx6Hx1moC.kSjxowtIKpk6Ty07HEXwpAqCsL_8usGh7_trUeAhfH08ETBD10R0klSPd. uwCgovOjzLByA596vZtO.ZnGcYzJYtpWZ61_AFOeCImwFiL.qyS0vdeqN4IKLowOP8JOrasccXur gWCAFZukag1sESorO7YMlnDrOC3FX80ARaXlQ.wWTKBzcVJWEqyRL__LX.txhl5tYUbNczq1zTMk qz35MPscKM8w9BUQhxm.JqYT3G_cINS5qL4KTamw5LVwuvN1YHR1PliexGCaDLJRr1FnGqooZ21P sxEzTS6fwrHhna370UsoIdyPf0Z8zmXhrdBrvLfJnmEovSsTiUO0fCuY.WtWU0Z_Z4BNs0.Z8q1X eZtRP6YqH.OwUv1yQwTf5rWIXCZSumDDVlMXVyXuoBDYOPKiiPRfoz_lv2g5G8QQ5pTXBu7.UEVv 2DlZ886y7YeWZI4QE66PTlN3RKOwJjxffyb317lx_5kdFuO.uDH5hviGRjWcz2W9QJwfRNaPbGWY edRa9_.BXNFeACJIsrarJviyoV0MW_sOu17.sdbaRZuOnez.g7Hd.eQvVKlxfx9.MBIYSRQohHVQ GfxUppGJVoX2N.0CA7B4mCPisv8a3VpfaFm9V8fvD_KjUgHFBnixOXiDA1KkGjKI8Uvth36SrHoY GRhCJpEhyTCxeEwYPdhkKeR_4qh042811qP6XTDnCxxRfaNxjrs2FDEMvmcAuK0a4oxKddxVj7YD H5g8- X-Sonic-MF: X-Sonic-ID: 1aa207e6-f1f0-4b06-bf90-4f6d5c5cac74 Received: from sonic.gate.mail.ne1.yahoo.com by sonic308.consmr.mail.gq1.yahoo.com with HTTP; Tue, 18 Apr 2023 23:04:53 +0000 Received: by hermes--production-ne1-7dbd98dd99-84p8v (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 7af015081c4cc3836a2da824971f8444; Tue, 18 Apr 2023 23:04:47 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: The import of openzfs vs. armv7: boot crashs From: Mark Millard In-Reply-To: Date: Tue, 18 Apr 2023 16:04:35 -0700 Cc: Current FreeBSD , freebsd-arm , Mateusz Guzik , Pawel Jakub Dawidek , Kyle Evans Content-Transfer-Encoding: quoted-printable Message-Id: References: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F.ref@yahoo.com> <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> To: Warner Losh X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4Q1KJ70RYvz3vHC X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Apr 18, 2023, at 15:46, Warner Losh wrote: > Fun... >=20 > I'm also fighting aarch64 issues... Of what kind? I've been able to use things as committed in FreeBSD (block_cloning never having been enabled but jumping from before the import to, effectively, after the FreeBSD adjustments). But I have not tried anything that is different as committed in openzfs. (I'm one of those that tested poudriere bulk activity via separate media from my normal aarch64 context. Those tests had no problems once the full set up adjustments was present in my context.) > Warner >=20 > On Tue, Apr 18, 2023, 4:45 PM Mark Millard wrote: > = https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c7413= aae9 >=20 > does not seem to cover armv7, just aarch64. (FreeBSD disabled > floating point for both armv7 and aarch64 but that is a > different change than above.) I probably should have explicitly noted that the fpu disabling was from after the snapshot being tested here. The point of the snapshot test (the most recent available) was to find out if armv7 crashed before the fpu-use disabling commit. > I used: >=20 > = FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img.= xz That is from after the import and after: =E2=80=A2 git: eb1feadc201a - main - zfs: fix null ap->a_fsizetd = NULL pointer derefernce Martin Matuska but with no other zfs changes. It does not contain: =E2=80=A2 git: d6e24901349d - main - zfs: disable kernel fpu usage = on arm and aarc64 Mateusz Guzik (But the openzfs changes are different.) > booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the > "undefined floating point instruction" notice): >=20 > # zpool import > ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow = panic! > Please consider adding 'options KSTACK_PAGES=3D4' to your kernel = config > panic: undefined floating point instruction in supervisor mode > cpuid =3D 2 > time =3D 1680784610 > KDB: stack backtrace: > db_trace_self() at db_trace_self > pc =3D 0xc05eb154 lr =3D 0xc007a688 = (db_trace_self_wrapper+0x30) > sp =3D 0xdd25c480 fp =3D 0xdd25c598 > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > pc =3D 0xc007a688 lr =3D 0xc02eb1b4 (vpanic+0x140) > sp =3D 0xdd25c5a0 fp =3D 0xdd25c5c0 > r4 =3D 0x00000100 r5 =3D 0x00000000 > r6 =3D 0xc0736bfc r7 =3D 0xc0b1aea8 > vpanic() at vpanic+0x140 > pc =3D 0xc02eb1b4 lr =3D 0xc02eaf94 (doadump) > sp =3D 0xdd25c5c8 fp =3D 0xdd25c5cc > r4 =3D 0xc0b92210 r5 =3D 0x00000000 > r6 =3D 0xc0610ca0 r7 =3D 0xf4210a0d > r8 =3D 0xddf32e4c r9 =3D 0x00000013 > r10 =3D 0xdd25c6c0 > doadump() at doadump > pc =3D 0xc02eaf94 lr =3D 0xc0610eb0 (vfp_new_thread) > sp =3D 0xdd25c5d4 fp =3D 0xdd25c638 > r4 =3D 0xdd25c6c0 r5 =3D 0xdd25c5cc > r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4 > vfp_new_thread() at vfp_new_thread > pc =3D 0xc0610eb0 lr =3D 0xc060ff84 = (undefinedinstruction+0x178) > sp =3D 0xdd25c640 fp =3D 0xdd25c6b8 > undefinedinstruction() at undefinedinstruction+0x178 > pc =3D 0xc060ff84 lr =3D 0xc05edaa8 (exception_exit) > sp =3D 0xdd25c6c0 fp =3D 0xdd25c750 > r4 =3D 0x20000013 r5 =3D 0xde45e000 > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > r8 =3D 0x00000000 r9 =3D 0x00000000 > r10 =3D 0xdd25c8c0 > exception_exit() at exception_exit > pc =3D 0xc05edaa8 lr =3D 0xddf31f20 (K256) > sp =3D 0xdd25c750 fp =3D 0xdd25c750 > r0 =3D 0xdd25c890 r1 =3D 0xde45e000 > r2 =3D 0xde45e400 r3 =3D 0xddf309fc > r4 =3D 0x00000400 r5 =3D 0xde45e000 > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > r8 =3D 0x00000000 r9 =3D 0x00000000 > r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0 > zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c > pc =3D 0xddf32e4c lr =3D 0xc0946e8c (pcpup) > sp =3D 0xdd25c758 fp =3D 0xc0b0aeec > r4 =3D 0xc0919610 r5 =3D 0xc0919630 > r6 =3D 0xc0919618 r7 =3D 0x642ebce2 > r8 =3D 0xc0b1b0ec r9 =3D 0xc0915e88 > r10 =3D 0xc0b1b0dc > Fatal kernel mode data abort: 'Translation Fault (L1)' on read > trapframe: 0xdd25c330 > FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3 > r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555 > r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c > r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0 > r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308 >=20 > panic: Fatal abort > . . . (repeats over and over) . . . >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Tue Apr 18 23:09:03 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1KP91x0Hz45pvy for ; Tue, 18 Apr 2023 23:09:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1KP84PZ9z438R for ; Tue, 18 Apr 2023 23:09:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-94a35b0d130so699543566b.3 for ; Tue, 18 Apr 2023 16:09:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1681859355; x=1684451355; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=pWCDpobE6xCBVw1n2TZ2xi4qOsNQRtSaBbrE0kELplQ=; b=zXDzGtl3uH3QvrPEY3+QUS47YanPCSeLsXio7U9cnSVRkp5AidxklnvF+BvLWwIdkb e/XoeR+LRBqidmVwe1xOwMZ0qJrOdNSMaTXpewWsVnQhucvP3Gs1hw4L8YdNr7HHlAG5 JvPGcQVWVbZHzEXGDUkmDyTapXrVIsP+EkDgsKsi+g6JcDXY+lrRc0N7ISbELDQM7VC5 cV/6VvmiSZEJKpEu/8ADW0X4pHo7V41FTQ4e/FA5MQb2UcA9q/DIUqFD7Ju3xcsqeiyn LJStza7jNOrB0gqNZDXIr8kMbJp9KDiat6SynsPRorXNeerh6pFoKxT1srt0eiU9n7DS PwNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681859355; x=1684451355; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=pWCDpobE6xCBVw1n2TZ2xi4qOsNQRtSaBbrE0kELplQ=; b=EmyRjnP7jx3YDXL+BzgjsaKmlL/Ha7H4IUGBv3/kIBQRUeEOTgZ00Hw3sXtHe2AZfY yg3uS+Z+wikRpPNVsbXQCgA13NXoMf4T+4EOFZY4caGPvYI7L+MF5DYQHD5GRcilSNGr yvwwO8xst4PH2vd0LMOFo8dwPt4N9V+c4l6A+x9wrNa0+SSNkBKWbwzBwE7NasXeib3U oO5LpprsDcdVP9F9QhPpD3JJYNzQlPtCWBs9tDv6dHOhIL1MVnTGJ7n8SX8UW2XNf5+j uQIBNBy/AAnyazFWZCIY0szNqU7BnOC9qcMDBy43r2pE1tJddLfTWX0+CgIONpmcdDhO bGag== X-Gm-Message-State: AAQBX9fI2no8bPw/Qz33OdK8bY2wlS+xe8zFj55CfkyzhP1xYNWQn3Z8 MGT1bcoBBgdTnODJgWZXwHAwFwdJDStCVuYSPeei+Q== X-Google-Smtp-Source: AKy350b8Wc1KjgR+hvHCSS5WQsiEcMs+6Brvg2xfAeo1OBJSQCjL1Ws5ExLFpaA9A7QG1h4ulHT8IpjDwoTB8uhObAc= X-Received: by 2002:a05:6402:745:b0:501:cced:9c6c with SMTP id p5-20020a056402074500b00501cced9c6cmr3373004edy.7.1681859354849; Tue, 18 Apr 2023 16:09:14 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F.ref@yahoo.com> <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> In-Reply-To: From: Warner Losh Date: Tue, 18 Apr 2023 17:09:03 -0600 Message-ID: Subject: Re: The import of openzfs vs. armv7: boot crashs To: Mark Millard Cc: Current FreeBSD , freebsd-arm , Mateusz Guzik , Pawel Jakub Dawidek , Kyle Evans Content-Type: multipart/alternative; boundary="00000000000068890105f9a465db" X-Rspamd-Queue-Id: 4Q1KP84PZ9z438R X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --00000000000068890105f9a465db Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Tue, Apr 18, 2023 at 5:04=E2=80=AFPM Mark Millard wr= ote: > On Apr 18, 2023, at 15:46, Warner Losh wrote: > > > Fun... > > > > I'm also fighting aarch64 issues... > > Of what kind? I've been able to use things as committed > in FreeBSD (block_cloning never having been enabled but > jumping from before the import to, effectively, after > the FreeBSD adjustments). But I have not tried anything > that is different as committed in openzfs. > > (I'm one of those that tested poudriere bulk activity > via separate media from my normal aarch64 context. Those > tests had no problems once the full set up adjustments > was present in my context.) > All boot loader issues for special environments. Not in the kernel, so maybe unrelated and I shouldn't have said anything. I'm guessing that upstream needs a generic way to disable all acceleration, even if that has bad performance. I'm looking for a good way to do that (once I get done with the bugs I was fixing when I noticed this issue). Warner > > Warner > > > > On Tue, Apr 18, 2023, 4:45 PM Mark Millard wrote: > > > https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c741= 3aae9 > > > > does not seem to cover armv7, just aarch64. (FreeBSD disabled > > floating point for both armv7 and aarch64 but that is a > > different change than above.) > > I probably should have explicitly noted that the fpu disabling > was from after the snapshot being tested here. > > The point of the snapshot test (the most recent available) was > to find out if armv7 crashed before the fpu-use disabling commit. > > > I used: > > > > > FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img= .xz > > That is from after the import and after: > > =E2=80=A2 git: eb1feadc201a - main - zfs: fix null ap->a_fsizetd NULL= pointer > derefernce Martin Matuska > > but with no other zfs changes. It does not contain: > > =E2=80=A2 git: d6e24901349d - main - zfs: disable kernel fpu usage on= arm and > aarc64 Mateusz Guzik > > (But the openzfs changes are different.) > > > booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the > > "undefined floating point instruction" notice): > > > > # zpool import > > ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow pani= c! > > Please consider adding 'options KSTACK_PAGES=3D4' to your kernel config > > panic: undefined floating point instruction in supervisor mode > > cpuid =3D 2 > > time =3D 1680784610 > > KDB: stack backtrace: > > db_trace_self() at db_trace_self > > pc =3D 0xc05eb154 lr =3D 0xc007a688 (db_trace_self_wrapper+0x= 30) > > sp =3D 0xdd25c480 fp =3D 0xdd25c598 > > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > > pc =3D 0xc007a688 lr =3D 0xc02eb1b4 (vpanic+0x140) > > sp =3D 0xdd25c5a0 fp =3D 0xdd25c5c0 > > r4 =3D 0x00000100 r5 =3D 0x00000000 > > r6 =3D 0xc0736bfc r7 =3D 0xc0b1aea8 > > vpanic() at vpanic+0x140 > > pc =3D 0xc02eb1b4 lr =3D 0xc02eaf94 (doadump) > > sp =3D 0xdd25c5c8 fp =3D 0xdd25c5cc > > r4 =3D 0xc0b92210 r5 =3D 0x00000000 > > r6 =3D 0xc0610ca0 r7 =3D 0xf4210a0d > > r8 =3D 0xddf32e4c r9 =3D 0x00000013 > > r10 =3D 0xdd25c6c0 > > doadump() at doadump > > pc =3D 0xc02eaf94 lr =3D 0xc0610eb0 (vfp_new_thread) > > sp =3D 0xdd25c5d4 fp =3D 0xdd25c638 > > r4 =3D 0xdd25c6c0 r5 =3D 0xdd25c5cc > > r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4 > > vfp_new_thread() at vfp_new_thread > > pc =3D 0xc0610eb0 lr =3D 0xc060ff84 (undefinedinstruction+0x1= 78) > > sp =3D 0xdd25c640 fp =3D 0xdd25c6b8 > > undefinedinstruction() at undefinedinstruction+0x178 > > pc =3D 0xc060ff84 lr =3D 0xc05edaa8 (exception_exit) > > sp =3D 0xdd25c6c0 fp =3D 0xdd25c750 > > r4 =3D 0x20000013 r5 =3D 0xde45e000 > > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > > r8 =3D 0x00000000 r9 =3D 0x00000000 > > r10 =3D 0xdd25c8c0 > > exception_exit() at exception_exit > > pc =3D 0xc05edaa8 lr =3D 0xddf31f20 (K256) > > sp =3D 0xdd25c750 fp =3D 0xdd25c750 > > r0 =3D 0xdd25c890 r1 =3D 0xde45e000 > > r2 =3D 0xde45e400 r3 =3D 0xddf309fc > > r4 =3D 0x00000400 r5 =3D 0xde45e000 > > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > > r8 =3D 0x00000000 r9 =3D 0x00000000 > > r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0 > > zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c > > pc =3D 0xddf32e4c lr =3D 0xc0946e8c (pcpup) > > sp =3D 0xdd25c758 fp =3D 0xc0b0aeec > > r4 =3D 0xc0919610 r5 =3D 0xc0919630 > > r6 =3D 0xc0919618 r7 =3D 0x642ebce2 > > r8 =3D 0xc0b1b0ec r9 =3D 0xc0915e88 > > r10 =3D 0xc0b1b0dc > > Fatal kernel mode data abort: 'Translation Fault (L1)' on read > > trapframe: 0xdd25c330 > > FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3 > > r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555 > > r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c > > r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0 > > r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308 > > > > panic: Fatal abort > > . . . (repeats over and over) . . . > > > > > > > =3D=3D=3D > Mark Millard > marklmi at yahoo.com > > --00000000000068890105f9a465db Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


=
On Tue, Apr 18, 2023 at 5:04=E2=80=AF= PM Mark Millard <marklmi@yahoo.com<= /a>> wrote:
O= n Apr 18, 2023, at 15:46, Warner Losh <imp@bsdimp.com> wrote:

> Fun...
>
> I'm also fighting aarch64 issues...

Of what kind? I've been able to use things as committed
in FreeBSD (block_cloning never having been enabled but
jumping from before the import to, effectively, after
the FreeBSD adjustments). But I have not tried anything
that is different as committed in openzfs.

(I'm one of those that tested poudriere bulk activity
via separate media from my normal aarch64 context. Those
tests had no problems once the full set up adjustments
was present in my context.)

All boot lo= ader issues for special environments. Not in the kernel,
so maybe= unrelated and I shouldn't have said anything.

I'm guessing that upstream needs a generic way to disable all
acceleration, even if that has bad performance. I'm looking for a
good way to do that (once I get done with the bugs I was fixing
when I noticed this issue).

Warner
=C2=A0
> Warner
>
> On Tue, Apr 18, 2023, 4:45 PM Mark Millard <marklmi@yahoo.com> wrote:
> https://github.co= m/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c7413aae9
>
> does not seem to cover armv7, just aarch64. (FreeBSD disabled
> floating point for both armv7 and aarch64 but that is a
> different change than above.)

I probably should have explicitly noted that the fpu disabling
was from after the snapshot being tested here.

The point of the snapshot test (the most recent available) was
to find out if armv7 crashed before the fpu-use disabling commit.

> I used:
>
> FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.= img.xz

That is from after the import and after:

=C2=A0 =C2=A0 =E2=80=A2 git: eb1feadc201a - main - zfs: fix null ap->a_f= sizetd NULL pointer derefernce Martin Matuska

but with no other zfs changes. It does not contain:

=C2=A0 =C2=A0 =E2=80=A2 git: d6e24901349d - main - zfs: disable kernel fpu = usage on arm and aarc64 Mateusz Guzik

(But the openzfs changes are different.)

> booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the > "undefined floating point instruction" notice):
>
> # zpool import
> ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow pan= ic!
> Please consider adding 'options KSTACK_PAGES=3D4' to your kern= el config
> panic: undefined floating point instruction in supervisor mode
> cpuid =3D 2
> time =3D 1680784610
> KDB: stack backtrace:
> db_trace_self() at db_trace_self
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc05eb154=C2=A0 lr =3D 0xc00= 7a688 (db_trace_self_wrapper+0x30)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c480=C2=A0 fp =3D 0xdd2= 5c598
> db_trace_self_wrapper() at db_trace_self_wrapper+0x30
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc007a688=C2=A0 lr =3D 0xc02= eb1b4 (vpanic+0x140)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c5a0=C2=A0 fp =3D 0xdd2= 5c5c0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0x00000100=C2=A0 r5 =3D 0x000= 00000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc0736bfc=C2=A0 r7 =3D 0xc0b= 1aea8
> vpanic() at vpanic+0x140
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc02eb1b4=C2=A0 lr =3D 0xc02= eaf94 (doadump)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c5c8=C2=A0 fp =3D 0xdd2= 5c5cc
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0xc0b92210=C2=A0 r5 =3D 0x000= 00000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc0610ca0=C2=A0 r7 =3D 0xf42= 10a0d
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0xddf32e4c=C2=A0 r9 =3D 0x000= 00013
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xdd25c6c0
> doadump() at doadump
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc02eaf94=C2=A0 lr =3D 0xc06= 10eb0 (vfp_new_thread)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c5d4=C2=A0 fp =3D 0xdd2= 5c638
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0xdd25c6c0=C2=A0 r5 =3D 0xdd2= 5c5cc
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4=
> vfp_new_thread() at vfp_new_thread
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc0610eb0=C2=A0 lr =3D 0xc06= 0ff84 (undefinedinstruction+0x178)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c640=C2=A0 fp =3D 0xdd2= 5c6b8
> undefinedinstruction() at undefinedinstruction+0x178
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc060ff84=C2=A0 lr =3D 0xc05= edaa8 (exception_exit)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c6c0=C2=A0 fp =3D 0xdd2= 5c750
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0x20000013=C2=A0 r5 =3D 0xde4= 5e000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xdd25c890=C2=A0 r7 =3D 0xdd2= 5c8b0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0x00000000=C2=A0 r9 =3D 0x000= 00000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xdd25c8c0
> exception_exit() at exception_exit
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xc05edaa8=C2=A0 lr =3D 0xddf= 31f20 (K256)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c750=C2=A0 fp =3D 0xdd2= 5c750
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r0 =3D 0xdd25c890=C2=A0 r1 =3D 0xde4= 5e000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r2 =3D 0xde45e400=C2=A0 r3 =3D 0xddf= 309fc
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0x00000400=C2=A0 r5 =3D 0xde4= 5e000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xdd25c890=C2=A0 r7 =3D 0xdd2= 5c8b0
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0x00000000=C2=A0 r9 =3D 0x000= 00000
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0=
> zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 pc =3D 0xddf32e4c=C2=A0 lr =3D 0xc09= 46e8c (pcpup)
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 sp =3D 0xdd25c758=C2=A0 fp =3D 0xc0b= 0aeec
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r4 =3D 0xc0919610=C2=A0 r5 =3D 0xc09= 19630
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r6 =3D 0xc0919618=C2=A0 r7 =3D 0x642= ebce2
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 r8 =3D 0xc0b1b0ec=C2=A0 r9 =3D 0xc09= 15e88
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0r10 =3D 0xc0b1b0dc
> Fatal kernel mode data abort: 'Translation Fault (L1)' on read=
> trapframe: 0xdd25c330
> FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3
> r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555
> r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c
> r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0
> r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308
>
> panic: Fatal abort
> . . . (repeats over and over) . . .
>




=3D=3D=3D
Mark Millard
marklmi at yahoo.com

--00000000000068890105f9a465db-- From nobody Tue Apr 18 23:55:56 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1LRL4X73z45t0B for ; Tue, 18 Apr 2023 23:56:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-25.consmr.mail.gq1.yahoo.com (sonic303-25.consmr.mail.gq1.yahoo.com [98.137.64.206]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1LRK3GzKz4H0R for ; Tue, 18 Apr 2023 23:56:13 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681862171; bh=NB1fltZ5kp32UB4/jd8DTRcsgcnvoTjB6TTBeZqpx9I=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=I9qqirZ1ozyPIa+7oLHJifYKpEqeV+FDJVYs3c2ohyNuaVDj1k/fGafyzHhosRUuvXl1MgiJkpDXLgkFz/OEoszIxoHlnnMCp5Ovg0YvK2DJZeHtPPPAg0XDQ7sp5eYmHfOdd4YCzBCap2jRdrB/tKNxCi690rrCx0hUwzTHEG+qUuE3ROlfnrUe5fNsxA59znfhDXIkydn4UZ6coGYU9j7sh/A0jNmwteebKLWP0He+qShMKyUZcKokI2LuZRgN4w+F7JiRA8iJ4R6EVoIfYJYVYzg8bb6chTeIvo/PD67O50bz5Rnr9klI0z++A4+wIdrf8lgG2RaPxI9mh7lRIw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681862171; bh=fXndzEeYOU+d59tlijJU70pwB0QBiTHCDhIaCMlPa6D=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=uiGeqnCOKyCnPOjOsnfgXMXZIB96m71YLgn/jEhsG9ps3WX5iwdfP5I2HL7zhxf7AVajzZdbciidHMX9SKjOqmzGe80TXFZvrAPt+YUZEHqeYqDdUmVVr8U3Fq449yn1pDc6zDZadSc4NLHxCeYX4f7g5a/E/o7WK/QuXdOI4jqXtCBPRAEu8JblCCSSLOxZdRs1AGgZCIpyO3pbbALhQdTda5e21+5wkT/bVoHEHoxGlbz8qEmNfyQb8yi2z8uw+67f8s94CJEJjmP6KnPy7T4+tM1cg+OjSwGeuj3sf5Lcf2kLoOWhbOn8LElVyLbl7Bwxia6xjO9VCyY5W3HgkA== X-YMail-OSG: ecA0eTgVM1mL2gWYvo5vfWKAo9oqcgoe6zwwQKOg5YqSRx38fi7kPF3WJgJGs75 AR8.ofmjL4il8agjs0UMJ3K_ek11M.OLGv.WjjfKA4iOUQgUoPpfctQ_KcXMI_f16a5YPFh12rGW tAQVw31BNM7YCs00nB05DXSoL7uKjFZ7ib.ctSjEhDnprUwmrHNCFvkRewUXWhz7ggINfouZsRfH rRQEwAaLE.M6TCo8PXUYKjuot7U0.ezYPKyv1jQ8hTsXLu5LwyQowG6JIuq.lygG2afzyWc23IqR qS7KrTxGB7sSvNzuFKkJoY50NcDzY1tFFDPzwGtcpuu4Hz89r5t4LiKPn7k4Npx1OBZYVdjLPz2x YuOUSrXeiGWIUeW6lZqEHTXIA9dNlPY0cyGtqNXuDvOzqQ0nw3MJ0bY_Gzxcx0StLfHpTskMmW93 PA4o5vlPvx0QovofHynK5U1emAaIlbWF4JWCq9MQ.ztbBa3yn8VQeG4bvjGpjzJNRlkzqc725nyF Qvvs_vQR5giXwHS555vXN_CyVZ_Agsoa0cq3NGKKbtBIkFF5myT6ZidF0JYGAvFVffNu.p3FngVj EbE7BLJbSXemwMwSSXmnG6KlSetDDMuwBRzBGXQ97nyxjk3F0q4nP6VlcQvzhv0vP2ieohKU4q1G TdrshFAVf13F2oLUYEgMNj4D1O5XY96ucnlc.l6xLLLLmoON.OUUv2jnLMMK0LwRN4vXi9Gv9aHd ujFr.MdpJzTfLiiHuVjmV2vW_2ccgWNqGCMYzf04iJ734GJe2vkSqhQbMmZqT7081oddU3luZkqD HAABeysho1WqOjh2B1NUROAYXBCCJdCmvMk60oUwgNM4Ft7r9SowTPheTjFNefe6p7ukOXUAtdyS nnb0rxorMfvVWmxkKoGYEdrQ8PFIhsAqHAA7zip6cZIwMqLMICS25D7QJ3l.XadfspHEW.XSyde9 Cz_SUMj5.iPmdVqY7nLIpw5IKgRr1PQsWm6zEXVTxmEXmZQdU407XNta7BoJEq63wuM2jAz2wgPY taJHh0to2ZsKVZ9PTwR_es07JJ87uHwlXSjss3fYu715nsCZ1jY6391Jo.yu3pG_6AxwkZzRbbrH O73hJvMkyML5cgqWSlPJXqfm0zr.NGmx4i2ff1Jdk2Vtkj68S56BJQ3bT3jCtVj.LdRZo.0LBIjR 3td3bqOUE8uwgPH.mgahyxzueTA_vWSniTXzNHqvFT0YdmQClSmwD7dofPCosAIqvLfEUuIEq6.v P_r0I1Naudk6Ge5ZzZ9kuM7WJ3_x2U6Od5luCDSmSrqvAaeHCGK29FfQ3nAIaqo4dEBSCf188an_ lRTV9qikz9nzfMV_pqkM6oKSDI95LbhqSvdJ6ZFd48b2o8O4knUi80p9VOoeqvDM7FGuwBouBjJ7 6uELeAuB1N2Qa63f5USvNJSGuEaw1K7Ve63nq0XTtT1oGOCJIYkSrlXDKTPZLNADek6S7wgy8rHK ABEMWyntM0YSl347GSd4wrAQQ6fiJlYjuaLiV_Az0vETQOkZSjxxb_zQBffbMXYTAnoo0uElaaBo 6AiNSxreK9D4saPTKf0JDU1Ixe8EkIj6erL3t51Nklexy5cHvp1DbpjeePgjml.8P95VUnpuMhl. 64Ncia55IGHaH.RTBzV.9NB1wpIvOzrJ9i0Hd5Gu5csBFrWWdrDhm2qfw1jYEsFbuWTk.xOdd_Ck DVLmf5O7VZ6dgaqrgZjhwiliydlRgXZkY5s1NhVsLiirgVP8HkrrUp8Kfugw03DOxtbg9tqjOA37 7WodXI_rN8KWf7bko0pAcdHUgVy6CuOGvyXayeFLe5rpO9OaRc_od4HqsdfAar4wfVTOaS_Za3oD ehaPtM4HLmk1YFcFNIMGXaU._IGBSbpefyWjIW.Ss7vZpKe0JuN_ofL_vmEiGY1S3D07BBBitWis YtkWDIlNodL3WOauYmjNZWVVoPPtXv_T8xwbVfT.Gl0VXSGlUcYYSmgtpFw_4IL09hVs9xVsg2cJ S1KoDGAzygVpxLNjOFksNT4QxqF892GHoli5Qg_h7L41XAAXc5mvQh3eiymTzkciZiyNsRO0ny.y kiWO79SJ5JOGDR5SxbgwhVmsKDY3IpvOBY.DUGHA1_RsfUycRMyQnBkOewfc_praSOmxyCvQ1CF5 fgR1uVo88R49jOwpuif4Xqt0ylaoU8Oh6IWoGxEU.O9mxtXmSb4YOUmPMdCXvowEKkNXlr89bb13 HdPqTZUCkAGBZqfoUCzmdF3cIMNIOpks4XIFwt3WkSnYVJsqAjb9yrUhPzLjwdjv8JE1KB9hCquW yu1t1kA-- X-Sonic-MF: X-Sonic-ID: 4e0e9d0a-0176-4df4-9913-681f2a24736b Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Tue, 18 Apr 2023 23:56:11 +0000 Received: by hermes--production-gq1-546798879c-8jjxz (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 4715eb63546015f84a72ead8d556e8a2; Tue, 18 Apr 2023 23:56:06 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: git: 2a58b312b62f - main - zfs: merge openzfs/zfs@431083f75 From: Mark Millard In-Reply-To: <8fecc2d8c2eb471ec70122a7d5fb249d@mail.yourbox.net> Date: Tue, 18 Apr 2023 16:55:56 -0700 Cc: Current FreeBSD Content-Transfer-Encoding: quoted-printable Message-Id: <9EA4F05D-F9BB-41DC-9C57-16DFEE61CC4F@yahoo.com> References: <963B0620-5342-4EC3-AA54-52DDD70D9E3D.ref@yahoo.com> <963B0620-5342-4EC3-AA54-52DDD70D9E3D@yahoo.com> <8fecc2d8c2eb471ec70122a7d5fb249d@mail.yourbox.net> To: =?utf-8?B?Sm9zw6kgUMOpcmV6?= X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Rspamd-Queue-Id: 4Q1LRK3GzKz4H0R X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N On Apr 18, 2023, at 15:02, Jos=C3=A9 P=C3=A9rez wrote: > El 2023-04-18 21:37, Mark Millard escribi=C3=B3: >>> In this case it does because the value is "active". If it's = "enabled" >>> you do not need to do anything. >> Well, if block_cloning is disabled it would not become active. > [...] >> So, in progressing past the vintage that corrupt zfs data, >> one could end up with block_cloning enabled in the process. >=20 > You still have to willingly issue the command > zpool upgrade > so you might not just end up with the feature enabled by running this > or that kernel, that's why I suggested step 0: verify if you are the > worst case scenario before you begin. I was not really worried about the no-zpool-upgrade/disabled case. I was worried about "enabled" vs "active" as the transition enabled -> active is automatic based on activity. But there is overall disabled vs. enabled vs. active for the block_cloning feature so I mentioned all 3. >>> Boot in single user mode and check if your pool has block cloning in >>> use: >>> # zpool get feature@block_cloning zroot >>> NAME PROPERTY VALUE SOURCE >>> zroot feature@block_cloning active local >>> In this case it does because the value is "active". If it's = "enabled" >>> you do not need to do anything. >=20 > If you did not upgrade the pool, the feature would just not be there = and > the pool is sane (*). "Not being there" vs. "disabled" has some context to it. I worded based on the way my context shows things. My example context has: # zpool get all zroot | grep compat zroot compatibility openzfs-2.1-freebsd = local which explains the particular list of disabled features reported below. (It is a "never had zpool upgrade" context as well.) # zpool get all zroot | grep disabled zroot feature@edonr disabled = local zroot feature@zilsaxattr disabled = local zroot feature@head_errlog disabled = local zroot feature@blake3 disabled = local zroot feature@block_cloning disabled = local so "not be there" seems to mean "disabled" as zpool presents things based on compatibility. Just to see the command you listed fully but in my type of context: # zpool get feature@block_cloning zroot NAME PROPERTY VALUE SOURCE zroot feature@block_cloning disabled local # zpool version zfs-2.1.99-FreeBSD_g431083f75 zfs-kmod-2.1.99-FreeBSD_g431083f75 (Those are software versions, not properties of specific pools.) I'll note that I see: # zpool get feature@JUNKNAME zroot #=20 So features that the software does not have in its list of possibilities get an empty result. > unaffected_machine# zpool get feature@block_cloning zroot > unaffected_machine# That is the same sort of output as in my feature@JUNKNAME test above. It is not clear from what is presented that the context had block_cloning in its list of possibilities. In my normal environment (that still predates the import of the openzfs update), I get the same sort of result for feature@block_cloning as you show above. > As said, if the feature has been enabled but no calls to > copy_file_range() occurred, the pool is also sane. A the time but more activity can change the status because copy_file_range() could be called. So I expect that the following step is relvant to avoid ending up with block_cloning becoming active: QUOTE When in single user mode set compression property to "off" on any zfs=20 active dataset that has compression other than "off" and the sync=20 property to something other than "disabled". END QUOTE > To summarize: > no feature -> sane > feature "enabled" -> sane > feature "active" -> might not be sane >=20 > BR, >=20 > (*) as per this bug. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Apr 19 02:10:20 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1PQR15wpz44mqw for ; Wed, 19 Apr 2023 02:10:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic305-21.consmr.mail.gq1.yahoo.com (sonic305-21.consmr.mail.gq1.yahoo.com [98.137.64.84]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1PQP4HQvz3GjV for ; Wed, 19 Apr 2023 02:10:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=StIujkG2; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.84 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681870235; bh=ajs9kJKCPqGfSIyv/yjgCnd05PrlLp0da7KFrW6Wu8s=; h=From:Subject:Date:Cc:To:References:From:Subject:Reply-To; b=StIujkG2hQMraK5D17h0NkHKOJLlztceDPsZPII+bXKUsIjk5Ey22srTfSgvKfo7/Tx3QOPeWwDGxbt1u2xP7XAA0p85p4FlST3J03IUzx33YNboX6Dme4ulnOpJyMtNdizUOwC1LDQ9wbxqzeicEWBRkker9GKJuq5smspaUWadlo4Q2YktHWrJ4omdzBVxLALph8M6p3qvF9JZEtgfPV35WLWYuS4UFvPm9zZamEMzJC5NklWc3ezO2mBi64EVPdEBKQujhU1nzsSE2cobebyrwnn2TGJyNlmw/Glfq2cAOZzP9euKyUDWnhBbqLgdWuaU7rbvvotvxMklQzbCKw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681870235; bh=CsOGCAnAA9b1XLGNosCOtr0h+2SqYlkl8HJSZFvFnB6=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=UGLHNQHYJAadiBXFHbHi8CE/+kJKCK5DpACM+dMhNl+cwRa126kmBkAtDCcor9Y8x2/my1OTR/nmyOKkwENLHsJ6pxn9Wt9IigRXKF0QB9u/Zc275uSiV9hjY3FT22KUVnbWMx1DAaS+Rz7NpH8CvF86An66VodcHpaZbf1pJMf0O4TcvlGmpwf+p34IdGrUd7a+tnZqaY2zJSe5FqBjZ4T47tssNnWUFLaFzq0q/3L3qZ/lYqjZ1oI0bU8+e/kS3OhDSSrv9dIxb1JWmt35siQYbsncSaHaUhNU2I2pfPmmdvkQ2rnPE3TcYHeORgSW5zKbwHlkuV65rwAYylGm7Q== X-YMail-OSG: CSH3GZcVM1l8sDoYBdFEe5.16r5NkD6T_1z0a5GCTptmeeSUaCSUrWh2bbZed6x vFUJZsXpYivLZ7Oxn6e61a2YhOiuJmyjQaxcUerNo0fSho_CGWoWGm4j4r2XKCI1rNiRZgLdHmpO _CZUkhiBJJcti5SdqOy_q3A1lEWj2OvVt0a_vc2MfeyfKhxYmuS9mo8avpEFmUMXmuXARqeqhZtQ 10tTuvCJ4qEJ6Xht6jEPDWG1e8ZCOxHNQbAPDuvnJ4xUKwsZn_F3t8Zi7vOjKSz6AwYk_qo32CKv LhAZinU1KYcXqUHN.wQM3kzmcLlK5S_Sx2LDUZ1Segb5Xzu6a99KVJJ51nmsJwCxHgjJzvGLmF7E KH4jggF6oOZ2YKEP8m3vcEVCe4A5VOKSOZPHXhYRJENSGUmVnkj5Rx3DUy8mjOThIGvGZb.Y2M8g weHkPA5.C51PX10Zj3mJ4fTHOW_668.7.i4fCJXhONyxORoshRaXEcES23zi_lMkLwMd0ptCjgG6 nG6YQgFgBJnZ1bRecnaaN5pv7k7SIUNr_siARYjF70ywfPqkdXPCazf2STgvSVQwEQCM._1qed5g SFZqhdsvSuPjNLr6CPBKi4TovGW.LwyRFTA4n4JvA8YQjHu1iEkpze44dpCtpplPnoXWL5Crj3jV WMwQal3hmO3kCwBEeDH2Qvk4iiNrTh5asSga3NuEPoTtIRtIStPoxUonEJ1LDsDl9V8VZEeYNmeP uswSdoi7gGxWTkV15QYxxeRv.F2.QE_wc8AsxGL19FaRAUjJHf.A_tM4sZEYM3i8v3N5sUV22FGS DYWARj7dYtI3AshaY9Pe8qHAhVE6G3p5evrJlcgcyGABs355cgAkwapiNfvwFxiH4XyX7sIo_QWb URxdWm_o5pQl04n_Ay8jBF_IiRSHwcJKrrfwjlnKVWy010oi42ENYFDH9vstXo8f8KS0mw.ne.lk wbRI6o1456S.YSzx3FfjMhQdB.fGyGC6EFI5o0tb._Ke3IR8fpn2L9oqFJr5oWqBZNL0Cauzxy7S CU76D_tB4u2BtwDh1RTMyrT_HCjr1ZPhVE.kKFEPDz6YP4fGRwdt5TxmqfsulZFAwVwIFxBXGuwS 0GoJjzntmW20ujDIb7fm8t1qHj439s.7x8ZonHf2NFYEb9m6GTa6BtWe4.Fdf5.vCof7amUbcx_j Kl3Y2_C8ygCVqWbjoFgDTo04lgToF3OTphWybbs9FN2D3BmGkh_7GcYvneZBVRYGq2hfnSbiwjR6 6AyD2kXwOx0QJzyu95AGAff_HdT4TSVZBJXE.hMiffVsDbvkqG2jzGPx9axRXcPE3wfSk8hH4xcq d8SkoabT8EMN7DjUzuu08U7AinO2Xk3mqkx9iemN8bun9_I8DlOPHH7rO4eHLnL2aAn6ibQCfgiq mH.QvwQ2u0C3xCdYCUlG9F5qWtdTp6lLjGeoLMtt6a189pcoAuQyH6gX.H5bm74n5OPoGibdU5Hr Vea_V2qbDdWXSWUV37nx87bK0bdAlcaSq9RO5yT0CGd74IODXPVOp8ONr46IzuxmkkySkQukxhnl CHoF2q37NfQEuYdHPZbnpTi1gza1k74JQQR4BNetLFP5risO2wU66vcJPtw3PiAV_RU3XNQ45L1I jBgq1amvlwGKXwKZ1Um5CVqHlbrnYhWF7tJZ7JkZDdCgtbESNcs2jsLUXOeSk_BoZ2L41qA0FdX4 MXbrcF.kt48t9T_W4QoUuGotgDEZgbGzyLWpWbpBcHBn8RD53gLiTspWImiA7f7xlLVfXF3Jk2JT vA07_fWntxP_GjI34oNL1w0PYJyjXVgBRyKZB1PIJysT5yKZQOca5iMs9eV_30nXq6Za3KUd29jD 4MiKuJq3V6_RiMEnrdpaS8sNAX.qPA86TVzvXbfj6uhPwX3uEw.L2mDiUe1lMhlZkQKsljK8Bv62 DgsTANWpXzHwq93bA7CfEK1BYBcFC.PGLAw828wq0g0Td0U0WhtxvevjNwtuax8otecLi.731IxL 0QqssMXE6t5GzfA.08nRFNjkviwMQ8ELL6I6dgAoxQAJokdmhUuLlrc8UNJBZTZTULvNkvR.Ur6X DG8aeAjvcOpsewrGQLsYqPBUkBkXXpMxNravlFAyUqV4.x8rCxEfK6fH9jB01gh1JCBovcOgviP4 sZZ8rOMKfa8SDEsAgkvIQINehKNnTzAUcgueZJkbgIYjhD5.1dxrNx05g8FO_l9Y0HGjwlgNkIYS ZM.DkSDG_w.89MBewxKBjDi4u7BfdNxf2qH8I8ZeYK.ViM9wTRnzt5YZrjTlJ1EqiI3T10JIw_p4 dPtY- X-Sonic-MF: X-Sonic-ID: 50a11fe7-efc8-402e-8e3e-ddd1a14a129a Received: from sonic.gate.mail.ne1.yahoo.com by sonic305.consmr.mail.gq1.yahoo.com with HTTP; Wed, 19 Apr 2023 02:10:35 +0000 Received: by hermes--production-gq1-546798879c-dcj2l (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 1a8973be78a9832e44f0774a888b203d; Wed, 19 Apr 2023 02:10:31 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: The still-pending openzfs updates status upstream, as far as I can tell Message-Id: Date: Tue, 18 Apr 2023 19:10:20 -0700 Cc: Mateusz Guzik , Pawel Jakub Dawidek , Martin Matuska To: Current FreeBSD X-Mailer: Apple Mail (2.3731.400.51.1.1) References: X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.998]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ARC_NA(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.84:from]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.64.84:from] X-Rspamd-Queue-Id: 4Q1PQP4HQvz3GjV X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N Some existing changes in FreeBSD need not be appropriate to openzfs but it is more likely that many of the same files would eventually have some sort of change in openzfs's main dealing with what FreeBSD has run into from the import. Looking around, the following are the FreeBSD adjustments in place in files that did not seem to be adjusted yet in openzfs's main. A few are just tracking changes in FreeBSD's main that are not tied to the import. I added a few notes in []'s. The 3 [TEMPORARY]'s may well never have related openzfs source code, for example. The other changes did seem to have code updates intended to deal with problems associated with the import. No claim to know how well the changes work when they do not match what my prior testing dealt with. include/os/freebsd/spl/sys/simd_arm.h (not updated in openzfs master): d6e24901349d zfs: disable kernel fpu usage on arm and aarc64 Mateusz = Guzik [ARM PART] include/os/freebsd/zfs/sys/zfs_context_os.h (not updated in openzfs = master): 8e9db62e7423 zfs: Appease set by unused warnings for spl_fstrans_*mark = stubs. John Baldwin [FREEBSD TRACKING] include/os/freebsd/zfs/sys/zfs_vfsops_os.h (not updated in openzfs = master): 068913e4ba3d zfs: Add vfs.zfs.bclone_enabled sysctl. Pawel Jakub Dawidek = [TEMPORARY] module/os/freebsd/zfs/zfs_ctldir.c (not updated in openzfs master): e2d997d1cbb9 zfs: add missing vop_fplookup_vexec assignments Mateusz = Guzik module/os/freebsd/zfs/zfs_vfsops.c (not updated in openzfs master): 068913e4ba3d zfs: Add vfs.zfs.bclone_enabled sysctl. Pawel Jakub Dawidek = [TEMPORARY] module/os/freebsd/zfs/zfs_vnops_os.c (not updated in openzfs master): eb1feadc201a zfs: fix null ap->a_fsizetd NULL pointer derefernce Martin = Matuska d012836fb616 zfs: fix up EXDEV handling for clone_range Mateusz Guzik 20be1b4fc4b7 zfs: try to fallback early if can't do optimized copy = Mateusz Guzik [OPTIMIZATION?] 182b21d46276 openzfs: adopt to the new vn_lock_pair() interface = Konstantin Belousov [FREEBSD TRACKING] 46ac8f2e7d96 zfs: don't use zfs_freebsd_copy_file_range Mateusz Guzik 068913e4ba3d zfs: Add vfs.zfs.bclone_enabled sysctl. Pawel Jakub Dawidek = [TEMPORARY] Warner's stand (loader) work caused by the import might be associated with more openzfs changes at some point, allowing for easily/directly avoiding use of "extra register sets" in the various boot loaders. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Apr 19 02:31:42 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1Ptw1Gw7z44p9S for ; Wed, 19 Apr 2023 02:31:52 +0000 (UTC) (envelope-from zlei@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1Ptw0rb5z47x2; Wed, 19 Apr 2023 02:31:52 +0000 (UTC) (envelope-from zlei@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681871512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0g5dyRpVZYEBPED/JFrHltyQ2AcxdhJai3Qluxar0HM=; b=aOdzIBxxf74laM8VTlnDObp1+E9yL+vGgBjG6hj1gIxT1PCHOPLIcjLyng/6z/dIwc4KBP H9MedA5hP/rnuX763tjXoi/sFoPjOWd+/G6VXapvHF2wPNqWxsilZ3wqHdIBCg67gAOxN9 xbSa4QlwismaD8wF/+H1ftwD088ElXO6T9gpn1Mfi/eGW59WOIpnW7VHJHhb1B8LRDkatw jVLixLPjUbzfzhrU6b76ByEgMgQiw/ob5gEkrTcfgB5wK20lISVcOxMUMPqRwT0LFDhOE2 33lxrm4jyptOmV7xRmSiIfOTjBXG9Di/qzt24LFM9HKESngpDxOIfpAJCfgIEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681871512; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0g5dyRpVZYEBPED/JFrHltyQ2AcxdhJai3Qluxar0HM=; b=pPJYViJTDpLLiJ5AznCMhIr0NWc18e5TSFe35sq43eqJAbSUv7Ed8a4vC1eAilWSKPZjjS QJ2XsVGI/k334nIVNZfvy+dmIgqBT1Uqp/jsdlDBfXiZ25Ma/usiK074KulH7ijGL7YMFf j24kb12rYE292dxnFXVi1ucmqvAXjZ6RZ4sfQPTn/TbgCVLEZa307N6odmfggyUjF3S8aK zXBt16x5ljWbJEmqIfnULAdMUkcVBIYVPlmuwFkimMiQnrTaHynMTYj14JMoBOr+5yK9Ce YCx56To/Y5NM+KrYktw+iIl0femdIUXlPZSy+DlboNYktWEQHMVHLn7KTM4dQw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681871512; a=rsa-sha256; cv=none; b=uaNAllyiBBYr6V37H5yHvS0qGBWfS4853IQRJW11L1q2TYeZKa09YTlTolOFX/QhAKIUey +1p3weHk/SrX9tkYfAU+qw1uREsez3oY+m9t+XMNMxrEKu9FD6ub2a3R4OjfiRALlTBBzl pNpM6YJxpZOTKIRBKDGYeFWyCTT/CzVr3lqlRyku4ASXVJDBf+I8Z52ST0nB63XScehEE1 dhP4Y957518EZJhiPHch3Hw0GAqzxgOjnXPW9yfjuBL63x02YNmYnNOJsf67ThZCLSWxUo sUAOzDbXu7x12mBlOaw53vQiwn/jgfqN9b1rs6fMLrLfXmllwFkysrTl1E5MXg== Received: from smtpclient.apple (unknown [IPv6:2001:19f0:6001:9db:98f0:9fe0:3545:10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) (Authenticated sender: zlei/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q1Ptt57QHz1KMk; Wed, 19 Apr 2023 02:31:50 +0000 (UTC) (envelope-from zlei@FreeBSD.org) From: Zhenlei Huang Message-Id: <2CFA5EBD-ABEB-4D22-9D9A-50725168112E@FreeBSD.org> Content-Type: multipart/alternative; boundary="Apple-Mail=_B7DDE398-2F32-4F00-BCD5-DEB9D957E844" List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.2\)) Subject: Re: Is it valid to combine CTLFLAG_TUN with CTLFLAG_VNET ? Date: Wed, 19 Apr 2023 10:31:42 +0800 In-Reply-To: <263045d4-409a-8a2d-87e1-50b1afcb7338@selasky.org> Cc: freebsd-current@freebsd.org To: Hans Petter Selasky , Gleb Smirnoff References: <94C1B333-9C0F-4874-BBB1-3E72F3DF3F6A@FreeBSD.org> <9dc65578-9312-1139-932f-396bc42e66b2@selasky.org> <263045d4-409a-8a2d-87e1-50b1afcb7338@selasky.org> X-Mailer: Apple Mail (2.3696.120.41.1.2) X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_B7DDE398-2F32-4F00-BCD5-DEB9D957E844 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Apr 6, 2023, at 3:56 AM, Hans Petter Selasky = wrote: >=20 > On 4/5/23 21:44, Hans Petter Selasky wrote: >> On 4/5/23 20:23, Gleb Smirnoff wrote: >>> What if we remove the CTLFLAG_VNET check from the code you posted = above? >>> I don't see anything going wrong, rather going right =F0=9F=98=84 >>>=20 >>> CTLFLAG_VNET will not mask away CTLFLAG_TUN. >> Hi Gleb, >> It's possible to bypass that check, but some work needs to be done = first. Then all jails created, will also start from those sysctl tunable = values. >> The problem is, where does the VNET base pointer come from? >> Especially those static sysctl's. You would need to make some design = there I guess and look at the SYSINIT() order. When are SYSINIT's filled = with tunable data's. And when is the default VNET created. >> Because the data pointer passed to the register sysctl function is = simply an offset pointer into a malloc'ed structure. >> --HPS >=20 > Hi Zhenlei, >=20 > Feel free to work on this, and add me as a reviewer and complete phase = two of: >=20 >> commit 3da1cf1e88f8448bb10c5f778ab56ff65c7a6938 >> Author: Hans Petter Selasky >> Date: Fri Jun 27 16:33:43 2014 +0000 >> Extend the meaning of the CTLFLAG_TUN flag to automatically check = if >> there is an environment variable which shall initialize the SYSCTL >> during early boot. This works for all SYSCTL types both statically = and >> dynamically created ones, except for the SYSCTL NODE type and = SYSCTLs >> which belong to VNETs. A new flag, CTLFLAG_NOFETCH, has been added = to >=20 > --HPS Posted to https://reviews.freebsd.org/D39638 = =20 CC freebsd-current if some people are interested in the fix. Best regards, Zhenlei --Apple-Mail=_B7DDE398-2F32-4F00-BCD5-DEB9D957E844 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Apr 6, 2023, at 3:56 AM, Hans Petter Selasky <hps@selasky.org> = wrote:

On 4/5/23 21:44, Hans Petter Selasky wrote:
On 4/5/23 20:23, Gleb = Smirnoff wrote:
What = if we remove the CTLFLAG_VNET check from the code you posted above?
I don't see anything going wrong, rather going right =F0=9F=98=84=

CTLFLAG_VNET will not mask away = CTLFLAG_TUN.
Hi Gleb,
It's = possible to bypass that check, but some work needs to be done first. = Then all jails created, will also start from those sysctl tunable = values.
The problem is, where does the VNET base pointer = come from?
Especially those static sysctl's. You would = need to make some design there I guess and look at the SYSINIT() order. = When are SYSINIT's filled with tunable data's. And when is the default = VNET created.
Because the data pointer passed to the = register sysctl function is simply an offset pointer into a malloc'ed = structure.
--HPS

Hi Zhenlei,

Feel free to work on = this, and add me as a reviewer and complete phase two of:

commit = 3da1cf1e88f8448bb10c5f778ab56ff65c7a6938
Author: Hans = Petter Selasky <hselasky@FreeBSD.org>
Date: =   Fri Jun 27 16:33:43 2014 +0000
=    Extend the meaning of the CTLFLAG_TUN flag to = automatically check if
   there is an = environment variable which shall initialize the SYSCTL
=    during early boot. This works for all SYSCTL types = both statically and
   dynamically created = ones, except for the SYSCTL NODE type and SYSCTLs
=    which belong to VNETs. A new flag, CTLFLAG_NOFETCH, = has been added to

--HPS

Posted = to https://reviews.freebsd.org/D39638 
CC freebsd-current if = some people are interested in the fix.

Best regards,
Zhenlei

= --Apple-Mail=_B7DDE398-2F32-4F00-BCD5-DEB9D957E844-- From nobody Wed Apr 19 06:55:17 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1WlH106xz454fr for ; Wed, 19 Apr 2023 06:55:39 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-23.consmr.mail.gq1.yahoo.com (sonic303-23.consmr.mail.gq1.yahoo.com [98.137.64.204]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1WlF6Z5Bz3r4H for ; Wed, 19 Apr 2023 06:55:37 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=uGB6bf1o; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.204 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681887335; bh=JmxLNP76HVh/TPX47yRWvnTYrTTGWOyHdIYWqt7TOZA=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=uGB6bf1ouzyPu8k+3AJ9ZmeLRrweezsPjlxyS7XPUCxWIH5Owh8YJ4My4K8GE458sbsbKSwQ+k91j/k5avIKzD1R02TduK8o1r6r/RUknnw8x2Pg8cRBvfzDX1FToKV6E64hPnkV9Ixw7CMtWlHRRoNDVPNWyEx2bsASLfqhWjGzISTLPuYq7QXxrqw+jBzpFuKuizxZnx7Q7QD0lVEwHQD0B3b/p6NWxkgV2sa7Cae9PuK0cOzDgR/YRpmXMBdpVVsZxNSGxU+tyL17fKAeZE3Z+g6K2KOk5YdHNweBkfamEAwJziozgLEfuZaTtmP6KT2gp3VJuqEE6kseqE1OgQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1681887335; bh=YsWMP3wgbLlmIYFAkBRBh3lpeN9Sel6s3mqQWFhw88T=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=csiLYK3tNx/EVZItjgXF5A0rPT9Y8M011rvhNTnpyJBERgRA6KztRYscrugXtLAA0BANtTv4zlvyPvmf/7p9SBupQ5e/q+hog66O1xi6PkPcOBp1Z0lH1LpHc0AKiyoK4sEVwicB+roQU9n5k9KFdrRgVHRPk2hk9AG26HctJlF37W7bvH1/6zbf5aRGEMPZQx4WY+P8jR0sc2bUxGy0gvwrMhWRsfnI43MkurK2lGS+beCAHKRV9Kbtp0BhuazJqMuj5sJDuLWQnXAaqvdGAILbo3e+/XY1nhXOywJB2KBOTtb1Li258xrItSmAvA+0fC2PYfzOOj6VJUH92h10wg== X-YMail-OSG: 25CPdC0VM1neVVUvu5b0cfUDDvN7PjVbIadjljHSRW5CcK_7HXa7ZIwsvlzyVSa GnhMGvHwMg.NsRFjWWzJUYpON1Q0CcobLoegIESJyJWxQy1IeO2WHW0ksl3S12zrb.fnEo72CYcm UpQu684Gdp3r5xOWR7FU1_B4DKLIKloCHWColUqgUExISCuqHIBKJljaDtVKJ6N_h3jYpicfouQv SSQmjCkhXm5PCCa9r3NKfScfr8Q7hTLWcuWUKtiFp24wGM7un6E9X8fQ9EMKcLl3JbIEQYBq3ik1 33eT5VveGwxAMfvTRwOSNCvdIeF7.lTnoJCs6T0GWkWfEUY9RMBFXF_i80efI_N3CxWrBquxzOkR d3.YqpBDPq7hEKkH5RLh0eu4S.EPjE8nUmrLB610ol.2FmXxNzesnm.l92gWEG6EGt2XkDQP7q1I ippe5dBFGqNHkHkCFbXiViqdmg.29KSgErWuEhaNcbRXBKq3_17Hill9OFTVzBc0bXrIMfqQjizm xipWN9jbkQbulN.0aWil02cL5gm3LKb.rDg2yf1F9M1i5_19D_3hVZcTR7.o5hzlePB1GsaTV59Y 63SabBBPPpapXh_JhuwGlz4kNUXjOaipYiYTH.mY5UHCOseJ6hxaeiJkGc3VJ.u1YaMJmPytUzhL qWHmZvJI.V0aznw9gaqLi4OfeeOHHINyoH4yGdUwlLkWUsm9gWtltDcVzDaYQbRZu8xm1MjHG_Z3 BYw28ybsA6OPuEBBFh.BYR3odOIojlhdwZRevYim1DdlfER0PizEERWfpozfVk.S6WgEd5vrNH95 .yGWCeJSZPZVFYRYbtJGC8LIl6eEoI_.bRniBsjWZ18y7muVW4Zp1EVFoG7sZoOjwTLrnHp5EIl2 4cxioUUBQNuNa9BrREhXOe6SeJvjBqU7ivXyVZql1y4X67txQ3bxHroZOnuE.ATSpcX0EAbgiGYS wB368JLTLHGfaB7oI4C2ktCbYR2gk2pyl3XewvCimj2FCTSf7gM_h92oqkyoo6.6S5_d1BAmBPXI 7w.jM_MDIQ21AiVvQnwBQ7krMtt8YP9bPXu.bmO5_Y2XpUCrqyTTz3wlcjUaCPOASeJyjpJhcRZ4 mE1JmFjstV3CerzHno5ui5LwdGiH98k022mzK9daOAakIDuTS_waCyQM_CjOABdc.lx6Amn1Upnq D2brgoEQlxLKxlOL2NtFnybxVa9aP98H9pv_H9RUWLo2kJLl_LUDbEVdcA1MWj3VDZx3jdKcJKPm AUpkqjZuygrlNyyZj6mhudBf0yDGWcIpc9HO2XnfSc0PJYaCy6r8BTNzTJe2Atox8ODmqDy5p19I q3ajPiIKDgp6ldPM16WbXk56fl_6dGkxH_e7DkWjT39G6VW0_9KD173f5iGayBnhhodDDSz7dbnq qeIK2I1pJneEJU92CBL3LMdrxGag0hB1POow4Rk2msDAUnoOaXi5Bw0bkFFS4c0RdDP8pWoM6M1p NIVirmTJ4cWU9ipX8CNUks985p3fQiHgW57VpnqzjzGixSBKczxSpzej._syb87TSaqiI3YL8JIv tTrAx4xuRxh.DQxnwa3XvEXIBCHbkZQXuNg2IZOEXK.WY8UgxhCjSsmvits.5BZkTD13WUWfUX.1 9KufNzYrdwShxupDl5pq.YO2M9VBrD7qW8uwBPn.6mvj7TW9B.A01125xUhVuSC2LLax8pDBj68f 5KNeY97ADhcl_O4ItS.J_Z1CRcgtNhntRg_3WgWW_R_mfBrJ.E5ETU6qc5hAR7wW3VN8ltIhzbQ0 kBrZ6GBGa7A_CQfbCv6S7SgACoZS1jLRRi1stjKyRJzlToRaDJ_vxqhbFf64wSC6evPQfA0Hvxg2 cOBgvlF4cbOEMLyhdHaDasrPYq79Hu9BSgNqxQf1eaadAyvu7xh1N3z9AouWUp341hSeFubLGeKd kWF_vcQvaQ2lWvSM6bsbnpS2LWmOz7MsF7jiNxLj10vXiQ1CQNHoDx4MCu3WiIBK6wxgQP0HFyUh 85LCHAlDGfwVcWrEQ8ZmJsNmy9UqeYgxH_qfhf9YfoghxoQFOW2rbeX4M4rlXzaD6HFNafQJhri7 j2APkZr9QMvdcR85T7Eial7n8KoHijzkiA6lkro.pG0.DtBg.RQKYEw5O42ZAlbLUtlUpSv4N1cD P6M9ZrAYnEjG.UqreK.qTlftADmKkkdd7lN6WY3C2pCSiSz0dgfJl1Q3P8Xqd3Eg.XfbqAK57z0T vu4nmajDFNm02iyI3Uy_arRKid0GfT4SPclC8VQQ1AU6y7yZd0AFrLwFvqItaD29j6rj6SSo3sQF rCZ4eXLhlWlk- X-Sonic-MF: X-Sonic-ID: 9ade6a74-6987-42f0-8a4e-28d6cfc8ac2d Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.gq1.yahoo.com with HTTP; Wed, 19 Apr 2023 06:55:35 +0000 Received: by hermes--production-bf1-5f9df5c5c4-p5s6l (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID a489cffba91d9a07a287463556104c64; Wed, 19 Apr 2023 06:55:30 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: The import of openzfs vs. armv7: boot crashs From: Mark Millard In-Reply-To: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> Date: Tue, 18 Apr 2023 23:55:17 -0700 Cc: "mjg@freebsd.org" , "pjd@freebsd.org" , Kyle Evans Content-Transfer-Encoding: quoted-printable Message-Id: References: <6CB8D120-1600-40E6-8A1E-87E709DCEC8F@yahoo.com> To: Current FreeBSD , freebsd-arm X-Mailer: Apple Mail (2.3731.400.51.1.1) X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.204:from]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCPT_COUNT_FIVE(0.00)[5]; DKIM_TRACE(0.00)[yahoo.com:+]; RCVD_COUNT_THREE(0.00)[3]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4Q1WlF6Z5Bz3r4H X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N On Apr 18, 2023, at 15:44, Mark Millard wrote: > = https://github.com/openzfs/zfs/commit/d0cbd9feaf5b82130f2e679256c71e0c7413= aae9 >=20 > does not seem to cover armv7, just aarch64. (FreeBSD disabled > floating point for both armv7 and aarch64 but that is a > different change than above.) >=20 > I used: >=20 > = FreeBSD-14.0-CURRENT-arm-armv7-GENERICSD-20230406-f21faa67ab6b-262010.img.= xz >=20 > booted an RPi2B v1.1 and tried (note the KSTACK_PAGES notice and the > "undefined floating point instruction" notice): >=20 > # zpool import > ZFS NOTICE: KSTACK_PAGES is 2 which could result in stack overflow = panic! > Please consider adding 'options KSTACK_PAGES=3D4' to your kernel = config > panic: undefined floating point instruction in supervisor mode > cpuid =3D 2 > time =3D 1680784610 > KDB: stack backtrace: > db_trace_self() at db_trace_self > pc =3D 0xc05eb154 lr =3D 0xc007a688 = (db_trace_self_wrapper+0x30) > sp =3D 0xdd25c480 fp =3D 0xdd25c598 > db_trace_self_wrapper() at db_trace_self_wrapper+0x30 > pc =3D 0xc007a688 lr =3D 0xc02eb1b4 (vpanic+0x140) > sp =3D 0xdd25c5a0 fp =3D 0xdd25c5c0 > r4 =3D 0x00000100 r5 =3D 0x00000000 > r6 =3D 0xc0736bfc r7 =3D 0xc0b1aea8 > vpanic() at vpanic+0x140 > pc =3D 0xc02eb1b4 lr =3D 0xc02eaf94 (doadump) > sp =3D 0xdd25c5c8 fp =3D 0xdd25c5cc > r4 =3D 0xc0b92210 r5 =3D 0x00000000 > r6 =3D 0xc0610ca0 r7 =3D 0xf4210a0d > r8 =3D 0xddf32e4c r9 =3D 0x00000013 > r10 =3D 0xdd25c6c0 > doadump() at doadump > pc =3D 0xc02eaf94 lr =3D 0xc0610eb0 (vfp_new_thread) > sp =3D 0xdd25c5d4 fp =3D 0xdd25c638 > r4 =3D 0xdd25c6c0 r5 =3D 0xdd25c5cc > r6 =3D 0xc02eaf94 r10 =3D 0xdd25c5d4 > vfp_new_thread() at vfp_new_thread > pc =3D 0xc0610eb0 lr =3D 0xc060ff84 = (undefinedinstruction+0x178) > sp =3D 0xdd25c640 fp =3D 0xdd25c6b8 > undefinedinstruction() at undefinedinstruction+0x178 > pc =3D 0xc060ff84 lr =3D 0xc05edaa8 (exception_exit) > sp =3D 0xdd25c6c0 fp =3D 0xdd25c750 > r4 =3D 0x20000013 r5 =3D 0xde45e000 > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > r8 =3D 0x00000000 r9 =3D 0x00000000 > r10 =3D 0xdd25c8c0 > exception_exit() at exception_exit > pc =3D 0xc05edaa8 lr =3D 0xddf31f20 (K256) > sp =3D 0xdd25c750 fp =3D 0xdd25c750 > r0 =3D 0xdd25c890 r1 =3D 0xde45e000 > r2 =3D 0xde45e400 r3 =3D 0xddf309fc > r4 =3D 0x00000400 r5 =3D 0xde45e000 > r6 =3D 0xdd25c890 r7 =3D 0xdd25c8b0 > r8 =3D 0x00000000 r9 =3D 0x00000000 > r10 =3D 0xdd25c8c0 r12 =3D 0xdd25c7a0 > zfs_sha256_block_neon() at zfs_sha256_block_neon+0x1c > pc =3D 0xddf32e4c lr =3D 0xc0946e8c (pcpup) > sp =3D 0xdd25c758 fp =3D 0xc0b0aeec > r4 =3D 0xc0919610 r5 =3D 0xc0919630 > r6 =3D 0xc0919618 r7 =3D 0x642ebce2 > r8 =3D 0xc0b1b0ec r9 =3D 0xc0915e88 > r10 =3D 0xc0b1b0dc > Fatal kernel mode data abort: 'Translation Fault (L1)' on read > trapframe: 0xdd25c330 > FSR=3D00000005, FAR=3D95e29398, spsr=3D200000d3 > r0 =3Ddd25c424, r1 =3D81000000, r2 =3D95e29395, r3 =3D55555555 > r4 =3Dc08ae93c, r5 =3D00004aa0, r6 =3D00004aa0, r7 =3Dc08d3e3c > r8 =3D00000001, r9 =3Dc079567a, r10=3D0000000b, r11=3Ddd25c3e0 > r12=3D00000000, ssp=3Ddd25c3c4, slr=3D00000001, pc =3Dc0610308 >=20 > panic: Fatal abort > . . . (repeats over and over) . . . I probably need to explain the subject line's reference to boot crashes, given the specific crash that I show is not that. I reported a simpler context than booting ZFS media: boot a UFS snapshot then force ZFS to start with no ZFS pools even being present. It is the same simplified type of test that I previously had reported for aarch64 as a simpler/easier test to set up than an actual root on ZFS crash that I had originally gotten for aarch64. The test I present should imply that a boot of, say, a root on ZFS context would crash. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Wed Apr 19 14:26:22 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1jlM1TJ7z462jg; Wed, 19 Apr 2023 14:26:23 +0000 (UTC) (envelope-from salvadore@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [96.47.72.132]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "freefall.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1jlM103jz3hMy; Wed, 19 Apr 2023 14:26:23 +0000 (UTC) (envelope-from salvadore@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681914383; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=JV2s/JkH9Ish3E3BvLWHck5+PWnbW5z5nZBTxT3Yg44=; b=cE9HjZnPWVcozrYikyaTxr/tkMktZlMAkbOeYFccJa2lqfU3ztd6u2a8Jww3f61RYyiHAZ nr0qGizfpSSAz+6qKzynJKjMmf1XM7NV4m0ygMtcHQDZyiR+tuSMqRv/zmco1iIdLqNRAy ACTdKCyRiAPh2xBbCS+aT9mNUaED+tAVF1fe8VmmKaqGTzOaldnCf8n7Nfw44GOcL2E9sh toAT81psV8YUzp/a3y8PyIk9n/9HDT+yNe40+C7d2KCPvAYypB4PEEYg670Es9mXIC3VsG P8oP9JFP4XOEOxK+Y8fqsD69nixQonHv7o+wQKKTuKiGqRNekJ1VdKwbwiyVCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681914383; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=JV2s/JkH9Ish3E3BvLWHck5+PWnbW5z5nZBTxT3Yg44=; b=GdXfCzDoyBO7SjCB9GTW8peifnNrUze/y+Q54naVn/RFSfyvaNK7Qo0RYG40awQtHNr81E TKWcJx3TQybWk7lUgiJT/iy6kh6ExSp3NyAd16ag92PM9fSZYxtX5SnJRppOMS72v9fBH9 Y/1ZiYm902M+HRY1MRNZASDuPgVJb1qlcBD8qXhVVUZZUy2iChJI+fsVIIR2UgQRBGu1Ji dVk7LA3VZmhT/TZ6xVsYtS4FvzyrHqffgWWTSn4TkccqQ/WfOGbwr5m0uQTKaE1tvGqmvM l3r46fQ+WBXJXVFHdFgaiuKWIleJGi7fT/+29tjNtkwEcVhbmxrsu2u/otkTxw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681914383; a=rsa-sha256; cv=none; b=dzwrWULQcAFe4d/PztAL/PbFnTuDJTQoVnk2ghWyYaHcry2Ql1sVS7b1mqGp40X7KZtV8q gcj73ZwVW75zVqc+hOQEtQfzjFyP23IREVtSxyFfijnHuBuHqyG46e14TqloRoQAT5OFxR yEEx+eqwSgFrlE2rZ2B8Ug5pEBQ2AR3609KNHdXqF3p2fymR/8JuzgLqpFXJnjYhJEGFx6 6kRa++Xt4B291oOwglRoPpNCH0IpTh/D5vnzTXC9AnxwrLKtNNFr5SO2NZBXLA+jY03U/u frgyopgnblugOoDcvZfQiF3hvazscxmrj5/o9K6WM5pkRVJpVz9VCoCkh3FwUA== Received: by freefall.freebsd.org (Postfix, from userid 1472) id 05AA91E88B; Wed, 19 Apr 2023 14:26:23 +0000 (UTC) Date: Wed, 19 Apr 2023 14:26:22 +0000 From: Lorenzo Salvadore To: freebsd-hackers@freebsd.org Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: FreeBSD Status Report - First Quarter 2023 Message-ID: List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline; filename="report.txt" Content-Transfer-Encoding: 8bit X-ThisMailContainsUnwantedMimeParts: N FreeBSD Status Report First Quarter 2023 Here is the first status report of 2023, including 25 reports: we have our usual team reports, some news about cloud projects, progress in the src, ports and doc trees and more. We also provide some information about 13.2-RELEASE, which was postponed to the beginning of 2023Q2; but since this report is being published after the new version release, it is already available for installation. Users of RELEASE versions can now take advantage of many improvements such as better support for iwlwifi(4) driver or the new rtw88(4) driver, topics that have been covered in past status reports. Have a nice read. Lorenzo Salvadore, on behalf of the status team. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ A rendered version of this report is available here: https://www.freebsd.org/status/report-2023-01-2023-03/ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Table of Contents • FreeBSD Team Reports □ FreeBSD Core Team □ FreeBSD Foundation □ FreeBSD Release Engineering Team □ Cluster Administration Team □ Continuous Integration □ Ports Collection □ Status Team • Userland □ daemon(8) improvements • Kernel □ Enabling Snapshots on Filesystems Using Journaled Soft Updates in 13.2 □ Improve the kinst DTrace provider □ Native Linux timerfd • Architectures □ Kernel Address Sanitizer on AArch64 □ bsd-user: Upstreaming and Status Report • Cloud □ FreeBSD as a Tier 1 cloud-init Platform □ OpenStack on FreeBSD • Documentation □ Documentation Engineering Team □ The FreeBSD Russian Documentation Project • Ports □ Freshports: SQL Injection Attack and Help Request □ DRM drivers (i.e. GPU drivers) □ KDE on FreeBSD □ FSX □ GCC on FreeBSD □ Valgrind - Preparing for Valgrind 3.21 • Third Party Projects □ PkgBase.live □ Containers and FreeBSD: Pot, Potluck and Potman ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Team Reports Entries from the various official and semi-official teams, as found in the Administration Page. FreeBSD Core Team Contact: FreeBSD Core Team The FreeBSD Core Team is the governing body of FreeBSD. Items Core Team Charter: Draft At the first Core Team meeting of 2023, Team delegates from the December 2022 meeting in Boulder, US presented the delegation’s conclusions to the entire Team. The Team will continue to discuss the issues and work together with the FreeBSD Foundation. FreeBSD annual developers survey The Core Team together with the FreeBSD Foundation have decided that the FreeBSD Foundation will be in charge of conducting the annual developers survey. Matrix IM solution The Core Team continues to evaluate Matrix as an IM solution for FreeBSD developers. An instance has already been prepared and tests are underway. Commit bits • Core approved the src commit bit for Cheng Cui (cc@) • Core approved the restore of the src commit bit for Joseph Koshy (jkoshy@). ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Foundation Links: FreeBSD Foundation URL: https://www.freebsdfoundation.org Technology Roadmap URL: https://freebsdfoundation.org/blog/technology-roadmap/ Donate URL: https://www.freebsdfoundation.org/donate/ Foundation Partnership Program URL: https://freebsdfoundation.org/our-donors/ freebsd-foundation-partnership-program/ FreeBSD Journal URL: https://www.freebsdfoundation.org/journal/ Foundation News and Events URL: https://www.freebsdfoundation.org/news-and-events/ Contact: Deb Goodkin The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Donations from individuals and corporations are used to fund and manage software development projects, conferences, and developer summits. We also provide travel grants to FreeBSD contributors, purchase and support hardware to improve and maintain FreeBSD infrastructure, and provide resources to improve security, quality assurance, and release engineering efforts. We publish marketing material to promote, educate, and advocate for the FreeBSD Project, facilitate collaboration between commercial vendors and FreeBSD developers, and finally, represent the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity. Fundraising Efforts We finally have our 2022 fundraising numbers in and we raised a total of $1,231,096! We were short of our goal, which forced us to pull around $74,000 from our longer term investments. Besides receiving a lot of donations from you our users and contributors, we received larger donations from Juniper, Meta, Arm, Netflix, Beckhoff, Tarsnap, Modirum, Koum Family Foundation, and Stormshield. I’d like to extend a heartfelt thank you on behalf of the Foundation to everyone, including individuals and corporations, for your financial contributions in 2022! This year our budget is around $2,230,000, which includes increased spending towards FreeBSD advocacy and software development. More than half our budget is allocated towards work directly related to improving FreeBSD and keeping it secure. To fund the 2023 budget, we increased our fundraising goal and plan on using some of our investment money. When we received our first million dollar donation, the plan was to use up to 10% of it each year to increase our work to improve FreeBSD, so this has been part of our funding plan for a few years now. The 2023 budget is in the process of being approved by the board of directors and will be published once it is approved. This quarter we received donations from Juniper, Tarsnap, Microsoft, and Stormshield. So, we are already off to a great start! But, we definitely need more to support our planned efforts for 2023. If you want to help us continue our efforts, please consider making a donation towards our 2023 fundraising campaign! https://www.freebsdfoundation.org/donate/ We also have a Partnership Program for larger commercial donors. You can read about it at https://freebsdfoundation.org/our-donors/freebsd-foundation-partnership-program/. OS Improvements During the first quarter of 2023, 226 src, 39 ports, and 12 doc tree commits identified the Foundation as a sponsor. Some of this sponsored work is described in separate report entries: • Continuous Integration • Enabling Snapshots on Filesystems Using Journaled Soft Updates • FreeBSD as a Tier 1 cloud-init Platform • FreeBSD Release Engineering Team • Improve the kinst DTrace provider • OpenStack on FreeBSD Other Foundation-sponsored work included: • OpenSSH fixes and updates to versions 9.2p1 and 9.3p1 • a vendor import and update of libpcap to version 1.10.3 • improvements to tmpfs, msdosfs, and makefs • the addition of a new kqueue1 syscall • man page updates • dtrace and bhyve fixes • LinuxKPI work Continuous Integration and Quality Assurance The Foundation provides a full-time staff member and funds projects to improve continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project. You can read more about CI work in a dedicated report entry. A current project that is being funded by the FreeBSD Foundation is one to develop a set of scripts to help src developers conduct CI tests themselves. One of the main goals is to offer more visibility at the pre-commit stage. A review for the first milestone has been submitted. FreeBSD Advocacy and Education Much of our effort is dedicated to the FreeBSD Project advocacy. This may involve highlighting interesting FreeBSD work, producing literature and video tutorials, attending events, or giving presentations. The goal of the literature we produce is to teach people FreeBSD basics and help make their path to adoption or contribution easier. Other than attending and presenting at events, we encourage and help community members run their own FreeBSD events, give presentations, or staff FreeBSD tables. The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, working together on projects, and facilitating collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. We are back to attending events mostly in person and began planning the in person May 2023 Developer Summit, co-located with BSDCan. In addition to attending and planning events, we are continually working on new training initiatives and updating our selection of how-to guides to facilitate getting more folks to try out FreeBSD. Check out some of our advocacy and education work: • Hosted a stand at FOSDEM 2023, February 4-5, 2023 in Brussels, Belgium. Check out the trip report. • Hosted a table at State of Open Con 2023, February, 7-8, 2023, in London, England. Read more about it. • Sponsored, held a workshop and hosted a booth at SCALE 20x, in March 9-12, 2023, Pasadena, California. Check out the trip report. • Sponsored Open Source 101, March 23 2023, in Charlotte, NC. • Sponsored and began planning the in-person May 2023 Developer Summit taking place May 17-18, 2023 in Ottawa, Ontario • Secured our Media Partner sponsorship status and submitted a workshop for All Things Open, October 15-17, 2023 in Raleigh, NC. • Submitted a Workshop proposal for FOSSY, July 13-16, 2023, in Portland, OR. • The FreeBSD Project was accepted as a Participating Organization for Google Summer of Code. • We held GSoC Office Hours to help prospective participants with questions. • Published March Newsletter • Additional Blog Posts □ Under the Hood with FreeBSD and Ampere Altra □ New Open Position: FreeBSD Userland Software Developer - Note: Posting is closed. □ BSDCan 2023 Travel Grant Application Now Open - Note: Applications are closed • FreeBSD in the News: □ VMBlog State of Open Con Q&A with Deb Goodkin We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.freebsdfoundation.org/journal/. You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/. Legal/FreeBSD IP The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise. Go to https://www.freebsdfoundation.org to find more about how we support FreeBSD and how we can help you! ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FreeBSD Release Engineering Team Links: FreeBSD 13.2-RELEASE schedule URL: https://www.freebsd.org/releases/13.2R/schedule/ FreeBSD 14.0-RELEASE schedule URL: https://www.freebsd.org/releases/14.0R/schedule/ FreeBSD releases URL: https://download.freebsd.org/releases/ISO-IMAGES/ FreeBSD development snapshots URL: https://download.freebsd.org/snapshots/ISO-IMAGES/ Contact: FreeBSD Release Engineering Team, The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things. During the first quarter of 2023, the Release Engineering Team started work on the upcoming 13.2-RELEASE. As of this writing, the 13.2 cycle has followed the originally set schedule, with the addition of fourth, fifth and sixth RC builds, postponing the final release from the end of March to early April. The Release Engineering Team continued providing weekly development snapshot builds for the main, stable/13, and stable/12 branches. Sponsor: Rubicon Communications, LLC ("Netgate") Sponsor: Tarsnap Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Cluster Administration Team Links: Cluster Administration Team members URL: https://www.freebsd.org/administration/#t-clusteradm Contact: Cluster Administration Team FreeBSD Cluster Administration Team members are responsible for managing the machines the Project relies on to synchronise its distributed work and communications. In this quarter, the team has worked on the following: • Regular support for FreeBSD.org user accounts. • Regular disk and parts support (and replacement) for all physical hosts and mirrors. • Improve the PowerPC package builders. □ With new parts obtained through the FreeBSD Foundation, the builders now have new NVMEs with heatsinks and more memory. It helped seize the heat issue, and they are building packages faster now. • Decouple dynamic resources from the main websites. □ Work in coordination with doceng and webmaster to decouple dynamic resources from the websites, www.FreeBSD.org, and docs.FreeBSD.org. Work in progress • Large-scale network upgrade at our primary site. □ New Juniper switches arrived at our primary site to replace the former ones. We thank Juniper for the donation. • Replace old servers in our primary site and a few mirrors. □ Besides the broken CI servers, we have a few old servers with broken disks and faulty PSUs. This task is in conjunction with the FreeBSD Foundation and donors/sponsors. • Deploy infrastructure to mirror the websites. □ Since the FreeBSD website is now mostly static, we have begun deploying infrastructure to mirror www.FreeBSD.org and docs.FreeBSD.org around the world in FreeBSD project-managed mirrors. • Create a new search database engine for internal FreeBSD.org searching needs like mailing list and docs. FreeBSD Official Mirrors Overview Current locations are Australia, Brazil, Germany, Japan (two full mirror sites), Malaysia, South Africa, Taiwan, United Kingdom (full mirror site), United States of America (California, New Jersey [the primary site], and Washington). The hardware and network connection have been generously provided by: • Bytemark Hosting • Cloud and SDN Laboratory at BroadBand Tower, Inc • Department of Computer Science, National Yang Ming Chiao Tung University • Equinix • Internet Association of Australia • Internet Systems Consortium • INX-ZA • KDDI Web Communications Inc • Malaysian Research & Education Network • Metapeer • New York Internet • NIC.br • Your.Org. The Frankfurt single server mirror is the primary Europe mirror in bandwidth and usage. We are still looking for an additional full mirror site (five servers) in Europe to replace old servers in the United Kingdom full mirror site. We see a good pattern in having single mirrors in Internet Exchange Points worldwide (Australia, Brazil, and South Africa); if you know or work for some of them that could sponsor a single mirror server, please get in touch. United States (West Coast) and Europe (anywhere) are preferable places. See generic mirrored layout for full mirror site specs and tiny-mirror for a single mirror site. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Continuous Integration Links: FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org FreeBSD Jenkins wiki URL: https://wiki.FreeBSD.org/Jenkins Hosted CI wiki URL: https://wiki.FreeBSD.org/HostedCI 3rd Party Software CI URL: https://wiki.FreeBSD.org/3rdPartySoftwareCI Tickets related to freebsd-testing@ URL: https://bugs.freebsd.org/bugzilla/buglist.cgi?bug_status=open&email1=testing%40FreeBSD.org&emailassigned_to1=1&emailcc1=1&emailtype1=equals FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci dev-ci Mailing List URL: https://lists.FreeBSD.org/subscription/dev-ci Contact: Jenkins Admin Contact: Li-Wen Hsu Contact: freebsd-testing Mailing List Contact: IRC #freebsd-ci channel on EFNet In the first quarter of 2023, we worked with the project contributors and developers to address their testing requirements. Concurrently, we collaborated with external projects and companies to enhance their products by testing more on FreeBSD. Important completed tasks: • FreeBSD-main-aarch64-KASAN_test and its supporting jobs have been added. • FreeBSD-stable-13-amd64-KASAN_test and its supporting jobs have been added. • FreeBSD-main-amd64-gcc12_build now sends failing reports to the committers whose commits may be related. • Various fixes or workarounds to the tests of non-x86 architectures from trasz@ • Present Testing/CI Status Update in AsiaBSDCon 2023 Developer Summit Work in progress tasks: • Designing and implementing pre-commit CI building and testing (to support the workflow working group) • Designing and implementing use of CI cluster to build release artifacts as release engineering does • Simplifying CI/test environment setting up for contributors and developers • Setting up the CI stage environment and putting the experimental jobs on it • Organizing the scripts in freebsd-ci repository to prepare for merging to src repository • Improving the hardware test lab and adding more hardware for testing • Merge https://reviews.freebsd.org/D38815 • Merge https://reviews.freebsd.org/D36257 Open or queued tasks: • Collecting and sorting CI tasks and ideas • Setting up public network access for the VM guest running tests • Implementing use of bare-metal hardware to run test suites • Adding drm ports building tests against -CURRENT • Planning to run ztest tests • Helping more software get FreeBSD support in its CI pipeline (Wiki pages: 3rdPartySoftwareCI, HostedCI) • Working with hosted CI providers to have better FreeBSD support Please see freebsd-testing@ related tickets for more WIP information, and don’t hesitate to join the effort! Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Ports Collection Links: About FreeBSD Ports URL: https://www.FreeBSD.org/ports/ Contributing to Ports URL: https://docs.freebsd.org/en/articles/contributing/#ports-contributing FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/ Ports Management Team URL: https://www.freebsd.org/portmgr/ Ports Tarball URL: http://ftp.freebsd.org/pub/FreeBSD/ports/ports/ Contact: René Ladan Contact: FreeBSD Ports Management Team The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages (through its subsidiary pkgmgr), and personnel matters. Below is what happened in this quarter. Currently we have around 33,500 ports in the tree. For these ports, there are 3,021 open problem reports, of which 764 are unassigned. The first three months of this year saw 9,021 commits by 163 committers for the main branch and 701 commits by 55 committers for the 2023Q1 branch. Compared to 2022Q4, this means a slight increase in the number of ports, port PRs, ports commits, and active port committers. During this quarter, we welcomed Robert Clausecker (fuz@), Vladimir Druzenko (vvd@), Robert Nagy (rnagy@), welcomed back Norikatsu Shigemura (nork@), and said goodbye to Marius Strobl (marius@). Portgmr added Muhammad Moinur Rahman (bofh@) as a new member after a successful lurkership. During the bi-weekly portmgr meetings, the following topics were discussed: • improving the situation of binary packages for kernel modules • ways to measure the impact of ports on their dependencies and how to maintain high-impact ports. During this quarter, 32 exp-runs were run to test port updates, updating default versions (LLVM to 15, MySQL to 8.0, Ruby to 3.1), and updating byacc in base. Furthermore, the default version of Go switched to 1.20 and that of Lazarus to 2.2.6. Four new USES were introduced: • budgie to support ports related to the Budgie Desktop • ldap to provide support for OpenLDAP, with a new default version of 26 (i.e. 2.6) • nextcloud to support Nextcloud applications • ruby to provide support for Ruby ports (formerly bsd.ruby.mk). ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Status Team Links: FreeBSD status reports URL: https://www.freebsd.org/status/ FreeBSD Status Report Process URL: https://docs.freebsd.org/en/articles/freebsd-status-report-process/ Archived status reports GitHub repository URL: https://github.com/freebsd/freebsd-quarterly Contact: The new workflow has started In the first quarter of this year, the status team has started implementing the new workflow that was announced at the end of 2022. Here are some details. New email addresses Last quarter we have announced the creation of new email addresses: • , for interacting with the status team directly; • , a mailing list to which you can subscribe to get reminders about status report submission deadlines. Unfortunately, the mailing list does not work as expected at the moment. The issue has been reported but no solution could be found yet. However, a work around allowed to send the second and the last reminder to the list. Automation Some automation has been introduced to ensure that no report submission is lost: • on Phabricator a herald rule automatically blocks any review touching the status reports directory: even if a report submitter forgets to add the status team as reviewer, salvadore@ (member of the status team) will block the patch anyway. The same rule will also block any review that includes the status team as reviewer, to ensure that at least one member of status has reviewed the patch before commit. • a GitHub action automatically adds the newly introduced status report label to any pull request touching the status reports directory. The GitHub action should be easily modifiable by anyone wanting to apply more labels automatically depending on the path of the modified files. More automation is planned. Documentation reorganization The status report README and How To have been updated and merged in one unique document: the FreeBSD status report process. You can check it out to read details about reports submission and publication. It will be updated regularly as the status team proceeds with the implementation of its new workflow. In particular, new material about automation is coming soon. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Userland Changes affecting the base system and programs in it. daemon(8) improvements Links: daemon(8) URL: https://man.freebsd.org/daemon/8 Libera IRC URL: https://libera.chat/ Contact: Ihor Antonov Contact: Kyle Evans An ongoing effort to improve code quality and supervision capabilities of the daemon utility. Daemon is a tool that can daemonize (send to background) or supervise any running process, automatically restarting it if it crashes. Daemon is widely used in the ports tree and can be used more in base. This quarter long_opts support was added and the codebase went through an initial refactoring phase to prepare it for further changes. There are no functional changes so far but more changes are coming. Please contact directly or on #freebsd-dev on Libera IRC if you encounter unexpected bugs. Planned work items for the next quarter: • use of kqueue for all event sources • fix Bug #268580 • fix Bug #236117 • fix Bug #254511 • fix Bug #212829 • procctl PROC_REAP_ACQUIRE We are looking for feedback, bug reports (old and new) and feature requests. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Kernel Updates to kernel subsystems/features, driver support, filesystems, and more. Enabling Snapshots on Filesystems Using Journaled Soft Updates in 13.2 Contact: Marshall Kirk McKusick The ability to make UFS/FFS filesystem snapshots when running with journaled soft updates, and using them for doing background dumps on a live filesystem, was merged to releng/13.2 during the first quarter of 2023, and lands in FreeBSD 13.2-RELEASE. Background dumps are requested by using the -L flag to dump(8). The details of this project were described in the 2022 fourth quarter report. Sponsored by: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Improve the kinst DTrace provider Links: libdtrace: implement inline function tracing URL: https://reviews.freebsd.org/D38825 dtrace(1): add -d flag to dump D script post-dt_sugar URL: https://reviews.freebsd.org/D38732 Contact: Christos Margiolis Contact: Mark Johnston kinst is a new DTrace provider created by christos@ and markj@ that allows for arbitrary instruction tracing in a kernel function. kinst has been added to the base system in FreeBSD 14.0. The 2022Q3 status report gives a brief introduction to kinst. We’re now working on inline function tracing (see review D38825 above) — a much-requested DTrace feature — by using kernel DWARF and ELF info to find the call sites of each inline copy and use that information to transform D syntax by turning kinst probes of the form: kinst::: // { } To: kinst:::, kinst:::, kinst::: // { } For example: # dtrace -dn 'kinst::cam_iosched_has_more_trim:entry { printf("\t%d\t%s", pid, execname); }' kinst::cam_iosched_get_trim:13, kinst::cam_iosched_next_bio:13, kinst::cam_iosched_schedule:40 { printf("\t%d\t%s", pid, execname); } dtrace: description 'kinst::cam_iosched_has_more_trim:entry ' matched 3 probes CPU ID FUNCTION:NAME 2 79315 cam_iosched_next_bio:13 0 kernel 2 79316 cam_iosched_schedule:40 0 kernel 0 79316 cam_iosched_schedule:40 12 intr 2 79315 cam_iosched_next_bio:13 0 kernel 2 79316 cam_iosched_schedule:40 0 kernel 0 79316 cam_iosched_schedule:40 12 intr ^C A new -d flag has also been added to dtrace(1) which dumps the D script after libdtrace has applied syntactic transformations. Further goals include: • Implement a locals structure in D which stores the local variables of the traced function. For example with kinst::foo:, we could print the local variable bar by doing print(locals→bar) inside a D script. • Port kinst to riscv and/or arm64. Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Native Linux timerfd Links: Differential revision URL: https://reviews.freebsd.org/D38459 Contact: Jake Freeland The timerfd facility is a set of Linux-standard system calls that operate on interval timers. These timers are analogous to per-process timers but are represented by a file descriptor, rather than a process. These file descriptors may be passed to other processes, are preserved across fork(2), and may be monitored via kevent(2), poll(2), or select(2). A timerfd implementation in FreeBSD already exists for Linux compatibility, but this differential revision makes the interface native. The goal behind this change is to ease the FreeBSD porting process for programs that include timerfd. This specific implementation avoids adding new names to the system call table. Instead, timerfd_create() is wrapped by the specialfd() system call. The timerfd_gettime() and `timerfd_settime() calls are wrapped ioctl() s. Developers that wish to support FreeBSD should avoid using timerfd. The kqueue () EVFILT_TIMER filter is preferred for establishing arbitrary timers. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Architectures Updating platform-specific features and bringing in support for new hardware platforms. Kernel Address Sanitizer on AArch64 Contact: Kyle Evans Sanitizers are bug detection facilities which use a combination of instrumentation inserted by the compiler (LLVM in this case) and runtime state tracking to detect bugs in C code. They can automatically detect many types of C programming bugs, such as use-after-frees and uses of uninitialized variables, which may otherwise require substantial effort to identify. They are particularly effective in combination with regression testing suites or fuzzing tools such as syzkaller. Unlike tools such as Valgrind, software must be recompiled to enable a given sanitizer, but sanitizers can be used in the kernel. Kernels with sanitizers enabled incur a significant performance overhead from the runtime, in both CPU utilization and memory usage. As of 89c52f9d59fa, the kernel address sanitizer that was previously exclusive to amd64 is ported to arm64. Prior testing has been done on a decent variety of machines, including: • Various Ampere Altra machines • QEMU • Microsoft’s "Volterra" Devkit • bhyve (WIP). Further testing on other hardware would be both welcomed and appreciated. Sponsor: Juniper Networks, Inc. Sponsor: Klara, Inc. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ bsd-user: Upstreaming and Status Report Links: QEMU Project URL: https://qemu.org FreeBSD bsd-user qemu fork URL: https://github.com/qemu-bsd-user/qemu-bsd-user QEMU Project’s gitlab mirror URL: https://gitlab.com/qemu-project/qemu Contact: Warner Losh In this quarter, Warner upstreamed two patch sets in the qemu-project repo (with a third pending). Doug Rabson submitted some optimizations to save a handle to the qemu-user emulator in the kernel for future exec. Contact has been made with some folks interested in getting bsd-user working on NetBSD. Summer of Code project to upstream shows some interest. Upstreaming Efforts The sysctl system call was upstreamed this quarter. Doug’s changes were also upstreamed (see below). Some cleanups around NetBSD and OpenBSD and to generate syscalls on the fly are pending. Doug Rabson’s Changes As part of his container work, Doug submitted changes that allows the kernel to cache the emulator used to run programs. This allows the kernel to directly exec the new binary with that cached emulator. This simplifies bsd-user and removes one source of difference between it and linux-user. Doug also provided an important fix that prevented aarch64 from running. Bug Fixes and Improvements In addition to Doug’s fixes, Warner cleaned up things a bit this quarter. • Warner removed the final bits of 'run on any BSD code' that was present in the emulator. • While the basic system calls could be emulated between all the BSDs, their system call interface has diverged too much, and it is too feature rich for this to be feasible any time soon. • Warner had planned to just remove the NetBSD and OpenBSD bits, but there is some interest from at least the NetBSD folks for making things build. • Now that the NetBSD folks have contact information, and know direction, Warner hopes they will submit a pull request to build bsd-user on NetBSD for NetBSD. • Warner added SIGSYS support so that we can catch unimplemented system calls sooner, and improved reporting of them to get more data about what fails. • Warner cleaned up some code in the blitz branch. • We’re merged up to 8.0rc1 in upstream in the blitz branch we’re using to stay current. Summer of Code Projects There’s much interest in the bsd-user upstreaming task Warner added to Qemu’s project list. With luck, we’ll have a student to fund to do the job of upstreaming all the system calls needed to run simple programs. With a lot of luck, we’ll be able to run any program that does the same thing(s) that clang does (one goal is to have it compile hello world). Future quarterly reports will provide details, should we be fortunate enough to get a slot for this. Help Needed We can always use help with bsd-user. • Pull requests for new system calls are welcome. • Automation in generating many of the things we do by hand would be helpful (like system call argument tracing). • Enthusiastic volunteers who want to help me with the upstreaming (many tasks are easy and quick if you don’t want to commit). • Coordination with the NetBSD folks and cleanup they come up with. • Bug fixes (especially thread bugs) are needed. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Cloud Updating cloud-specific features and bringing in support for new cloud platforms. FreeBSD as a Tier 1 cloud-init Platform Links: cloud-init Website URL: https://cloud-init.io/ cloud-init Documentation URL: https://cloudinit.readthedocs.io/en/latest/ cloud-init ongoing refactorization URL: https://github.com/canonical/cloud-init/blob/main/WIP-ONGOING-REFACTORIZATION.rst Contact: Mina Galić cloud-init is the standard way of provisioning servers in the cloud. Unfortunately, cloud-init support for operating systems other than Linux is rather poor, and the lack of cloud-init support on FreeBSD is a hindrance to cloud providers who want to offer FreeBSD as a Tier 1 platform. To remedy the situation, this project aims to bring FreeBSD cloud-init support on par with Linux support. The broader plan is to lift support across all BSDs. This quarter has been going very, very slowly, for personal reasons — also for lack of access to the right resources. I have been trying to port the Infiniband functions. This has proven difficult, because it falsified my thesis that ifconfig(8) is all that is needed to figure out network interfaces on FreeBSD. While waiting for resources, I debugged a boot panic and got it fixed: 499171a98c88. This now makes it possible to boot FreeBSD on LXD — cloud-init’s CI platform. We still need to fix the high CPU usage problem, but there is already an accepted review: D38898 A cloud-init colleague who works for Azure managed to give me access to an HPC VM on Azure. Unfortunately, it was only for a limited time, and that was not enough to figure out how to get Infiniband up and running on FreeBSD — a task handled by Azure Agent on Linux, but FreeBSD’s sysutils/azure-agent is rather lacking. People interested in helping with this project could provide ifconfig(8), ibstat(8), ibv_devinfo(1), etc… pastes from their Infiniband systems. I would also be very happy about getting access to hardware with Infiniband NICs, or hearing from people who have successfully used FreeBSD on Azure HPC with Infiniband. If there is interest in that platform, I will direct some energy to fixing Azure Agent. Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ OpenStack on FreeBSD Links: OpenStack URL: https://www.openstack.org/ OpenStack on FreeBSD URL: https://github.com/openstack-on-freebsd Contact: Chih-Hsin Chang Contact: Li-Wen Hsu This project aims to port key OpenStack components so that FreeBSD can function as an OpenStack host. In 2023 Q1, the big news is that we’re able to spawn FreeBSD instances with bhyve(8) on the OpenStack platform. But there are still some major limitations regarding the capabilities of the spawned instances that need to be resolved: • No self-service networks (only support the flat network) • No network connectivity inside the instance • Only support FreeBSD raw images (FreeBSD-13.1-RELEASE-amd64.raw tested) • No disk resize • No console integration (need to use cu(1) command manually) The step-by-step documents for constructing a POC site can be found in the docs repository. The patched version of each OpenStack component is under the same GitHub organization. Also, we attended AsiaBSDCon 2023 at the end of March and gave a short talk about the current project status at the developer summit. We got precious feedback at the event and will focus on the following for the next quarter: • Resolve the Open vSwitch networking issue • Convert each OpenStack component into FreeBSD ports People interested in helping with the project can first help check the documentation by following the installation guide. And here is an open task for the project: • FreeBSD-specific implementation for the oslo.privsep library Feedback and help are always welcome. Sponsor: The FreeBSD Foundation ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Documentation Noteworthy changes in the documentation tree, manual pages, or new external books/documents. Documentation Engineering Team Links: FreeBSD Documentation Project URL: https://www.freebsd.org/docproj/ FreeBSD Documentation Project Primer for New Contributors URL: https://docs.freebsd.org/en/books/fdp-primer/ Documentation Engineering Team URL: https://www.freebsd.org/administration/#t-doceng Contact: FreeBSD Doceng Team The doceng@ team is a body to handle some of the meta-project issues associated with the FreeBSD Documentation Project; for more information, see FreeBSD Doceng Team Charter. During the last quarter: • The doc commit bit for Pau Amma was taken in. • Lorenzo Salvadore has been proposed as doc committer. carlavilla@ and dbaio@ will mentor him. • Ryusuke SUZUKI steps down from doceng. doceng would like to thank ryusuke@ for his service. Items pending and in the discussion: • A new document about licensing has been added to the documentation set. Porter’s Handbook: Three new Uses knobs have been added to the Handbook: • New Uses = ruby. • New Uses = ldap. • New Uses = budgie. Also: • The NVIDIA install and configure options have been fixed • The Advanced Networking chapter has been improved FreeBSD Translations on Weblate Link: Translate FreeBSD on Weblate Link: FreeBSD Weblate Instance Q4 2022 Status • 12 languages • 150 registered users Languages • Chinese (Simplified) (zh-cn) (progress: 14%) • Chinese (Traditional) (zh-tw) (progress: 11%) • Dutch (nl) (progress: 1%) • French (fr) (progress: 1%) • German (de) (progress: 1%) • Indonesian (id) (progress: 1%) • Italian (it) (progress: 10%) • Korean (ko) (progress: 11%) • Norwegian (nb-no) (progress: 1%) • Persian (fa-ir) (progress: 6%) • Portuguese (pt-br) (progress: 29%) • Sinhala (si) (progress: 1%) • Spanish (es) (progress: 37%) • Turkish (tr) (progress: 5%) We want to thank everyone that contributed, translating or reviewing documents. And please, help promote this effort on your local user group, we always need more volunteers. FreeBSD Handbook working group Contact: Sergio Carlavilla Chapters 1 to 6 have been updated. Chapter 7 is work in progress. FreeBSD Website Revamp - WebApps working group Contact: Sergio Carlavilla Working group in charge of creating the new FreeBSD Documentation Portal and redesigning the FreeBSD main website and its components. FreeBSD developers can follow and join the working group on the FreeBSD Slack channel #wg-www21. The work will be divided into four phases: 1. Redesign of the Documentation Portal Create a new design, responsive and with global search. (Complete) 2. Redesign of the Manual Pages on web Scripts to generate the HTML pages using mandoc. (Complete) Public instance on https://man-dev.FreeBSD.org 3. Redesign of the Ports page on web Ports scripts to create an applications portal. (Work in progress) 4. Redesign of the FreeBSD main website New design, responsive and dark theme. (Work in progress) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ The FreeBSD Russian Documentation Project Links: FAQ URL: https://docs.freebsd.org/ru/books/faq/ Web URL: https://www.freebsd.org/ru/ Contact: Andrey Zakhvatov The FreeBSD Russian Documentation Project current goal is to provide up-to-date Russian translations of the most significant parts of FreeBSD documentation (FAQ, Handbook, Web). It is important to support Russian-speaking persons with high-quality official technical materials and increase acceptance of the operating system around the globe. We hope that this activity will receive some support within the Russian-speaking FreeBSD community and lead to an increased number of translated materials. FAQ translation was updated and synched with the latest original version. There is also a very slight progress with web pages updates. Check the official translation guide in case you are willing to help. We will appreciate your help with translation of the following materials: • Web pages (easy) • Handbook sections (only the X11 section is in progress right now) • Articles ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Ports Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves. Freshports: SQL Injection Attack and Help Request Links: FreshPorts URL: freshports.org FreshPorts blog URL: https://news.freshports.org/ Contact: Dan Langille FreshPorts and FreshSource have reported upon FreeBSD commits for 20 years. They cover all commits, not just ports. FreshPorts tracks the commits and extracts data from the port Makefiles to create a database of information useful to both port maintainers and port users. For example, https://www.freshports.org/security/acme.sh/ shows the history of the security/acme.sh port, back to its creation in May 2017. Also available are dependencies, flavors, configuration options, and available packages. All of this is useful for both users and developers of ports. SQL Injection Attack In March, an SQL injection attack was noticed and the website was patched. Notices were sent out via our Twitter account, our status page, and a notice on the top of each page of the website. The immediate attack vector was shutdown and soon patched. Additional preventative patches were added across the website. Everything we know about has been fixed. Users were notified and advised to change their passwords. Details at: • https://news.freshports.org/2023/03/24/sql-inejection-issues-fixed/ • https://news.freshports.org/2023/03/24/freshsource-code-fixes/ Help Needed It has been over 22 years since FreshPorts started. Others must take over eventually. I’d like to start that process now. There are several aspects to FreshPorts: • FreeBSD admin (updating the OS and packages) • front end code (website - mostly PHP) • back end code (commit processing - Perl, Python, shell) • database design (PostgreSQL). The database does not change very often and requires little maintenance compared to the applications and OS. The website pretty much runs itself. From time to time, a change to the FreeBSD ports infrastructure breaks something or requires a modification, but there is rarely any urgency to fix that. This is not a huge time commitment. There is a lot of learning. While not a complex application, FreshPorts is also not trivial. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ DRM drivers (i.e. GPU drivers) Links: Git repository on GitHub URL: https://github.com/freebsd/drm-kmod Contact: Emmanuel Vadot Contact: Jean-Sébastien Pédron Contact: The Graphics team GPUs are driven by DRM drivers. They are developed specifically for Linux using a permissive license. Our mission is to port those drivers to FreeBSD to make sure modern GPUs are fully supported. We didn’t publish a report to share our progress for a long time. Therefore this status report entry will cover more than just the last quarter. Update to Linux 5.15 LTS and Linux 5.16 As of this status report, the graphics/drm-kmod meta port still installs the DRM drivers from Linux 5.10 (released on December 13, 2020) on FreeBSD 13.1 and greater. This version of the driver lacks support for recent GPUs, in particular Intel 12th gen Alder Lake ones. In the past months, we worked to update the DRM drivers to bring support for more modern AMD and Intel GPUs. The drm-kmod Git repository master branch was first updated to Linux 5.15 (released on October 31, 2021). This is an LTS branch in Linux and we wanted to take advantage of that. Thus at that point, we followed two paths: • A 5.15-lts branch was created to backport all bug fixes from Linux 5.15.x patch releases. This work is now available in the drm-515-kmod port. • The porting effort from subsequent Linux versions continued. The master branch is now at Linux 5.16 (release on January 9, 2022). The Intel driver from Linux 5.15 LTS supports 12th gen GPUs (Alder Lake). It looks to work on FreeBSD but we only tested it lightly so far. We still need more of that, that’s why graphics/drm-kmod still installs graphics/drm-510-kmod instead of graphics/drm-515-kmod. At last, FreeBSD should run as a desktop on this GPU generation and several new AMD GPUs, though problems will surely appear through real test and use. In the process, we updated firmwares to linux-firmware 20230210. Linux 5.17 and future work DRM drivers from Linux 5.17 (released on March 20, 2022) were already ported but this work still sits in its own branch. A couple of issues block further testing and the merge into the master branch: • Our current integration with vt(4), the console/terminal driver, is quite far from the DRM drivers expectations which are based on Linux' fbdev KPI. Something changed in both the Intel and AMD drivers, meaning that vt(4) breaks with the 5.17 update. • The initial Linux 5.17 release does not contain the fixes backported to Linux 5.15 LTS. It seems quite unstable with the Intel 12th gen GPU mentioned earlier. To address the issue with our vt(4) integration layer, we started to write a new vt backend specifically to use the fbdev callbacks exposed by the DRM drivers. This backend will be provided with the DRM drivers, not the FreeBSD kernel, to make it easier to maintain as the drivers evolve. This is still a work in progress and locking in particular is tricky to get right. Regarding the bad support of Intel 12th gen in the 5.17 update, bug fixes backported to Linux 5.17.x patch releases will probably not be ported as part of this work. Instead we will focus on Linux 5.18 (released on May 22, 2022) and following. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ KDE on FreeBSD Links: KDE FreeBSD URL: https://freebsd.kde.org/ KDE Community FreeBSD URL:https://community.kde.org/FreeBSD Contact: Adriaan de Groot The KDE on FreeBSD project packages CMake, Qt, and software from the KDE Community, for the FreeBSD ports tree. The software includes a full desktop environment called KDE Plasma (for both X11 and Wayland) and hundreds of applications that can be used on any FreeBSD machine. The KDE team (kde@) is part of desktop@ and x11@, building the software stack to make FreeBSD beautiful and usable as a daily-driver graphics-based desktop machine. The notes below describe mostly ports for KDE, but also include items that are important for the entire desktop stack. Infrastructure • The Qt5 ports were updated to the KDE patch collection release 5.15.8. • The Qt6 ports — these are not used by KDE yet, but there are many ports that can use Qt6 and have Qt6 flavors — were updated to release 6.4.2. Python bindings for the Qt6 release of WebEngine were added. • The cmake ports were updated to release 3.25.1 and the CPack generator for FreeBSD packages was repaired. • The graphics/poppler port — used by many PDF-viewers — was updated to release 23.01. • The sysutils/bsdisks port — used as a shim for applications that expect Linux udisks, which means most desktop environments — was updated to release 0.29. KDE Stack KDE Gear releases happen every quarter, KDE Plasma updates once a month, and KDE Frameworks have a new release every month as well. These (large) updates land shortly after their upstream release and are not listed separately. • KDE Frameworks updated to 5.104. • KDE Gear updated to 22.12.3. • KDE Plasma Desktop was updated to version 5.27. This was a long delayed update, due to unresolved issues in the support stack and a misplaced patch from an earlier release of KDE Plasma. Thanks to arrowd@ and Serenity Cybersecurity, LLC for sorting that out. • New port devel/ktextaddons was added to the tree. This is part of the KDE PIM suite, and slated to become a new KDE Framework in some future release. Related Ports • audio/amarok, one of the most popular KDE audio players of the early 2000’s, has been marked deprecated in the ports tree. It is no longer maintained upstream. • astro/kstars, an interactive planetarium, was updated to release 3.6.3. • devel/gitqlient, a graphical user interface for git, was updated to release 1.6.1 with support for new git commands. • devel/okteta, a hex viewer and editor for binary files, was updated to release 0.26.10. • devel/qcoro, C++ coroutines with Qt support, was updated to release 0.8.0. • graphics/krita, an application for painting and graphical work, was updated to release 5.1.5. • graphics/quickqanava, a graph visualization library, got a real release and an update in the ports tree. • irc/kvirc, an IRC client, was updated to the latest commit; there is no real release but there are bugfixes. • multimedia/haruna, a video and audio player, was updated to release 0.10.3. • net-im/neochat, one of a handful of Matrix clients, was updated to chase a new release of net-im/libquotient. There are continuing troubles with compatibility with older FreeBSD releases, leading to the KDE-FreeBSD team to declare FreeBSD 12 releases "effectively unsupported". • net-im/ruqola, a Rocket Chat client, was updated to release 1.9.1. • security/keysmith, a two-factor-authentication support application, was updated to release 23.01.0. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ FSX Links: GitHub URL: https://github.com/asomers/fsx-rs FreshPorts URL: https://www.freshports.org/devel/fsx/ Contact: Alan Somers The venerable FSX (File System eXerciser) tool, first written at Apple Computer in the nineties, has been a part of FreeBSD since 5.0. It stress tests file systems with a stream of randomly generated operations, verifying file data after every read. However, it has never been installed as part of the OS; it only exists in the source tree. That makes it difficult to use in CI pipelines. It has some other limitations, too. So this quarter I rewrote the entire tool in Rust. The rewrite is byte-for-byte compatible with the original, given identical seed values. Future versions will break backwards-compatibility, however, in order to add new features like fspacectl and copy_file_range. The new version can be found in the ports tree, and in time I’ll remove the original. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ GCC on FreeBSD Links: GCC Project URL: https://gcc.gnu.org GCC 11 release series URL: https://gcc.gnu.org/gcc-11/ GCC 12 release series URL: https://gcc.gnu.org/gcc-12/ Contact: Lorenzo Salvadore Contact: Gerald Pfeifer The main news this quarter is the cleaning of old GCC versions from the ports tree: this will allow for a more efficient approach to bugs. Deprecation of old GCC ports The ports tree still contains several ports related to old and unsupported GCC versions. They are usually needed as dependencies for a few old ports, that it would be better to either update to use a supported GCC release, or deprecate. Bug reports have been created to track the issue and work has already started towards its resolution. Thanks to all ports contributors who are helping. Deprecation of USE_GCC=X+ Gerald, who maintained the GCC ports for many years until recently, still contributes to the GCC maintenance on FreeBSD by helping simplify the GCC infrastructure in the ports tree, for example by removing special cases that deal with old unsupported GCC versions. This quarter the most significant of his changes is probably the removal of support for the USE_GCC=X+ construct: any port depending on GCC should set USE_GCC=yes if GCC_DEFAULT works; if not, it should require a specific version (e.g. USE_GCC=11); it cannot ask for a minimal version anymore (e.g. USE_GCC= 11+). ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Valgrind - Preparing for Valgrind 3.21 Links: Valgrind Home Page URL: https://www.valgrind.org/ Valgrind News URL: https://www.valgrind.org/docs/manual/dist.news.html Contact: Paul Floyd The devel/valgrind-devel port had an intermediate update which was submitted on 2023-02-20. This contains most of what will be in the official release of Valgrind 3.21 which is due out shortly after this status report. There is a nice improvement to the vgdb interface. It’s now much easier to see which bits of memory are initialized or not. There are a couple of fixes to the thread checks done by Helgrind. For FreeBSD specifically, the address space limit has been raised to be the same as Linux and Solaris on amd64. It was 32Gbytes and now it is 128Gbytes. The kern.proc.pathname.PID sysctl(3) has been fixed so that it returns the path of the guest exe and not that of the Valgrind host. At the same time I fixed some _umtx_op false positives and corrected auxv AT_EXECPATH in a way similar to kern.proc.pathname.PID. Syscall wrappers have been added for sctp_generic_sendmsg(2) and sctp_generic_recvmsg(2). Not yet available in the ports versions of Valgrind, there is a workaround for the use of rfork(2). Previously, since it is not supported, it would cause Valgrind to abort. Now it fails gracefully setting either EINVAL or ENOSYS. The main use of this system call is in posix_spawn(3), which will fall back to using vfork(2). The mknodat(2) syscall wrapper was incorrectly implemented on i386 and has now been fixed. There is a reworking of all of the aligned allocation functions so that they behave less like Linux glibc and more like the Valgrind build platform. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Third Party Projects Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions. PkgBase.live Links: Website (archive.org) URL: https://web.archive.org/web/20221220222828/https://alpha.pkgbase.live/ Website source URL: https://codeberg.org/pkgbase/website Contact: Mina Galić PkgBase.live was an unofficial repository for the FreeBSD PkgBase project. As a service, PkgBase.live was inspired by https://up.bsd.lv/, which provides freebsd-update(8) for STABLE and CURRENT branches. Hardware for PkgBase was kindly sponsored by a member of the FreeBSD community. However, as life and projects moved on, they had to decommission the hardware, giving me three months' notice. In that time, my own life was rather turbulent after a recent move to a different country so I haven’t been able to find a replacement. For the time being, PkgBase.live is dead. The website, and with it the How Did She Do it?! are still available in Git. I highly encourage copy-cats. I will also happily accept a new hardware sponsor! Please note that I have contacted the FreeBSD Project, and they are working on integrating PkgBase into release engineering. However, they are not yet ready, they also cannot "simply" take over PkgBase.live because it uses a completely different process. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Containers and FreeBSD: Pot, Potluck and Potman Links: Pot organization on GitHub URL: https://github.com/bsdpot Contact: Luca Pizzamiglio (Pot) Contact: Bretton Vine (Potluck) Contact: Michael Gmelin (Potman) Pot is a jail management tool that also supports orchestration through Nomad. During the last quarter, pot received a number of minor fixes but no new version has been released yet. Potluck aims to be to FreeBSD and pot what Dockerhub is to Linux and Docker: a repository of pot flavours and complete container images for usage with pot and in many cases Nomad. All Potluck images have been rebuilt to include the latest FreeBSD security advisories, a new Smokeping network latency monitoring image has been added, again a lot of work went into the Jitsi image, which unfortunately still seems to have some reliability issues. Also, two new blog posts are available showing how easy it is to use Potluck images, one explaining how to set up Nextcloud with Minio as object storage and Prometheus for monitoring, one showing how to run your own Matrix Synapse server using OpenLDAP for access management. As always, feedback and patches are welcome. ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Last modified on: April 19, 2023 by Lorenzo Salvadore Legal Notices | © 1995-2023 The FreeBSD Project All rights reserved. The mark FreeBSD is a registered trademark of The FreeBSD Foundation and is used by The FreeBSD Project with the permission of The FreeBSD Foundation. Contact From nobody Wed Apr 19 14:35:32 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1jy85z9Kz463Fx; Wed, 19 Apr 2023 14:35:44 +0000 (UTC) (envelope-from developer@lorenzosalvadore.it) Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1jy809nBz3yth; Wed, 19 Apr 2023 14:35:44 +0000 (UTC) (envelope-from developer@lorenzosalvadore.it) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=lorenzosalvadore.it header.s=protonmail2 header.b=IGqsQI9F; spf=pass (mx1.freebsd.org: domain of developer@lorenzosalvadore.it designates 185.70.40.136 as permitted sender) smtp.mailfrom=developer@lorenzosalvadore.it; dmarc=pass (policy=quarantine) header.from=lorenzosalvadore.it Date: Wed, 19 Apr 2023 14:35:32 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lorenzosalvadore.it; s=protonmail2; t=1681914941; x=1682174141; bh=hTG1LBaFj0Aj3OcnAcCleHHMouY+aKDBblrpqoxhf50=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=IGqsQI9FRu2IlhSxWUlpZBHNnACYhB4MdAB23LwNPu4H+4dZIHuy57NkFf1OIH6Ym X3z1anAO+aggn5g/MqnYV4tPjmHWq9LerfxNU+yI5l93OgdFlEuHE/8TchBWJDdngx yqb1FVcDrzTDvKD2ZUF2ULNN3RyCTtc1afIC4OwzKWz3ZKpLawqYVML1FeNwkWUNno TYlsD1xwbjlYCmskBnRjPNIXJAzoZGftIsDFHr7uIFATPueSxQ6u9gCKQ4PbqB8hKl JPEzbi6o5CHcrSi9xCdKwfPAbO1a+5HinpautT/XOwL49+a8p7Bm7QLmuCywghscFe cho/S2kJ3J4LQ== To: freebsd-hackers@freebsd.org From: Lorenzo Salvadore Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD Status Report - First Quarter 2023 Message-ID: <3I6mKfXCcePfrAqIhoVuCxBJ5jvu6SlTCpavwH9J5ylbifaO7sFdlxCb9Co9ANCRftFaHxEt_mt6MP9C1sa1FlI_l1c92mo-vwHtaQKj9N0=@lorenzosalvadore.it> In-Reply-To: References: Feedback-ID: 53711648:user:proton List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.00 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[lorenzosalvadore.it,quarantine]; R_SPF_ALLOW(-0.20)[+ip4:185.70.40.0/24:c]; R_DKIM_ALLOW(-0.20)[lorenzosalvadore.it:s=protonmail2]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org,freebsd-stable@freebsd.org]; RCVD_COUNT_ZERO(0.00)[0]; ASN(0.00)[asn:62371, ipnet:185.70.40.0/24, country:CH]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; DKIM_TRACE(0.00)[lorenzosalvadore.it:+]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_NONE(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[185.70.40.136:from] X-Rspamd-Queue-Id: 4Q1jy809nBz3yth X-Spamd-Bar: --- X-ThisMailContainsUnwantedMimeParts: N ------- Original Message ------- On Wednesday, April 19th, 2023 at 4:26 PM, Lorenzo Salvadore wrote: > Last modified on: April 19, 2023 by Lorenzo Salvadore >=20 > Legal Notices | =C2=A9 1995-2023 The FreeBSD Project All rights reserved.= The mark > FreeBSD is a registered trademark of The FreeBSD Foundation and is used b= y The > FreeBSD Project with the permission of The FreeBSD Foundation. Contact I am sorry for the footer: this should not have been included in the mail v= ersion of the status report, indeed it is the footer of the web page where the rep= ort is rendered: https://www.freebsd.org/status/report-2023-01-2023-03/ . Lorenzo Salvadore From nobody Wed Apr 19 19:39:49 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q1rjH290nz46N0n for ; Wed, 19 Apr 2023 19:40:03 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q1rjG1J5jz3t1v for ; Wed, 19 Apr 2023 19:40:02 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20221208.gappssmtp.com header.s=20221208 header.b="X5/qMhXH"; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::62a) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-94e53ef6815so11463466b.1 for ; Wed, 19 Apr 2023 12:40:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1681933201; x=1684525201; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=P579+KCtonUdcQk6BFuCe322FkmHF60AnbObRsuvVrE=; b=X5/qMhXHbUlTE7eKw23S7HH47NLMKEjCOQTnHmLfoHN9iQe58eNbhpgpyIPqVJ1K3+ JUg9GAjN+ETL+Epthic4yWti983ca97zx3FEPuvartZZY7wi83FgiOI9WgOEh01Nw8lb Ir9RZhFFS0/7xJqbFGkAqUEkYM0anSyABa8KKDBLOfHstV7sCrxV9d1SGxrOP7MG5Tly BsCukH6L+f60abzk1sg8VQaHVdHfnhive/icm5cD+bRl5QPgRY/8TSAxNI81tPyrGUZ4 UbvWj0R8ZvlJD9KjuJtjjd+7Vk/z8UILZlL8IE8TSkyA699+fYvf9rQAKjK7fcpWDlon pofA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681933201; x=1684525201; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=P579+KCtonUdcQk6BFuCe322FkmHF60AnbObRsuvVrE=; b=NA4bvj7/5lOKZlTp17TyPLCDN1QzBXffmiQVSlc4vB1ukEKIbu/Oc/PlI47yAf4lfF z6GvOhFfQin/HRv0fvM6WRMgXmVJlNWM3J+hMDjh7iXldWsIQOut1cZQaA3VZJfjO6w6 nd6l4tNuu0SmZyZTTdGhca8YkIJPx5Y4urDBvvt50+RLJ5slMpqP/QRMzTe3koBmL+gM JfVb/OrKOTRXWJzQOxxSwWqiPBoG3gxbw5chCnw4c0UsCyFRq+/cx28dLxdrl8cTHTaQ k2QNes+FsSSlPjsTcEVx21zluBLg9cPJ5EGBiIjEyMM1f6+lDwBBaJpnOAiqipM/eXR+ vMKQ== X-Gm-Message-State: AAQBX9c7r8aZ87RrG3H0hGBBCngM2Wu7RKyEaatNSth3HR817TIq8QaW YnlyRbhvXfJAhzcwKCVrfa2YyKRvr7rS+pfCHA7JdA0qsqgMiarr X-Google-Smtp-Source: AKy350aelHWGDmAesKZu65QNWAh37Sprm4gS0gSY1HOYmsgeybDGya3mHxpydCFLnV1bpPVUeiQDR28jbsFgvDUG9vE= X-Received: by 2002:a05:6402:158:b0:506:b211:489 with SMTP id s24-20020a056402015800b00506b2110489mr7078362edu.35.1681933200722; Wed, 19 Apr 2023 12:40:00 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <20210520180155.3e23500e@bsd64.grem.de> <20210520185917.GL14975@funkthat.com> <20220619140624.21263e46@bsd64.grem.de> <20230418123557.443cd189@bsd64.grem.de> In-Reply-To: <20230418123557.443cd189@bsd64.grem.de> From: Warner Losh Date: Wed, 19 Apr 2023 13:39:49 -0600 Message-ID: Subject: Re: Reducing SIGINFO verbosity To: Michael Gmelin Cc: Ceri Davies , Conrad Meyer , "freebsd-current@freebsd.org" Content-Type: multipart/alternative; boundary="000000000000f7229305f9b5969d" X-Spamd-Result: default: False [-2.99 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.992]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20221208.gappssmtp.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; R_SPF_NA(0.00)[no SPF record]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::62a:from]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[bsdimp-com.20221208.gappssmtp.com:+]; RCPT_COUNT_THREE(0.00)[4]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[bsdimp.com]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com] X-Rspamd-Queue-Id: 4Q1rjG1J5jz3t1v X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N --000000000000f7229305f9b5969d Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Doh! Slipped through the cracks... Building test kernel now and will push if it succeeds and boots. Warner On Tue, Apr 18, 2023 at 4:36=E2=80=AFAM Michael Gmelin wr= ote: > > > On Thu, 23 Jun 2022 11:15:55 -0600 > Warner Losh wrote: > > > On Sun, Jun 19, 2022 at 6:06 AM Michael Gmelin > > wrote: > > > > > > > > > > > On Fri, 21 May 2021 08:36:49 -0600 > > > Warner Losh wrote: > > > > > > > On Fri, May 21, 2021 at 7:38 AM Ceri Davies > > > > wrote: > > > > > > > > > On Thu, May 20, 2021 at 03:57:17PM -0700, Conrad Meyer wrote: > > > > > > No, I don=E2=80=99t think there=E2=80=99s any reason to default= it > > > > > > differently on stable > > > > > vs > > > > > > current. I think it=E2=80=99s useful (and I prefer the more ver= bose > > > > > > form, which isn=E2=80=99t the default). > > > > > > > > > > I agree that there's no reason for the default to be different, > > > > > but I would say that it is much easier for someone who knows > > > > > that there is a default to be changed to change it, than it is > > > > > for someone who does not. Therefore, if the information is not > > > > > useful to someone who does not know how to get rid of it, then > > > > > it should default to not being displayed, IMHO. > > > > > > > > > > > > > I plan on changing the default for non-INVARIANT kernels back to > > > > the old behavior. > > > > > > > > INVARIANT kernels will keep this behavior because it's a debugging > > > > kernel and this is one more thing to help debugging problems. > > > > > > > > > > Did this ever happen? I just installed a fresh 13.1-RELEASE > > > production system (non-INVARIANT kernel) and it seems like SIGINFO > > > still outputs kernel stack information. > > > > > > > https://reviews.freebsd.org/D35576 for those who wish to weigh in. > > > > Warner > > > > > > Hi Warner, > > I just installed 13.2-RELEASE, seems like this was never MFCd (it is in > main, but not in stable/13 or releng/13.2). TBH, I could've checked > myself back then (it's so easy to forget to MFC). > > Cheers > Michael > > p.s. Learned about it by hitting ctrl-t to check if freebsd-update on my > slow test machine is actually alive :D > > -- > Michael Gmelin > --000000000000f7229305f9b5969d Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Doh! Slipped through=C2=A0the cracks...=C2=A0

Building test kernel now and will push if it succeeds and boots.

Warner

On Tue, Apr 18, 2023 at 4:36=E2=80=AFAM M= ichael Gmelin <freebsd@grem.de>= ; wrote:


On Thu, 23 Jun 2022 11:15:55 -0600
Warner Losh <imp@bsd= imp.com> wrote:

> On Sun, Jun 19, 2022 at 6:06 AM Michael Gmelin <freebsd@grem.de>
> wrote:
>
> >
> >
> > On Fri, 21 May 2021 08:36:49 -0600
> > Warner Losh <imp@bsdimp.com> wrote:
> >=C2=A0
> > > On Fri, May 21, 2021 at 7:38 AM Ceri Davies <ceri@submonkey.net>
> > > wrote:
> > >=C2=A0
> > > > On Thu, May 20, 2021 at 03:57:17PM -0700, Conrad Meyer = wrote:=C2=A0
> > > > > No, I don=E2=80=99t think there=E2=80=99s any reas= on to default it
> > > > > differently on stable=C2=A0
> > > > vs=C2=A0
> > > > > current. I think it=E2=80=99s useful (and I prefer= the more verbose
> > > > > form, which isn=E2=80=99t the default).=C2=A0
> > > >
> > > > I agree that there's no reason for the default to b= e different,
> > > > but I would say that it is much easier for someone who = knows
> > > > that there is a default to be changed to change it, tha= n it is
> > > > for someone who does not. Therefore, if the information= is not
> > > > useful to someone who does not know how to get rid of i= t, then
> > > > it should default to not being displayed, IMHO.
> > > >=C2=A0
> > >
> > > I plan on changing the default for non-INVARIANT kernels bac= k to
> > > the old behavior.
> > >
> > > INVARIANT kernels will keep this behavior because it's a= debugging
> > > kernel and this is one more thing to help debugging problems= .
> > >=C2=A0
> >
> > Did this ever happen? I just installed a fresh 13.1-RELEASE
> > production system (non-INVARIANT kernel) and it seems like SIGINF= O
> > still outputs kernel stack information.
> >=C2=A0
>
> https://reviews.freebsd.org/D35576 for those who wish to = weigh in.
>
> Warner
>
>

Hi Warner,

I just installed 13.2-RELEASE, seems like this was never MFCd (it is in
main, but not in stable/13 or releng/13.2). TBH, I could've checked
myself back then (it's so easy to forget to MFC).

Cheers
Michael

p.s. Learned about it by hitting ctrl-t to check if freebsd-update on my slow test machine is actually alive :D

--
Michael Gmelin
--000000000000f7229305f9b5969d-- From nobody Thu Apr 20 10:07:09 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q2Cxy5ftVz46c1d for ; Thu, 20 Apr 2023 10:07:18 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q2Cxw6G5Xz3GNQ for ; Thu, 20 Apr 2023 10:07:16 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of jamie@catflap.org designates 2001:19f0:7400:8808:123::1 as permitted sender) smtp.mailfrom=jamie@catflap.org; dmarc=pass (policy=none) header.from=catflap.org X-Catflap-Envelope-From: X-Catflap-Envelope-To: current@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 33KA792M077627; Thu, 20 Apr 2023 11:07:09 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 33KA79wJ077626; Thu, 20 Apr 2023 11:07:09 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202304201007.33KA79wJ077626@donotpassgo.dyslexicfish.net> Date: Thu, 20 Apr 2023 11:07:09 +0100 Organization: Dyslexic Fish To: phk@phk.freebsd.dk.yuri@aetern.org, delphij@gmail.com Cc: current@FreeBSD.org Subject: Re: find(1): I18N gone wild ? References: <202304172106.33HL6RUX051407@critter.freebsd.dk> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Thu, 20 Apr 2023 11:07:09 +0100 (BST) X-Spamd-Result: default: False [-0.35 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-0.95)[-0.947]; DMARC_POLICY_ALLOW(-0.50)[catflap.org,none]; R_SPF_ALLOW(-0.20)[+mx:dyslexicfish.net]; NEURAL_SPAM_SHORT(0.18)[0.176]; NEURAL_SPAM_MEDIUM(0.12)[0.123]; RCVD_NO_TLS_LAST(0.10)[]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[current@FreeBSD.org]; FREEMAIL_TO(0.00)[aetern.org,gmail.com]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; HAS_ORG_HEADER(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[jamie]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US]; TO_DN_NONE(0.00)[]; R_DKIM_NA(0.00)[] X-Rspamd-Queue-Id: 4Q2Cxw6G5Xz3GNQ X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N Xin LI wrote: > This is expected behavior (in en_US.UTF-8 the ordering is AaBb, not ABab). > You might want to set LC_COLLATE to C if C behavior is desirable. > > On Mon, Apr 17, 2023 at 2:06 PM Poul-Henning Kamp > wrote: > > > This surprised me: > > > > # mkdir /tmp/P > > # cd /tmp/P > > # touch FOO > > # touch bar > > # env LANG=C.UTF-8 find . -name '[A-Z]*' -print > > ./FOO > > # env LANG=en_US.UTF-8 find . -name '[A-Z]*' -print > > ./FOO > > ./bar > > > > Really ?! TL;DR Fix find(1) so it works as you expected. It's "legal" to do so. Not quite expected behaviour. It used to be, but now the behaviour is officially undefined, (as mentined in the section that Yuri quoted) When the locale collation first came in, there were numerous issues like this, causing POSIX to change it to undefined (My guess is that it had been one way for too long for them to specifically redefine it, so "undefined" it became.) However, "undefined" would also cover the original way of doing things, and as so many things break unexpectedly, many applications now treat such ranges as they did pre-locales. There would be nothing wrong in therefore changing find(1) to give the results you expected. (and in my opinion, I hope that that becomes the defacto standard) For further justification, note that "awk" in base (in newer versions at least) already gives the results you'd expect, as now does "gawk". In fact, a good summary of the situation, and why the gawk owner reverted the code to treat all character ranges as the tradional pre-locale situation is here: https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html Let's follow suit! Cheers, Jamie From nobody Thu Apr 20 10:08:29 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q2CzQ44Blz46cFs for ; Thu, 20 Apr 2023 10:08:34 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q2CzP3cDPz3JFx for ; Thu, 20 Apr 2023 10:08:33 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=pass (mx1.freebsd.org: domain of jamie@catflap.org designates 2001:19f0:7400:8808:123::1 as permitted sender) smtp.mailfrom=jamie@catflap.org; dmarc=pass (policy=none) header.from=catflap.org X-Catflap-Envelope-From: X-Catflap-Envelope-To: current@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 33KA8UOu077656; Thu, 20 Apr 2023 11:08:30 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 33KA8TpX077655; Thu, 20 Apr 2023 11:08:29 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202304201008.33KA8TpX077655@donotpassgo.dyslexicfish.net> Date: Thu, 20 Apr 2023 11:08:29 +0100 Organization: Dyslexic Fish To: yuri@aetern.org, phk@phk.freebsd.dk, delphij@gmail.com Cc: current@FreeBSD.org Subject: Re: find(1): I18N gone wild ? References: <202304172106.33HL6RUX051407@critter.freebsd.dk> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Thu, 20 Apr 2023 11:08:30 +0100 (BST) X-Spamd-Result: default: False [-2.10 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-0.99)[-0.991]; NEURAL_HAM_MEDIUM(-0.73)[-0.728]; NEURAL_HAM_SHORT(-0.68)[-0.677]; DMARC_POLICY_ALLOW(-0.50)[catflap.org,none]; R_SPF_ALLOW(-0.20)[+mx:dyslexicfish.net:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_NO_TLS_LAST(0.10)[]; MLMMJ_DEST(0.00)[current@FreeBSD.org]; FREEMAIL_TO(0.00)[aetern.org,phk.freebsd.dk,gmail.com]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US]; FROM_HAS_DN(0.00)[]; FREEFALL_USER(0.00)[jamie]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; HAS_ORG_HEADER(0.00)[]; TO_DN_NONE(0.00)[]; MIME_TRACE(0.00)[0:+] X-Rspamd-Queue-Id: 4Q2CzP3cDPz3JFx X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Xin LI wrote: > This is expected behavior (in en_US.UTF-8 the ordering is AaBb, not ABab). > You might want to set LC_COLLATE to C if C behavior is desirable. > > On Mon, Apr 17, 2023 at 2:06 PM Poul-Henning Kamp > wrote: > > > This surprised me: > > > > # mkdir /tmp/P > > # cd /tmp/P > > # touch FOO > > # touch bar > > # env LANG=C.UTF-8 find . -name '[A-Z]*' -print > > ./FOO > > # env LANG=en_US.UTF-8 find . -name '[A-Z]*' -print > > ./FOO > > ./bar > > > > Really ?! TL;DR Fix find(1) so it works as you expected. It's "legal" to do so. Not quite expected behaviour. It used to be, but now the behaviour is officially undefined, (as mentined in the section that Yuri quoted) When the locale collation first came in, there were numerous issues like this, causing POSIX to change it to undefined (My guess is that it had been one way for too long for them to specifically redefine it, so "undefined" it became.) However, "undefined" would also cover the original way of doing things, and as so many things break unexpectedly, many applications now treat such ranges as they did pre-locales. There would be nothing wrong in therefore changing find(1) to give the results you expected. (and in my opinion, I hope that that becomes the defacto standard) For further justification, note that "awk" in base (in newer versions at least) already gives the results you'd expect, as now does "gawk". In fact, a good summary of the situation, and why the gawk owner reverted the code to treat all character ranges as the tradional pre-locale situation is here: https://www.gnu.org/software/gawk/manual/html_node/Ranges-and-Locales.html Let's follow suit! Cheers, Jamie From nobody Thu Apr 20 10:25:23 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q2DLv4YNRz46dDx for ; Thu, 20 Apr 2023 10:25:27 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q2DLv0vvKz42Lb for ; Thu, 20 Apr 2023 10:25:27 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 33KAPO6E077976; Thu, 20 Apr 2023 11:25:24 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 33KAPNS5077974; Thu, 20 Apr 2023 11:25:23 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202304201025.33KAPNS5077974@donotpassgo.dyslexicfish.net> Date: Thu, 20 Apr 2023 11:25:23 +0100 Organization: Dyslexic Fish To: yuri@aetern.org, phk@phk.freebsd.dk, jamie@catflap.org, delphij@gmail.com Cc: current@FreeBSD.org Subject: Re: find(1): I18N gone wild ? References: <202304172106.33HL6RUX051407@critter.freebsd.dk> <202304201008.33KA8TpX077655@donotpassgo.dyslexicfish.net> In-Reply-To: <202304201008.33KA8TpX077655@donotpassgo.dyslexicfish.net> User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Thu, 20 Apr 2023 11:25:24 +0100 (BST) X-Rspamd-Queue-Id: 4Q2DLv0vvKz42Lb X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Jamie Landeg-Jones wrote: > When the locale collation first came in, there were numerous issues > like this, causing POSIX to change it to undefined (My guess is that > it had been one way for too long for them to specifically redefine it, > so "undefined" it became.) I found the official POSIX rational for the change (not this isn't the same URL that Yuri posted, but a seperate appendex to it) https://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xbd_chap09.html#tag_21_09_03_05 cheers, Jamie From nobody Fri Apr 21 08:16:05 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q2nRG5jkFz465Mr for ; Fri, 21 Apr 2023 08:16:10 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout5-smtp.messagingengine.com (wout5-smtp.messagingengine.com [64.147.123.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q2nRF6GP3z3Ffs for ; Fri, 21 Apr 2023 08:16:09 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b=N9kxeE7A; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=cqYAVuwO; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.21 as permitted sender) smtp.mailfrom=yuri@aetern.org Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 372FD32004AE for ; Fri, 21 Apr 2023 04:16:07 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 21 Apr 2023 04:16:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:message-id:mime-version:reply-to:sender :subject:subject:to:to; s=fm2; t=1682064966; x=1682151366; bh=MX iCo2ZhFnwaCTUHDP/K4SxO2HeOdqFRnZryv2KUiTA=; b=N9kxeE7AZCuMAfq/4Q kO0kvzLysDZ0lYYOFZB51o66cUi2zYu1leJ/ZDx7ZF3wA5rDO5UZJ6kl7AvB1CE8 cqSE0udfp9dGaHn5W35oO27zqQRG34iYI7Klxjc1q+JwUW6K2ZEEcmuRFfR582hg eqXY7xCG61/joe6DxIS+2BlVPpp+HjNF90quLhGOq8xenn4Qh9q6fKw2cNe+6c1w bItDJuF4+NO/lz9zbSJVyfPUU6Z7h5iOXdJghyLfbcgiUrdipOceCL7IWBjViq/A 4dFhC0Y4Td0kGlU1TwOw5QODtkB1yudYnujmr/A6LWPM5ZG9XDgE4NfHkZiNqBko SlzQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:sender:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1682064966; x=1682151366; bh=MXiCo2ZhFnwaC TUHDP/K4SxO2HeOdqFRnZryv2KUiTA=; b=cqYAVuwOR50P2pbHiB2yKCssk/CLm is5Jpt8ZEOtrAPPj5mDiLRHaTf54LU5xPvSmLn8/cD+iih77AjYtEYhhvLa68zjN k4XaQCxbPl0zbVYHdY0p2dJZChqdvtXnb4zfUleRvTK5n33P95cFgi11om0wRTlw TSOCEh9F+IatIAjquWWrTRKtlH4UwgTbJImh1kq8V2jhy2fNbVmULcMqUhezjJ5p uGGgRzcKf43UeXLdresns1zqQcRQMIM82os0A1+7g5Z6hKHpXj4B8LBmFRXsKdtL Cey6M6JzzG3KOmPARCTKPRefa+WgBBXiFqy+Dcn+CJCUChyJkkZNwxiYA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedgtdduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefkffggfgfvhffutgfgsehtjeertd dtfeejnecuhfhrohhmpegjuhhrihcuoeihuhhrihesrggvthgvrhhnrdhorhhgqeenucgg tffrrghtthgvrhhnpeekgfdtffeghfehgeelvdeuveekvdekjeetuefgveehiedtgfekhf evleehvdeljeenucffohhmrghinhepfhhrvggvsghsugdrohhrghenucevlhhushhtvghr ufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeihuhhrihesrggvthgvrhhnrd horhhg X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 04:16:06 -0400 (EDT) Message-ID: <4d4af609-e835-a702-0e7d-1d44fd0b686f@aetern.org> Date: Fri, 21 Apr 2023 10:16:05 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Content-Language: en-US To: current@freebsd.org From: Yuri Subject: adopting zone1970.tab changes in tzsetup Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q2nRF6GP3z3Ffs X-Spamd-Bar: / X-Spamd-Result: default: False [-0.40 / 15.00]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.21]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; PREVIOUSLY_DELIVERED(0.00)[current@freebsd.org]; local_wl_from(0.00)[yuri@aetern.org]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Hi, After tzsetup was switched to use zone1970.tab in 513419f4047 it mostly works, but there are some content changes compared to zone.tab that was used previously which need to be taken care of (would like to do so before 14.0). If you have spare time, please take a look at the reviews below. https://reviews.freebsd.org/D39634 - add baseline to tzsetup to check that zone1970.tab changes don't break the assumptions of file format we make in tzsetup (and tzsetup changes itself don't have unintended effects) https://reviews.freebsd.org/D39606 - adopt zone1970.tab changes - assumption that single-zone countries do not have description is no longer correct; do not try to optimize this case as it's only going to make the code more confusing and we now have menus with a single zone selection because of this - remove the single-country continent short cut, it also only serves to confuse users as we now have such a continent - instead add a single-zone contry short cut (see above), now all single-zone countries fall here - use the @# continent overrides that zone1970.tab introduces (this is visible at least fixing Iceland being currently listed under Africa) - add Arctic Ocean "continent" coming only from the overrides at the moment - update baseline with the changes From nobody Fri Apr 21 10:01:16 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q2qmj4y9lz46C8t for ; Fri, 21 Apr 2023 10:01:25 +0000 (UTC) (envelope-from SRS0=VdIV=AM=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q2qmh3QrLz3LRc for ; Fri, 21 Apr 2023 10:01:24 +0000 (UTC) (envelope-from SRS0=VdIV=AM=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=zS2jS7HB; spf=pass (mx1.freebsd.org: domain of "SRS0=VdIV=AM=klop.ws=ronald-lists@realworks.nl" designates 194.109.157.24 as permitted sender) smtp.mailfrom="SRS0=VdIV=AM=klop.ws=ronald-lists@realworks.nl"; dmarc=pass (policy=quarantine) header.from=klop.ws Date: Fri, 21 Apr 2023 12:01:16 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1682071276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=n5XZPi08GNy2WAgxhTE6VDlO6cWj8N0fwIduz7lzczg=; b=zS2jS7HBKuMF6AFT8bwjOdTScaagG+3Dtwz10eBbZozj0QXcSILiY0AvWLX23Qr7r5cEOX xzwY7WoNj21fB6CufnjOg/nzjXrupZDU+5/UChG+VuPdwC8XLx4dsL+jq48LuqJfWqRQpR jxLaex9NNYqRqeuIaGk2qDROkyGdDrNJkC/snnC7lBdigl4yLtbTJvA2bYDmkinjjythRc CEzAS5zQ4ZsJu3qBdPGYo35k2yglcTy4rVEaoLpf+7E0/3JWbF0pxaqSq9yCqApvMnvFgQ LyQW1IWNKBZxE2RIG+ZtIYI0N45pWdDNbrifRwItQri80Y/zUsIqkRsWOazYDQ== From: Ronald Klop To: Poul-Henning Kamp Cc: current@freebsd.org Message-ID: <564252502.12.1682071276296@mailrelay> In-Reply-To: <202304172106.33HL6RUX051407@critter.freebsd.dk> References: <202304172106.33HL6RUX051407@critter.freebsd.dk> Subject: Re: find(1): I18N gone wild ? List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_11_1668579163.1682071276075" X-Mailer: Realworks (653.239) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Result: default: False [-2.02 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.82)[-0.817]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; MID_RHS_NOT_FQDN(0.50)[]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=VdIV=AM=klop.ws=ronald-lists@realworks.nl]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; R_SPF_ALLOW(-0.20)[+ip4:194.109.157.0/24]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_ZERO(0.00)[0]; MLMMJ_DEST(0.00)[current@freebsd.org]; DKIM_TRACE(0.00)[klop.ws:+]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; HAS_X_PRIO_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL]; TO_MATCH_ENVRCPT_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=VdIV=AM=klop.ws=ronald-lists@realworks.nl] X-Rspamd-Queue-Id: 4Q2qmh3QrLz3LRc X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N ------=_Part_11_1668579163.1682071276075 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable =C3=82=20 Van: Poul-Henning Kamp Datum: maandag, 17 april 2023 23:06 Aan: current@freebsd.org Onderwerp: find(1): I18N gone wild ? >=20 > This surprised me: >=20 > # mkdir /tmp/P > # cd /tmp/P > # touch FOO > # touch bar > # env LANG=3DC.UTF-8 find . -name '[A-Z]*' -print > ./FOO > # env LANG=3Den_US.UTF-8 find . -name '[A-Z]*' -print > ./FOO > ./bar >=20 > Really ?! >=20 > --=20 > Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 > phk@FreeBSD.ORG | TCP/IP since RFC 956 > FreeBSD committer | BSD since 4.3-tahoe > Never attribute to malice what can adequately be explained by incompetenc= e. > =C3=82=20 >=20 >=20 >=20 My Mac and a Linux server only give ./FOO in both cases. Just a 2 cents rem= ark. Regards, Ronald. =C3=82=20 ------=_Part_11_1668579163.1682071276075 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable =C3=82 

Van: Poul-Henning Kamp <phk@phk.freebsd.dk>
Datum: maandag, 17 april 2023 23:06
Aan: current@freebsd.org
Onderwerp: find(1): I18N gone wild ?

This surprised me:

    # mkdir /tmp/P
    # cd /tmp/P
    # touch FOO
    # touch bar
    # env LANG=3DC.UTF-8 find . -name '[A-Z]*' -print     ./FOO
    # env LANG=3Den_US.UTF-8 find . -name '[A-Z]*' -pri= nt
    ./FOO
    ./bar

Really ?!

-- 
Poul-Henning Kamp       | UNIX since Zilog Ze= us 3.20
phk@FreeBSD.ORG         | TCP/IP si= nce RFC 956
FreeBSD committer       | BSD since 4.3-tahoe=
Never attribute to malice what can adequately be explained by incompetence.=
=C3=82 


My Mac and a Linux server only give ./FOO in both cases. Just a 2 cents rem= ark.

Regards,
Ronald.
=C3=82  ------=_Part_11_1668579163.1682071276075-- From nobody Fri Apr 21 10:38:05 2023 X-Original-To: current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q2rbM4XYWz46FC9 for ; Fri, 21 Apr 2023 10:38:23 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q2rbM3w9gz3mM9; Fri, 21 Apr 2023 10:38:23 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682073503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=X1yu9E26lmYrtPNDqXR7V639KnAGd6TPgg3RysP9gBU=; b=ox8cVonDPLT7OIFRWkTh4P8Q9jUei6ZANamfn7m/vnke7Ski8jJlldDu9kMQgtPq8Q+KZw l+3TL4a7WlDpePzs3Qmn6q0+smJWq8xxGgwhdw8J4yVaHYYREfDmEHeqYfwIQ/NaAm74Yq eUCrqOsSF/ZAZlO4VBYh9/kO7ESqrDeU4Ee/r45BAGjn+x5LtWyXRDjUGLbT0TblOOiKV9 l0FbK6lfWp3KFvimamTjKcsC67nM+uKgLf5JKWftsMyu3vzCqm1E89iV2nxFJ6RmPdLrCe PGU81XG17ViC1lIkFHTiS7SaiN6UmaeS0aORlMq95RS4YNdHsgGT7h3sLapMCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1682073503; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=X1yu9E26lmYrtPNDqXR7V639KnAGd6TPgg3RysP9gBU=; b=DKhpr3unnCxXUyeKuMXzt8XuqDotlb4Qg4ZC0ZjfpIGaJZlKQK32ozmJLqEreih+h7jppY ZhVUE+cSDaaCy/xeGRRkYw5kF6hpqiX/5nyIEsuJnfhM3aAj6iNzEz1YNSooiPlG0lweMH fxCmWTWwd/iOUrXEgRavX+g0h02PO5xwI/eSgSfYMPiPOUP5stqFvcoKuK9I2M3cSmoNW3 4mVCzj7nrVCgqO2ucmDSDu6vQexISdv9gqMZ8Z7Fy73EUnz+3HGhqKgkMZgnh2PvYt6Pjy oCnHbYSVX0JHXu5qcD7fqo7gZfSE8FoF7hIHJRF6TwZpLQbqAwU5OQcQDnjrxg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1682073503; a=rsa-sha256; cv=none; b=XkcW2s9YqS/eYGuDiQH9O4N42uoOlUZg1iUyLgIpXysf8wLYMMto/vW1TlGbZg/imoYYpb lgV5X+HJ8vi6I3xBplxktOQrbO0xX+LxAHTE6dzKau0OYSNheQGHFDQQOWGVCrAj8m7VRn 9KBmvjssIh7SYNgkh7fbM7Px8aOAMa91Pe8ktScQjClJQmFzWKpyIpkR24gvjWfX8b+//z FGS5XWnyK9uMqwUbE3OJTbjZZGpBUFFODJ6BUbDuf3QSeXBVPE4iGdVECPV/7VOdap1Ikk 0/VS46e/pfYGjiWY7p83KnAKDmMvMIhonfhvhHR1iA2Tcv+3R9oDTL1cYNllIQ== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4Q2rbM2DBfz19HY; Fri, 21 Apr 2023 10:38:23 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id E1F33685B4; Fri, 21 Apr 2023 12:38:21 +0200 (CEST) Content-Type: multipart/signed; boundary="Apple-Mail=_58B1808E-7CDD-464A-BEDF-DB6D3AC97BF1"; protocol="application/pgp-signature"; micalg=pgp-sha1 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.500.231\)) Subject: Re: find(1): I18N gone wild ? From: Dimitry Andric X-Priority: 3 (Normal) In-Reply-To: <564252502.12.1682071276296@mailrelay> Date: Fri, 21 Apr 2023 12:38:05 +0200 Cc: Poul-Henning Kamp , current@freebsd.org Message-Id: References: <202304172106.33HL6RUX051407@critter.freebsd.dk> <564252502.12.1682071276296@mailrelay> To: Ronald Klop X-Mailer: Apple Mail (2.3731.500.231) X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_58B1808E-7CDD-464A-BEDF-DB6D3AC97BF1 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 21 Apr 2023, at 12:01, Ronald Klop wrote: > Van: Poul-Henning Kamp > Datum: maandag, 17 april 2023 23:06 > Aan: current@freebsd.org > Onderwerp: find(1): I18N gone wild ? > This surprised me: >=20 > # mkdir /tmp/P > # cd /tmp/P > # touch FOO > # touch bar > # env LANG=3DC.UTF-8 find . -name '[A-Z]*' -print > ./FOO > # env LANG=3Den_US.UTF-8 find . -name '[A-Z]*' -print > ./FOO > ./bar >=20 > Really ?! ... > My Mac and a Linux server only give ./FOO in both cases. Just a 2 = cents remark. Same here. However, I have read that with unicode, you should *never* use [A-Z] or [0-9], but character classes instead. That seems to give both files on macOS and Linux with [[:alpha:]]: $ LANG=3Den_US.UTF-8 find . -name '[[:alpha:]]*' -print ./BAR ./foo and only the lowercase file with [[:lower:]]: $ LANG=3Den_US.UTF-8 find . -name '[[:lower:]]*' -print ./foo But on FreeBSD, these don't work at all: $ LANG=3Den_US.UTF-8 find . -name '[[:alpha:]]*' -print $ LANG=3Den_US.UTF-8 find . -name '[[:lower:]]*' -print This is an interesting rabbit hole... :) -Dimitry --Apple-Mail=_58B1808E-7CDD-464A-BEDF-DB6D3AC97BF1 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCZEJnjQAKCRCwXqMKLiCW o1q/AJ9GDBFlhlXhv7jPnhbEdImI8MKrjACfefJ7A7gkn2K2LVHkevKiXtA/7sk= =5KGL -----END PGP SIGNATURE----- --Apple-Mail=_58B1808E-7CDD-464A-BEDF-DB6D3AC97BF1-- From nobody Fri Apr 21 17:41:45 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q320G208Cz46gJs for ; Fri, 21 Apr 2023 17:42:06 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic307-55.consmr.mail.gq1.yahoo.com (sonic307-55.consmr.mail.gq1.yahoo.com [98.137.64.31]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q320D6klcz3LRT for ; Fri, 21 Apr 2023 17:42:04 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=H9rMDLHh; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.64.31 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682098923; bh=zoxGnW09lDDLqxwCT3s3QYAvgdBNpO/YoXcldRGW6bc=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=H9rMDLHhXiro8woJuvCTD8jQnHF5YDYajLpjM5KqA6b4Mn1H+fnqSzUEr/x6zBB6NRP4d8Fzk+CHQBL6114Z/8z/7N2K/J4wmxLQeQrL6aYjFzKvNG82N5IYKGh3A2//7mgfSYf2hPOTfl1335YXP90fJeBdy7vP0ZkzMszajIbLXScKZxfcsrAW4e54PDuAV95ljO1uz4rMv/v1Rxt5b9nctQNX8ruWlZKCCAhsT3JSpsSEl2TtkZaKAjICIBcSqrW4+S8piAAmjvOL4CsANW62n+ybw0DXWMpmsr6DoEhQcfWlKYW00A+pbQb5PIZXbAWQPj2b1/Xk9F5Le/SeLQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682098923; bh=z/+EZGwHYUJe9NAMGRjB69/WN48RvX6vkIKTDAIlbQ8=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=CBNzPIdh7xtEQUnL59979aqDziyPJwMnoAdyI0begdf6x4oFkHA9VtWYLtrqbwGQMZ+AY+tCcU98X7vLgzYDVsCFvb8SeMJss368meCT2H/wihjpfmFG4uzqVPSO6q2lGmlU+cH0Dy3tlJd0cT+uPvjyHmLCWDGdI6yv+QnKW9M9x43CBBK72PeBjdZ5zBoUHly0jD7oIhkueT9m/9GiIdH8t5XGUq36zAM7UyIJiIFKxGM/rctNOGbkgHktTqiF5OXfbBPpuT3p+KlO2XjgIkYXt0s7/g8Et99sFCfh7lJJ+4L1nO8W3MrzATvATP4ICAbMi4fH58NPJCQQ1qHdMA== X-YMail-OSG: n9qZatsVM1nT6526WRcg03dVQaIEv955vRXgl1CtHjbhN3w2glGBSHY.1zHDYL7 5kC00943CnY6XPfm9ubC6md6JBazeyaBJqS0Y2jIL9AdTXqDFpNRD0Cjm75yaVoRGOClavQYtekk 3mJhx2Z1MX0.xuvj8v1R..C6KSzdQGh3ZQO1y5Zer2oIfgAwkpJWzzIVBzWLlUNkoj6S_BWnomA9 0VyqIiujiV4iQ_fK1TWoxI9heXAs_LI2WbbkrGItnXvEiYgTDCk1abatTdtXPzA6YNu0P6iYHtlD BA1n9P8jCYNRYvAvDwnc1tO2BqcWBJ4TTIuirzXGDOqhjvUrfcUgx_GMQz29sGlz4nyu_DK6gUVp CN7Y_Kj4nlY6SqRFzby0JjMiEQ3n5Yw.tB11sNNJnB0SW3yJimXGiMhzr.SXmmJJLSR5R5B4aawK ucYcr9ZMxZfYuvVSdDU_RBcGdvY3M.r0R60o1wsPshUlaaB8hU_ZD625k93dAbOsndhM8tIZbmAO sdlxEoOsJqN8FiW60uLYi3yxbslv8H92CKtjBGwP_msLL7LrLRsMSYOLQ7SN1Y6tZykmUGJq3ttW iFWpay_y5MBGdCbFonDmL.3l9oruY5HJdgouv02cBouz4iJx0mR3hqRyO7OBg0QYBBQ7yWxXo.9L 9OXINeMSbmqU9GtLEESBKdvGPUW4NcxnNFgjGkjZLZouJtnrCQN3bZrSMUjeY4IYdBJ9qa3CqIsb RUyCUqPYJHthXNPxN9nSBM0cvjZr5SG3aA5TCaQmJA_tIEsI1jTKoha70lEnNMtvpsKCnvfjulpP 4exHEo1D9lssnjMtmgg9.oFsmDx1fvSAE3rnmWSHUrIgt288rHx6e56SI8YnesYm90rx3B_C7vwZ q4VnG6NcfJP7FEdyinz77poOobAT473Ia8Fg3tbK.qioZq4JuGHwbKhuRYpMFqSt.Gbjs7Dr9Dhf HNy0VVvHENuMS_i0vDdR9dviIlakb7HpswudXXRoj.cHiXXnQRJ.Fiw3wxvNDlaZMDq7_BwYm7sS 4rrCJe30_W7OvNukVNNZUWkTZ4lzmWlaomzhmgr2B.iMs9JyHY_8d07FYi.BzyTDRyQ1DG8CFy4t jpGXclaNuPA9p8VD20movsaquF_qkAEAFqs8HlkA3_O8vtPN2NhXtGSw2ldIxD_aC4pEQXeO5hUZ k6tSm4bFMkAIkcy6crUx4YSH4F_AALqny46gz4XrQN8IPLWBZ_N328LvlpFPv1q7t8XNWk_WAyMh naclqOMbKuXZREP7CQtwURdGrEy6S6h2KjIrqtBVSbtzl8To8KmvthTMk_BBG3uNpoArS89.bp_0 emNqFTH19ckElo3VCB_6oT4uKPxxqzlApjFkERmiSGT1sDs9sY8Tw1E.rpeYO4pl6hzNG.Mg8GPY Zji3EVIkKv6gFqhryCecNLjRLuUe5ZW5vRJA9Ly5nZQtT6rQcWObNZz9A.r4.5SNSue4sJZB.bR3 GU0fYd_YVfqJ_VNIP1C03ak1ObyNMn4X.cl8LLu9oJfMqNMrn7rNnIw_CjdB3Uwrl7FKy43nlXaC 04geLAW73HJ9gT3ao4YDxEZExPUM8GhmoPtr0NXQGZgUur4Iq0HW5g4jSjLAQo9i_w_.27z4f6al BhiWdEX.5sTILguWkEUMs1NO6PvSnqSKhWGs0phWfUzpaHOcggGHKvNy3uGWgk5qoGv_7fCqMAqq z_bIqqbr8tIYC3koDRqCqLNCIIOL5H0cxynf91IePrJp.PV0XYi.HdFqV_BvMWzpTIb3fnK62REI qo2SUf5KH63VSwxj9lG.mIs6fbpnEJa_MuuYW3j2W4FSKsZfRCAC5Sy7JNnOaXndxCw7Z9fReOCt xHGdmBKQwGlSevFhghVTQo.MWGCK9sPElootu5GfdAmCrRInau6kKPMr2E.0fXFtNIcvkfU0ilmb 01gyWeNThkAXulWjQfSSkWnKT2_1_t_bvB5BwmaxDRg5GaBqrCS9MuqREWloCmcsid_mKt6kecS2 t2qPOeo_KcDopAYDetV7b7eEohmO9UdhIceCF0XQqKUAUjESFFVtPr2xKoHpzPUcZfJr4.gxgpU7 ziaZcK.zG3NnJYqDIQRLsaNAsUPCGFMHiF56xYLPOrcnHxnO67Ly8wHroIxcJGl_LQgUumuXieX_ xOlixp6PqKTr3fdrP0Qz5MHVCgqWISGTTuN5XD_FeFPSvrTzZpAKhwvt_813x8Gp.CCxeTj.g01R 4HJvxv06.FFsZnqBh7iL5WB3_HjvkDmvOajzQ0zsxoAQgiYcObNi2pZLrVRgAdUbmC0.UEWpAih8 - X-Sonic-MF: X-Sonic-ID: bb7358e4-0a06-40fe-a4a6-187e6b861d8c Received: from sonic.gate.mail.ne1.yahoo.com by sonic307.consmr.mail.gq1.yahoo.com with HTTP; Fri, 21 Apr 2023 17:42:03 +0000 Received: by hermes--production-bf1-5f9df5c5c4-fgkgh (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID be2fab7cf84034fb03bb60815cefc56d; Fri, 21 Apr 2023 17:41:58 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: find(1): I18N gone wild ? Message-Id: Date: Fri, 21 Apr 2023 10:41:45 -0700 To: Dimitry Andric , Current FreeBSD X-Mailer: Apple Mail (2.3731.400.51.1.1) References: X-Spamd-Result: default: False [-2.50 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.64.31:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4Q320D6klcz3LRT X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Dimitry Andric wrote on Date: Fri, 21 Apr 2023 10:38:05 UTC : > On 21 Apr 2023, at 12:01, Ronald Klop wrote: > > Van: Poul-Henning Kamp > > Datum: maandag, 17 april 2023 23:06 > > Aan: current@freebsd.org > > Onderwerp: find(1): I18N gone wild ? > > This surprised me: > >=20 > > # mkdir /tmp/P > > # cd /tmp/P > > # touch FOO > > # touch bar > > # env LANG=3DC.UTF-8 find . -name '[A-Z]*' -print > > ./FOO > > # env LANG=3Den_US.UTF-8 find . -name '[A-Z]*' -print > > ./FOO > > ./bar > >=20 > > Really ?! > ... > > My Mac and a Linux server only give ./FOO in both cases. Just a 2 = cents remark. >=20 > Same here. However, I have read that with unicode, you should *never* > use [A-Z] or [0-9], but character classes instead. That seems to give > both files on macOS and Linux with [[:alpha:]]: >=20 > $ LANG=3Den_US.UTF-8 find . -name '[[:alpha:]]*' -print > ./BAR > ./foo >=20 > and only the lowercase file with [[:lower:]]: >=20 > $ LANG=3Den_US.UTF-8 find . -name '[[:lower:]]*' -print > ./foo >=20 > But on FreeBSD, these don't work at all: >=20 > $ LANG=3Den_US.UTF-8 find . -name '[[:alpha:]]*' -print > >=20 > $ LANG=3Den_US.UTF-8 find . -name '[[:lower:]]*' -print > >=20 > This is an interesting rabbit hole... :) FreeBSD: -name pattern True if the last component of the pathname being examined = matches pattern. Special shell pattern matching characters = (=E2=80=9C[=E2=80=9D, =E2=80=9C]=E2=80=9D, =E2=80=9C*=E2=80=9D, and =E2=80=9C?=E2=80=9D) may be used = as part of pattern. These characters may be matched explicitly by escaping them with a backslash (=E2=80=9C\=E2=80=9D). I conclude that [[:alpha:]] and [[:lower:]] were not considered "Special shell pattern"s. "man glob" indicates it is a shell specific builtin. macOS says similarly. Different shells, different pattern notations and capabilities? Well, "man bash" reports: QUOTE Pattern Matching . . . Within [ and ], character classes can be specified using = the syntax [:class:], where class is one of the following classes = defined in the POSIX standard: alnum alpha ascii blank cntrl digit graph lower print = punct space upper word xdigit A character class matches any character belonging to that = class. The word character class matches letters, digits, and the = character _. Within [ and ], an equivalence class can be specified = using the syntax [=3Dc=3D], which matches all characters with the same = collation weight (as defined by the current locale) as the character c. Within [ and ], the syntax [.symbol.] matches the = collating symbol symbol. END QUOTE "man zsh" does not document patterns but: sh-3.2$ echo $SHELL /bin/zsh sh-3.2$ find . -name '[[:lower:]]*' -print ./bar % ls -Tldt /bin/*sh -r-xr-xr-x 1 root wheel 1326688 Feb 9 01:39:53 2023 /bin/bash -rwxr-xr-x 2 root wheel 1153216 Feb 9 01:39:53 2023 /bin/csh -rwxr-xr-x 1 root wheel 307232 Feb 9 01:39:53 2023 /bin/dash -r-xr-xr-x 1 root wheel 2598864 Feb 9 01:39:53 2023 /bin/ksh -rwxr-xr-x 1 root wheel 134000 Feb 9 01:39:53 2023 /bin/sh -rwxr-xr-x 2 root wheel 1153216 Feb 9 01:39:53 2023 /bin/tcsh -rwxr-xr-x 1 root wheel 1377616 Feb 9 01:39:53 2023 /bin/zsh But in each, even bash, % echo $SHELL /bin/zsh With "find" not being part of the kernel, Linux may have a number of variations across the operating systems. Picking one . . . openSUSE tumbleweed: -name pattern Base of file name (the path with the leading directories = removed) matches shell pattern pattern. Because the leading directories = are removed, the file names considered for a match with -name will never include a slash, so `-name a/b' will = never match anything (you probably need to use -path instead). A = warning is issued if you try to do this, unless the en- vironment variable POSIXLY_CORRECT is set. The = metacharacters (`*', `?', and `[]') match a `.' at the start of the base = name (this is a change in findutils-4.2.2; see section STAN- DARDS CONFORMANCE below). To ignore a directory and the = files under it, use -prune rather than checking every file in the tree; = see an example in the description of that action. Braces are not recognised as being special, despite the = fact that some shells including Bash imbue braces with a special meaning = in shell patterns. The filename matching is per- formed with the use of the fnmatch(3) library function. = Don't forget to enclose the pattern in quotes in order to protect it = from expansion by the shell. "man 3 fnmatch" says: The fnmatch() function checks whether the string argument matches = the pattern argument, which is a shell wildcard pattern (see glob(7)). "man 7 glob" (not shell specific) in turn has a section on "Character classes and internationalization" that reports: QUOTE . . . . . . Therefore, POSIX extended the bracket notation greatly, both for wildcard patterns and for regular expressions. In = the above we saw three types of items that can occur in a bracket = expression: namely (i) the negation, (ii) explicit single characters, and (iii) ranges. POSIX specifies ranges in an = internationally more useful way and adds three more types: (iii) Ranges X-Y comprise all characters that fall between X and = Y (inclusive) in the current collating sequence as defined by the = LC_COLLATE category in the current locale. (iv) Named character classes, like [:alnum:] [:alpha:] [:blank:] [:cntrl:] [:digit:] [:graph:] [:lower:] [:print:] [:punct:] [:space:] [:upper:] [:xdigit:] so that one can say "[[:lower:]]" instead of "[a-z]", and have = things work in Denmark, too, where there are three letters past 'z' in = the alphabet. These character classes are defined by the LC_CTYPE category in the current locale. (v) Collating symbols, like "[.ch.]" or "[.a-acute.]", where the = string between "[." and ".]" is a collating element defined for the = current locale. Note that this may be a multicharacter element. (vi) Equivalence class expressions, like "[=3Da=3D]", where the = string between "[=3D" and "=3D]" is any collating element from its = equivalence class, as defined for the current locale. For exam- ple, "[[=3Da=3D]]" might be equivalent to "[a=C3=A1=C3=A0=C3=A4=C3=A2= ]", that is, to "[a[.a-acute.][.a-grave.][.a-umlaut.][.a-circumflex.]]". END QUOTE # file /usr/bin/sh /usr/bin/sh: symbolic link to bash Seems like: pick your shell (as shown by echo $SHELL) and that picks the pattern match rules used. (May be controllable in the specific shell.) =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Apr 21 18:03:30 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q32T23kR4z46hVT for ; Fri, 21 Apr 2023 18:03:34 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q32T21DwPz4DDZ for ; Fri, 21 Apr 2023 18:03:34 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b=FnMIhnQj; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="J rygpY4"; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=yuri@aetern.org Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 6CC503200ADC for ; Fri, 21 Apr 2023 14:03:32 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 21 Apr 2023 14:03:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1682100212; x=1682186612; bh=4GD0021YdgyLmrPbfUpHTEE2zecUXgjcrot NBS4LFCA=; b=FnMIhnQjNvm9DE0SZmb8V8Aa93j4YniJCzXjzSWKgLAy0hxlsO2 Wd+ZMZZqETD+YfAkLF/H+1OPWh0fyR01J1vQ+Xtsmn+tqs31QlF2ESD1gt8AVc2W c+mVZH9XcjSM0AP6WLD2TojygczzANb2x4wlenRD7QAHLDHa8EiAFMxpCOBGm7n3 1wH4wbRKcMtHGH57kFBb+ds2pPqcQG6rV7/oozIwnbTR0biTkRp5MM4z+awPgVdx +WpE41TRrAnUJrWOl+xiXzlwppZ/EkrZgKOiabDmBiVXVrefW9U1q5qt2qO7rJm9 QFgvSzpD+rMiJjZWnLsehzgZr0UMS99b/xQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1682100212; x= 1682186612; bh=4GD0021YdgyLmrPbfUpHTEE2zecUXgjcrotNBS4LFCA=; b=J rygpY4oPgLkKeeklZMBxmPwbZZpI6u+9oOn4xuL+vP0SCcK97Y5VD4fkwt4wmNrv 56jCcKeH3cHTaZ4C+31wujvIrAd0oa4HljssmodGzGAQR884VX42gdQQ0mwyiP93 nrunAbV5nttwUL/jpSkeu48ZgrxSme28KTdiUEz++UJN4sdFJjsK3YJMu5gIhVya xiPAG9mct7TI7A5fsBb+glyB+ImTmvnewHgFUmruxY7O45E2FfraLKW9hNVSZSXd PJldbgEpCfYDSS1QUAKclqLou5S24WeGDkfGEBtgKJxk4jssnlt0ahdoqrNjfJE+ DehYNWF2fv915EF/XW6DQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedguddukecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enfghrlhcuvffnffculddufedmnecujfgurhepkfffgggfuffvfhfhjggtgfesthekredt tdefjeenucfhrhhomhepjghurhhiuceohihurhhisegrvghtvghrnhdrohhrgheqnecugg ftrfgrthhtvghrnhepjeegtddvfeejieeutedvtdeikeffhfeuheejuedtheffveduteel geehhffgudefnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgpdhophgvnhhgrhhouh hprdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhephihurhhisegrvghtvghrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 14:03:31 -0400 (EDT) Message-ID: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> Date: Fri, 21 Apr 2023 20:03:30 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild ? Content-Language: en-US To: Current FreeBSD References: From: Yuri In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Q32T21DwPz4DDZ X-Spamd-Bar: / X-Spamd-Result: default: False [0.60 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; local_wl_from(0.00)[yuri@aetern.org]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Mark Millard wrote: > Dimitry Andric wrote on > Date: Fri, 21 Apr 2023 10:38:05 UTC : > >> On 21 Apr 2023, at 12:01, Ronald Klop wrote: >>> Van: Poul-Henning Kamp >>> Datum: maandag, 17 april 2023 23:06 >>> Aan: current@freebsd.org >>> Onderwerp: find(1): I18N gone wild ? >>> This surprised me: >>> >>> # mkdir /tmp/P >>> # cd /tmp/P >>> # touch FOO >>> # touch bar >>> # env LANG=C.UTF-8 find . -name '[A-Z]*' -print >>> ./FOO >>> # env LANG=en_US.UTF-8 find . -name '[A-Z]*' -print >>> ./FOO >>> ./bar >>> >>> Really ?! >> ... >>> My Mac and a Linux server only give ./FOO in both cases. Just a 2 cents remark. >> >> Same here. However, I have read that with unicode, you should *never* >> use [A-Z] or [0-9], but character classes instead. That seems to give >> both files on macOS and Linux with [[:alpha:]]: >> >> $ LANG=en_US.UTF-8 find . -name '[[:alpha:]]*' -print >> ./BAR >> ./foo >> >> and only the lowercase file with [[:lower:]]: >> >> $ LANG=en_US.UTF-8 find . -name '[[:lower:]]*' -print >> ./foo >> >> But on FreeBSD, these don't work at all: >> >> $ LANG=en_US.UTF-8 find . -name '[[:alpha:]]*' -print >> >> >> $ LANG=en_US.UTF-8 find . -name '[[:lower:]]*' -print >> >> >> This is an interesting rabbit hole... :) > > FreeBSD: > > -name pattern > True if the last component of the pathname being examined matches > pattern. Special shell pattern matching characters (“[”, “]”, > “*”, and “?”) may be used as part of pattern. These characters > may be matched explicitly by escaping them with a backslash > (“\”). > > I conclude that [[:alpha:]] and [[:lower:]] were not > considered "Special shell pattern"s. "man glob" > indicates it is a shell specific builtin. > > macOS says similarly. Different shells, different > pattern notations and capabilities? Well, "man bash" > reports: [snip] > Seems like: pick your shell (as shown by echo $SHELL) and > that picks the pattern match rules used. (May be controllable > in the specific shell.) No, the pattern is not passed to shell and shell used should not matter (pattern should be properly escaped). The rules are here: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 ...which in turn refers to the following link for bracket expressions: https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05 Why we don't support all of that is different story. From nobody Fri Apr 21 18:18:21 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q32p83yWWz46jFL for ; Fri, 21 Apr 2023 18:18:24 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q32p81Xjhz3LtS for ; Fri, 21 Apr 2023 18:18:24 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b="gd9QHFu/"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="Q fEP9ff"; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=yuri@aetern.org; dmarc=none Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 5721B3200B0A for ; Fri, 21 Apr 2023 14:18:23 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 21 Apr 2023 14:18:23 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1682101102; x=1682187502; bh=fExMugTqZlDaB3MvxNo5lZ2T0vSpRq6HXLT mNgnDniA=; b=gd9QHFu/QU/M7TBDeLAZjSHIURt8kQMKhB485M5jdx7UY4S0Qzx kWTNNx2iN1rGLQigAche4EHfyxH7dikKJX1S18mSwx+ZFW25U7nJz5ybTO1er1Tv FKCNL64xu7zL5DCTAH7qb4DLuVZW30X8AmUMcqFIyJFSSqMb8OjhBVx7RgksdfTj 15Gv0eVICbUA1Sg0/jl5f5uvU+XFhvf2MvI7WLbrFNYNoac1MED+PMfxvD7Pyr7I CP93oYTYLXIjuPWjU9tDkA22PTDWEudaPwEvsmNS0juD3VtJRMYsPzrFPx/I3X35 HjqpUw8eZ9LPPXRas1/MvXhejaWHDMIEqeA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1682101102; x= 1682187502; bh=fExMugTqZlDaB3MvxNo5lZ2T0vSpRq6HXLTmNgnDniA=; b=Q fEP9ffbSwhzln/NCoiISZr/Kuod2jQCtQ0HsKGNROxKpE/bxRGakivSGSojMwDcs Dwmfbewrg2oe8d9VMHFwa7Ne1lFlJrmHvBSO0JXVGK8NndrW13/ARuRx6iJXkm4c OjyoMroal7b2yBJ7DrhVEW0qFfnwY61F94bQeYZYWni53oN9oTIY6CPrEJwE/HBI 7fGI+Qc2lPSu+4K1E1Tjb6IkpD9PkdkXq3truoRtzIfP9z/hXO9/cr/rZL9Cb9+H LtaUuaHMe/hv36KUwxYIkKB97YIWjvAhzYX8Mka31hJpWwCEUuBjdxxoqoN3/1cj fbReLOE0QUw1L75N0BOTw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedguddvudcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enfghrlhcuvffnffculddufedmnecujfgurhepkfffgggfuffhvfhfjggtgfesthekredt tdefjeenucfhrhhomhepjghurhhiuceohihurhhisegrvghtvghrnhdrohhrgheqnecugg ftrfgrthhtvghrnhepffejgeduuedvueekfeeghffgteekhefhgfegkedtvdejiedtffdu ieekleehvdefnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgpdhophgvnhhgrhhouh hprdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhho mhephihurhhisegrvghtvghrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 14:18:22 -0400 (EDT) Message-ID: Date: Fri, 21 Apr 2023 20:18:21 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild ? Content-Language: en-US From: Yuri To: Current FreeBSD References: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> In-Reply-To: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4Q32p81Xjhz3LtS X-Spamd-Bar: / X-Spamd-Result: default: False [-0.50 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19:c]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[aetern.org]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; local_wl_from(0.00)[yuri@aetern.org]; ARC_NA(0.00)[] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Yuri wrote: > Mark Millard wrote: >> Dimitry Andric wrote on >> Date: Fri, 21 Apr 2023 10:38:05 UTC : >> >>> On 21 Apr 2023, at 12:01, Ronald Klop wrote: >>>> Van: Poul-Henning Kamp >>>> Datum: maandag, 17 april 2023 23:06 >>>> Aan: current@freebsd.org >>>> Onderwerp: find(1): I18N gone wild ? >>>> This surprised me: >>>> >>>> # mkdir /tmp/P >>>> # cd /tmp/P >>>> # touch FOO >>>> # touch bar >>>> # env LANG=C.UTF-8 find . -name '[A-Z]*' -print >>>> ./FOO >>>> # env LANG=en_US.UTF-8 find . -name '[A-Z]*' -print >>>> ./FOO >>>> ./bar >>>> >>>> Really ?! >>> ... >>>> My Mac and a Linux server only give ./FOO in both cases. Just a 2 cents remark. >>> >>> Same here. However, I have read that with unicode, you should *never* >>> use [A-Z] or [0-9], but character classes instead. That seems to give >>> both files on macOS and Linux with [[:alpha:]]: >>> >>> $ LANG=en_US.UTF-8 find . -name '[[:alpha:]]*' -print >>> ./BAR >>> ./foo >>> >>> and only the lowercase file with [[:lower:]]: >>> >>> $ LANG=en_US.UTF-8 find . -name '[[:lower:]]*' -print >>> ./foo >>> >>> But on FreeBSD, these don't work at all: >>> >>> $ LANG=en_US.UTF-8 find . -name '[[:alpha:]]*' -print >>> >>> >>> $ LANG=en_US.UTF-8 find . -name '[[:lower:]]*' -print >>> >>> >>> This is an interesting rabbit hole... :) >> >> FreeBSD: >> >> -name pattern >> True if the last component of the pathname being examined matches >> pattern. Special shell pattern matching characters (“[”, “]”, >> “*”, and “?”) may be used as part of pattern. These characters >> may be matched explicitly by escaping them with a backslash >> (“\”). >> >> I conclude that [[:alpha:]] and [[:lower:]] were not >> considered "Special shell pattern"s. "man glob" >> indicates it is a shell specific builtin. >> >> macOS says similarly. Different shells, different >> pattern notations and capabilities? Well, "man bash" >> reports: > [snip] >> Seems like: pick your shell (as shown by echo $SHELL) and >> that picks the pattern match rules used. (May be controllable >> in the specific shell.) > > No, the pattern is not passed to shell and shell used should not matter > (pattern should be properly escaped). The rules are here: > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 > > ...which in turn refers to the following link for bracket expressions: > > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05 > > Why we don't support all of that is different story. A bit more on this; first link applies both to find(1) and fnmatch(3), and find uses fnmatch() internally (which is good), but even the function that processes bracket expressions is called rangematch() and that's really all it does ignoring other bracket expression rules: https://cgit.freebsd.org/src/tree/lib/libc/gen/fnmatch.c#n234 So to "fix" find we just need to implement the bracket expressions properly in fnmatch(). From nobody Fri Apr 21 19:04:29 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q33qY23pHz46l4n for ; Fri, 21 Apr 2023 19:04:41 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q33qX6pk6z3spQ for ; Fri, 21 Apr 2023 19:04:40 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: X-Catflap-Envelope-To: freebsd-current@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 33LJ4T8l044066; Fri, 21 Apr 2023 20:04:29 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 33LJ4TA3044065; Fri, 21 Apr 2023 20:04:29 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202304211904.33LJ4TA3044065@donotpassgo.dyslexicfish.net> Date: Fri, 21 Apr 2023 20:04:29 +0100 Organization: Dyslexic Fish To: yuri@aetern.org, freebsd-current@FreeBSD.org Subject: Re: find(1): I18N gone wild ? References: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Fri, 21 Apr 2023 20:04:29 +0100 (BST) X-Rspamd-Queue-Id: 4Q33qX6pk6z3spQ X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Yuri wrote: > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 > > > > ...which in turn refers to the following link for bracket expressions: > > > > https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05 > > > > Why we don't support all of that is different story. > > A bit more on this; first link applies both to find(1) and fnmatch(3), > and find uses fnmatch() internally (which is good), but even the > function that processes bracket expressions is called rangematch() and > that's really all it does ignoring other bracket expression rules: > > https://cgit.freebsd.org/src/tree/lib/libc/gen/fnmatch.c#n234 > > So to "fix" find we just need to implement the bracket expressions > properly in fnmatch(). No. find "-name" works with GLOBs not regex. The functionality you are talking about, and the page you quoted refer to regular expressions. For that, you use find "-regex" - note that the regex is assumed anchored, so: $ LANG=en_US.UTF-8 find -E /etc/rc.d -regex '.*[[:upper:]]+' -print /etc/rc.d/NETWORKING /etc/rc.d/FILESYSTEMS /etc/rc.d/SERVERS /etc/rc.d/DAEMON /etc/rc.d/LOGIN $ Jamie From nobody Fri Apr 21 19:12:11 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q340G2CX1z46m7Y for ; Fri, 21 Apr 2023 19:12:14 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q340G0zNWz465J for ; Fri, 21 Apr 2023 19:12:14 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: X-Catflap-Envelope-To: freebsd-current@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 33LJCBY0044167; Fri, 21 Apr 2023 20:12:11 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 33LJCBaq044166; Fri, 21 Apr 2023 20:12:11 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202304211912.33LJCBaq044166@donotpassgo.dyslexicfish.net> Date: Fri, 21 Apr 2023 20:12:11 +0100 Organization: Dyslexic Fish To: yuri@aetern.org, freebsd-current@FreeBSD.org Subject: Re: find(1): I18N gone wild ? References: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Fri, 21 Apr 2023 20:12:11 +0100 (BST) X-Rspamd-Queue-Id: 4Q340G0zNWz465J X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N My mistake. Character classes are indeed part of glob https://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html 2.13.3 Patterns Used for Filename Expansion Sorry for the noise. From nobody Fri Apr 21 19:12:40 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q340v1sdbz46lxL for ; Fri, 21 Apr 2023 19:12:47 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q340t4sYYz47lf for ; Fri, 21 Apr 2023 19:12:46 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b=lKxr2FXx; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="G wqSPkj"; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=yuri@aetern.org Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id C0C8632002E8 for ; Fri, 21 Apr 2023 15:12:42 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 21 Apr 2023 15:12:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1682104362; x=1682190762; bh=yS5MNIFZxlDDzUxsWOi/LcdeHn7WK7YtnTm B4c5vJSM=; b=lKxr2FXx/Ljwqh/qk/1u/MUBTrC2GRFLKTnMoaPs5SY+ACRdDtp ssDxAIbqPJatrrylQNkFKYU0GOhdTrDwAc0qiA3bsFkHAk8z4Vj3XiJ0WBW1q7kC Vc2ntLgzbNMio9PxCAQcTiQN7NoyT3MOCJMiDhjBKUDxDbmPF+iGbioH/QhxdNRb K8rhAWrcDbpfKLTPzsUhU6LuNJRLRNpoOTY3u1Z8DDkv9X6DYJkj6j8GqC/tJ7au KxwbEIGgkmNR1Qca9JF/Sq6DxmjZ8MT2yxtnMBVsC4jRHP4ppPBq7wc0X5QegD3B XjMuSJfVtRB19GYhCZhtCMOiP0GFQSU6Mzg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1682104362; x= 1682190762; bh=yS5MNIFZxlDDzUxsWOi/LcdeHn7WK7YtnTmB4c5vJSM=; b=G wqSPkj/SMWKcFh/SednxyBhOa8ey3RsrZspqwDrHygf70JGJM+b+D+bXc3FDR6Jb S2gFUr1WSwXpFv3pX+FrQI97MQ9T7Aceg70QQZ99qGyLcFBs5vgdcumxFXE6N7p7 wHOfZ7nWbtbacnVGyqHIiEBea7L0y9s0lMwWR0OZ5Dq1vepbaOZSpfXslUyV8nrs dalnhJVMpempiZNfI4rzKqTdx7F6y/+T94XEPWWc+EZz+du4TNIAfJd8+wwjEbqC lOk1gj0iZh8A4w/6v0DVWthdEHbmMKmICm5VN6g++3zembzFM/Axdv8zF4HfZZs2 YZFk8zhUw+HiAaCy+7VHg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedgudeffecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffvfhfhjggtgfesth ejredttdefjeenucfhrhhomhepjghurhhiuceohihurhhisegrvghtvghrnhdrohhrgheq necuggftrfgrthhtvghrnheptdfhvedtgeetgfeiveduteeiheduueevgfekffdtffejle ehleduvddugedvjeelnecuffhomhgrihhnpehophgvnhhgrhhouhhprdhorhhgpdhfrhgv vggsshgurdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilh hfrhhomhephihurhhisegrvghtvghrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 15:12:41 -0400 (EDT) Message-ID: Date: Fri, 21 Apr 2023 21:12:40 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild ? Content-Language: en-US To: freebsd-current@FreeBSD.org References: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> <202304211904.33LJ4TA3044065@donotpassgo.dyslexicfish.net> From: Yuri In-Reply-To: <202304211904.33LJ4TA3044065@donotpassgo.dyslexicfish.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q340t4sYYz47lf X-Spamd-Bar: / X-Spamd-Result: default: False [0.60 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; local_wl_from(0.00)[yuri@aetern.org]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Jamie Landeg-Jones wrote: > Yuri wrote: > >>> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_13 >>> >>> ...which in turn refers to the following link for bracket expressions: >>> >>> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05 >>> >>> Why we don't support all of that is different story. >> >> A bit more on this; first link applies both to find(1) and fnmatch(3), >> and find uses fnmatch() internally (which is good), but even the >> function that processes bracket expressions is called rangematch() and >> that's really all it does ignoring other bracket expression rules: >> >> https://cgit.freebsd.org/src/tree/lib/libc/gen/fnmatch.c#n234 >> >> So to "fix" find we just need to implement the bracket expressions >> properly in fnmatch(). > > No. find "-name" works with GLOBs not regex. No, find "-name" works with pattern rules in the first link, please see: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html > The functionality you are talking about, and the page you quoted refer to regular expressions. The functionality I am talking about is defined in find specification above, the only relation to REs is for bracket expressions (with ! instead of ^). > For that, you use find "-regex" - note that the regex is assumed anchored, so: > > $ LANG=en_US.UTF-8 find -E /etc/rc.d -regex '.*[[:upper:]]+' -print > /etc/rc.d/NETWORKING > /etc/rc.d/FILESYSTEMS > /etc/rc.d/SERVERS > /etc/rc.d/DAEMON > /etc/rc.d/LOGIN > $ > > Jamie > From nobody Fri Apr 21 19:14:21 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q343z6zDKz46mQv for ; Fri, 21 Apr 2023 19:15:27 +0000 (UTC) (envelope-from parv.0zero9@gmail.com) Received: from mail-yb1-xb2e.google.com (mail-yb1-xb2e.google.com [IPv6:2607:f8b0:4864:20::b2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q343y1d9Xz4G6H for ; Fri, 21 Apr 2023 19:15:26 +0000 (UTC) (envelope-from parv.0zero9@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20221208 header.b=MvZJ30e5; spf=pass (mx1.freebsd.org: domain of parv.0zero9@gmail.com designates 2607:f8b0:4864:20::b2e as permitted sender) smtp.mailfrom=parv.0zero9@gmail.com; dmarc=pass (policy=none) header.from=gmail.com Received: by mail-yb1-xb2e.google.com with SMTP id 3f1490d57ef6-b8ed0fa7546so2752853276.3 for ; Fri, 21 Apr 2023 12:15:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1682104525; x=1684696525; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=Sg9uJISx5LTcMGP+Aq8K86/X0tLzgpSPhZaihTAp6Mc=; b=MvZJ30e54GBsoENvvBeHbmeUfuPRINvowscGSOkHfM/MkG3AA7Qp1uw7P3nzJSzXfc y9ZluGbpwR3WOtWPQ+QgsrlyqYSXGKOHBO/x6W+9fz0Gx3nwYAFnpyjTsG4l75s17r1+ O5E+c3Nkxhd0IQmtOgZzinCtLVAHCCF5IdN1qAg9xW+WN6z6vK1yKkmMpo3m11D+ahMF t3OXPELqkNh3Gi0zm2KuzaAMAb7+sJhmLBjsIDGvrBvVw8+FKgVRAKwEwsOomG/u5gRO eCXib4078Eycwv4H80E97gnY0NXjCYKuXrR3An6tVFi7DdeoU9E/BX3bw8tHY5Vl6dvr BpAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682104525; x=1684696525; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Sg9uJISx5LTcMGP+Aq8K86/X0tLzgpSPhZaihTAp6Mc=; b=QY5Nh6SSQNf0Jb3fYENvQiAs7pjEgTV3l9R07IE+y4h+wHX3IVmhzraYa8PEonoS57 OyqnBUL00PQoaajXjGvHn9K/lJ/IQBfzI/ipWWZLYhMIbCs0IaTXHExR5oPz0/E8Ijj9 JAJA95UlhnHihHCFjsd4pSgHJJC9ZPYmoWajBog4QWVrpXa5+gptTsWhkNdX4F24/NQg n27b/8+cnlcynjbzicU+1dcr8Gei/2jjJaNo18BIW2o7QxSaF5LLgPaO+//D6b1MOKGV fPK4zRA+grNk+BZwFYQjGGWQWZmNVw4rDJbkQ0VIvceDeDZswgI2mNIxRxZUZ/yPLBUJ dhcA== X-Gm-Message-State: AAQBX9cDU0vgBm609/S1vn0GDSKoBcD+atcC+3HQZ97r42vjSCpfOKWa CujhNffhx1hiZVGxMKtpuxY1pvH7yiBd7j6IMCueXI1ESlkegg== X-Google-Smtp-Source: AKy350bDz5bP9hkjxJFrHmdAqcYX7/7vG0H48wEcfJRuTyhCABazl/ob8qfldCctFfsu/RuUz+4MKhhNlZdxGpXYAo4= X-Received: by 2002:a0d:dd93:0:b0:555:d4ad:8067 with SMTP id g141-20020a0ddd93000000b00555d4ad8067mr2370893ywe.17.1682104525285; Fri, 21 Apr 2023 12:15:25 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 From: "parv/FreeBSD" Date: Fri, 21 Apr 2023 09:14:21 -1000 Message-ID: Subject: Re: find(1): I18N gone wild? [[:alpha:]] not a substitute to refer 26 English letters A-Z To: freebsd-current@freebsd.org Content-Type: multipart/alternative; boundary="000000000000b46d6805f9dd7a33" X-Spamd-Result: default: False [-3.00 / 15.00]; FAKE_REPLY(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.997]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; SUBJECT_HAS_QUESTION(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::b2e:from]; BLOCKLISTDE_FAIL(0.00)[2607:f8b0:4864:20::b2e:server fail]; ARC_NA(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; DKIM_TRACE(0.00)[gmail.com:+]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; TO_DN_NONE(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TAGGED_FROM(0.00)[freebsd]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4Q343y1d9Xz4G6H X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N --000000000000b46d6805f9dd7a33 Content-Type: text/plain; charset="UTF-8" Wrote Dimitry Andric on Fri, 21 Apr 2023 10:38:05 UTC (via https://lists.freebsd.org/archives/freebsd-current/2023-April/003556.html ) > > ... However, I have read that with unicode, you should *never* > use [A-Z] or [0-9], but character classes instead. That seems to give > both files on macOS and Linux with [[:alpha:]]: ... Subject to the locale, problem with that is "[[:alpha:]]" will match more than 26 English letters "A" through "Z" (besides also matching lower case "a" through "z") even if none of 26 * 2 English alphabets appear in a string. - parv --000000000000b46d6805f9dd7a33 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Wrote=20 Dimitry Andric on=20 Fri, 21 Apr 2023 10:38:05 UTC
(via https://lists.freebsd.org/archives/fre= ebsd-current/2023-April/003556.html )
>
> ... However, I have read that with un= icode, you should *never*
> use [A-Z] or [0-9], but character classes instead. Th= at seems to give
> both files on macOS and Linux with [[:alpha:]]:
...

Subject to the locale, problem w= ith that is "[[:alpha:]]" will match
more than 26 English letters "A&= quot; through "Z" (besides also matching
lower case "a" through = "z") even if none of 26 * 2 English alphabets
appear in a string.


- parv

--000000000000b46d6805f9dd7a33-- From nobody Fri Apr 21 19:18:16 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q347H2hgLz46mYj for ; Fri, 21 Apr 2023 19:18:19 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:7400:8808:123::1]) by mx1.freebsd.org (Postfix) with ESMTP id 4Q347H1FqFz4Lkb for ; Fri, 21 Apr 2023 19:18:19 +0000 (UTC) (envelope-from jamie@catflap.org) Authentication-Results: mx1.freebsd.org; none X-Catflap-Envelope-From: X-Catflap-Envelope-To: freebsd-current@FreeBSD.org Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [209.250.224.51]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 33LJIGiW044267; Fri, 21 Apr 2023 20:18:16 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 33LJIGLc044266; Fri, 21 Apr 2023 20:18:16 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202304211918.33LJIGLc044266@donotpassgo.dyslexicfish.net> Date: Fri, 21 Apr 2023 20:18:16 +0100 Organization: Dyslexic Fish To: yuri@aetern.org, freebsd-current@FreeBSD.org Subject: Re: find(1): I18N gone wild ? References: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> <202304211904.33LJ4TA3044065@donotpassgo.dyslexicfish.net> In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [209.250.224.51]); Fri, 21 Apr 2023 20:18:16 +0100 (BST) X-Rspamd-Queue-Id: 4Q347H1FqFz4Lkb X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:7400::/38, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N Yuri wrote: > No, find "-name" works with pattern rules in the first link, please see: > > https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html Yeah. *blush* Sorry about that. I had thought character classes were for regular expressions only. /gets coat From nobody Fri Apr 21 19:24:09 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q34G4311Yz46mss for ; Fri, 21 Apr 2023 19:24:12 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q34G418Csz4X8m for ; Fri, 21 Apr 2023 19:24:12 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b=Nby6JTaV; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="J FgJyI4"; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=yuri@aetern.org; dmarc=none Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailout.west.internal (Postfix) with ESMTP id 219C33200AF6 for ; Fri, 21 Apr 2023 15:24:11 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Fri, 21 Apr 2023 15:24:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1682105050; x=1682191450; bh=RHqoCLXwePvBlTbX5eLOq7So5XMWSGwDEsb usYLNHRw=; b=Nby6JTaVAv9lPFrfFLWv7o7lXK1bphAEnfWh2fFtSHhiuP1wIH6 rENmJkVs1uSJv29o03+TlBW3ZLwC2Et8K1XLPQTTvQLKdwIloh/ku7fAGZxdMbBp J1O28jDLg8CA0cZPrsq18WpmL//mqdr/9JZBka22VecXLgrrwZEJrznLbRwPD9oL wXrIHb74PEne200PnShf+a0IgLu8DDn2RBKCJOU+OKqynMwqGHvF9kx2hdsm6l0b 7JaFwPULmK/P97ZV8BbMiwDeuc1qkBGH7pv6V47kEX9n6yqK795nfndxFTosqBJ6 +n57R4PbIHhS+xXxFZVHhGwdmqllZBcNEAA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1682105050; x= 1682191450; bh=RHqoCLXwePvBlTbX5eLOq7So5XMWSGwDEsbusYLNHRw=; b=J FgJyI4ttzAkHGEb3qZ6h+Sn2OMFGoapVOSBAQctAbFTq9cwpIVz+zPbi6mWEQ5DC E9/TKaD8Ee0oOWEqyTxdkaqourzOCNyNSsR6twxpXy4bCtKeOwXxPFjHfJ4ifvhu O6gnl78cE8FDK0Yluh2rLk3UqSVZB1qa7PPUhhmq4xiFYT3b436kthoAQm0xz3dB w+6vnX0HxcCXKVFeO82XjdMw2OuNlXHV+RlLEYgFHDKKRlTvEJTm7vqPWiiDU/vY 0C2NoYeUO1Qds4xJBCPSkJZMiJSaJCS/p4KzneTi3LTmN2sgIJj52cNu+AVJbg0z rpM4TqwgsOiLbL6U/vuhQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedgudefhecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffvfhfhjggtgfesth ejredttdefjeenucfhrhhomhepjghurhhiuceohihurhhisegrvghtvghrnhdrohhrgheq necuggftrfgrthhtvghrnhepgeehgeeiheejffegteevjeevfefgkeefveduudelkedvie ejvedttedtgffhvdevnecuffhomhgrihhnpehophgvnhhgrhhouhhprdhorhhgnecuvehl uhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephihurhhisegrvg htvghrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 15:24:10 -0400 (EDT) Message-ID: Date: Fri, 21 Apr 2023 21:24:09 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild ? Content-Language: en-US To: freebsd-current@FreeBSD.org References: <3e473603-f384-f176-e7cb-03409e16ec9c@aetern.org> <202304211904.33LJ4TA3044065@donotpassgo.dyslexicfish.net> <202304211918.33LJIGLc044266@donotpassgo.dyslexicfish.net> From: Yuri In-Reply-To: <202304211918.33LJIGLc044266@donotpassgo.dyslexicfish.net> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q34G418Csz4X8m X-Spamd-Bar: / X-Spamd-Result: default: False [-0.50 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19:c]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[aetern.org]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; local_wl_from(0.00)[yuri@aetern.org]; ARC_NA(0.00)[] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Jamie Landeg-Jones wrote: > Yuri wrote: > >> No, find "-name" works with pattern rules in the first link, please see: >> >> https://pubs.opengroup.org/onlinepubs/9699919799/utilities/find.html > > Yeah. *blush* Sorry about that. I had thought character classes were > for regular expressions only. No worries, I too am trying to understand the problem completely, which comes in form of (too many) links :-) From nobody Fri Apr 21 19:36:05 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q34Ws1Z9fz46n9L for ; Fri, 21 Apr 2023 19:36:09 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q34Wr62MZz3RCR for ; Fri, 21 Apr 2023 19:36:08 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b="k/H4idCF"; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="D qAcU1Q"; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=yuri@aetern.org; dmarc=none Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 8628C3200B54 for ; Fri, 21 Apr 2023 15:36:07 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 21 Apr 2023 15:36:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1682105767; x=1682192167; bh=7EEtz4ZOueefrcScFWdKQ3cn48j9B8qp9ik 4yruZQfY=; b=k/H4idCFDv7HPhEm1JdipoKDatl5lzdcLjxhL5bbP/5yHJaB6Bz KOFnnzDiEf8eejjlNyVYaX3L8Zvpj4xBtwfxMA3dgvW4cOapms0FSD2bftkzV5tE Imlq9MA0gb2BlfERktqLQmZRlFnKB7sSFqduF8n/LIgnP9C/XqnlUupnL93S2Mgz lK6LXHsyQhaZw3bXzIBXOfBIb1jLTgRO5O2tqhk/X57FDuYLyPvZsOfHy3WS307m /m32T9fJbs2BS8SufD2qBZqfCMe4mkg1uKXShE5aNcesJKcVrDt2xJHAKZxGpgja pBp58suLcAkn+GnG3pY2wB5W6wyjo9EYG8w== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1682105767; x= 1682192167; bh=7EEtz4ZOueefrcScFWdKQ3cn48j9B8qp9ik4yruZQfY=; b=D qAcU1QXK0EsodR3XX+Dkz+neHa/rUUOgyztyU7e57wt/qMB51eQ8e6LzlKI6mks6 c2MGuM+/hw4nDZxJv690+dgoW4BufhYkwzl7lCLIuRd0yqJnI5sITixoknkP/1pA /EWP9YwROwfcOnFLVwoD05p5gXWt3DupvdPncMGF26X3JAvJCui+CyjHdKtTmfgY VFyohUhjrIy+yiz8cYe3SP3fT2e3fxFAxaTOKTUYEcRC/HygzTEsebWLZ10KuC7G Z8qA6dMUeNhDA8dCc2RUCCRlO+uASAOhaPX63Kl7cRDiUcRuG4R/nfZ43OHE94iV Q/kM2ZV6s7jkjJ+HoLIZw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedgudefkecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffvfhfhjggtgfesth ejredttdefjeenucfhrhhomhepjghurhhiuceohihurhhisegrvghtvghrnhdrohhrgheq necuggftrfgrthhtvghrnhepgeehleevueehhffggfetteffieffhfduteduteeuvdehvd fhffdvtefhffejjedvnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephihurhhisegrvghtvg hrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 15:36:06 -0400 (EDT) Message-ID: <86efedcf-e3ed-be0c-79ab-03f0d4a743af@aetern.org> Date: Fri, 21 Apr 2023 21:36:05 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild? [[:alpha:]] not a substitute to refer 26 English letters A-Z Content-Language: en-US To: freebsd-current@freebsd.org References: From: Yuri In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q34Wr62MZz3RCR X-Spamd-Bar: - X-Spamd-Result: default: False [-1.50 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19:c]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; ARC_NA(0.00)[]; DMARC_NA(0.00)[aetern.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; local_wl_from(0.00)[yuri@aetern.org]; SUBJECT_HAS_QUESTION(0.00)[] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N parv/FreeBSD wrote: > Wrote Dimitry Andric on Fri, 21 Apr 2023 10:38:05 UTC > (via > https://lists.freebsd.org/archives/freebsd-current/2023-April/003556.html ) >> >> ... However, I have read that with unicode, you should *never* >> use [A-Z] or [0-9], but character classes instead. That seems to give >> both files on macOS and Linux with [[:alpha:]]: > ... > > Subject to the locale, problem with that is "[[:alpha:]]" will match > more than 26 English letters "A" through "Z" (besides also matching > lower case "a" through "z") even if none of 26 * 2 English alphabets > appear in a string. (replying to random recent message) And there is a bit of quite recent history for fnmatch() related to [a-z], same was done for regex with the same outcome -- attempt to make [a-z] (guess [A-Z] as well) range non-collating failed. I am not aware of the encountered failures, hopefully someone should remember: -------- commit 5a5807dd4ca34467ac5fb458bc19f12bf62075a5 Author: Andrey A. Chernov Date: Sun Jul 10 03:49:38 2016 +0000 Remove broken support for collation in [a-z] type ranges. Only first 256 wide chars are considered currently, all other are just dropped from the range. Proper implementation require reverse tables database lookup, since objects are really big as max UTF-8 (1114112 code points), so just the same scanning as it was for 256 chars will slow things down. POSIX does not require collation for [a-z] type ranges and does not prohibit it for non-POSIX locales. POSIX require collation for ranges only for POSIX (or C) locale which is equal to ASCII and binary for other chars, so we already have it. No other *BSD implements collation for [a-z] type ranges. Restore ABI compatibility with unused now __collate_range_cmp() which is visible from outside (will be removed later). -------- commit 1daad8f5ad767dfe7896b8d1959a329785c9a76b Author: Andrey A. Chernov Date: Thu Jul 14 08:18:12 2016 +0000 Back out non-collating [a-z] ranges. Instead of changing whole course to another POSIX-permitted way for consistency and uniformity I decide to completely ignore missing regex fucntionality and concentrace on fixing bugs in what we have now, too many small obstacles instead, counting ports. -------- commit 12eae8c8f346cb459a388259ca98faebdac47038 Author: Andrey A. Chernov Date: Thu Jul 14 09:07:25 2016 +0000 1) Eliminate possibility to call __*collate_range_cmp() with inclomplete locale (which cause core dump) by removing whole 'table' argument by which it passed. 2) Restore __collate_range_cmp() in __sccl(). 3) Collating [a-z] range in regcomp() only for single bytes locales (we can't do it now for other ones). In previous state only first 256 wchars are considered and all others are just silently dropped from the range. -------- From nobody Fri Apr 21 19:51:55 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q34tQ0fs4z46pQy for ; Fri, 21 Apr 2023 19:52:14 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-21.consmr.mail.gq1.yahoo.com (sonic310-21.consmr.mail.gq1.yahoo.com [98.137.69.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q34tN4wjZz4CS9 for ; Fri, 21 Apr 2023 19:52:12 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=r5lpY+bJ; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.147 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682106730; bh=t4Z5XG/3uEunjp+HwWi8wENjFHNUic9wH5UbWuf0VrY=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=r5lpY+bJ3I75sB7w7ZIVO1g1rRBTWEp+23lBDbzZGZHiTN7MV2YHs6hatGUTG5/8A5k7OHwGVGDSSQqakzA6wFE7CFgkhd1ubO6m0DPLFD/cst76MAMXkGw/KSJBF0e9yNYqi6J/+5adsRqhDmp7BCkI4ts7Ve7F0PPUEsDJFrV2aNSZNCi/Cst/eYhDxoxFdIIyxNttEfmlZAr8MAAmTGh+6Tf9oZE84BJ4NYPiKvYrHoU5xh7ignuAocE8wP9IgoE8CpiEkpeWUotek37hP/dLGjtZOgAcfFRE0r+p0lJt6bjqnI7FLvohKny/0W0jaqPNHqMyRNvFZwcQJt8eeQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1682106730; bh=RUPj5vVjowWg2AS/nFVX/rbXQGQBhBpph8x5Shxa7K6=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=ZMpiDON+74GMKlsvgAziah/p0JbjPzjFdsCQ4d1i0z2x0pXwKjAujGKYfisZ/RMytibysZRmM9uixUP5nIe3Ma4kw0Z/+dGhuLfpSfXzSLS56no/qBNjBJhdrjCcmEZp1NWn0tYuOHtzlDZrqHrVT99TWCOROBU99dBuwZ7/QlMFA8TxiHtOb49Pd4XTwJYfZ1KYqvMFbjEkqKe4dDl5wJQ7tooWxUfS7w2gxV5aY1dpbFbkebIri+58icYfbnPPoQBEs5gLT4kAkVG3Tqs86pIdtHtIOSWxqd+lFvD911tRZ6hRUNxuWH2iTmYWIoWUxGWCxu0QGkAFKUPkkcvAqA== X-YMail-OSG: dCQ3USIVM1mT9S0D790wyM.y2ku71YlGMQllJgx03PM9kdgJrmXZm76PZ_GKQrf VFCi824jx.TxGFEqxm8ibR0aDuP28MKrQM0TF7uU9whn8QJEW6A9zOI1CDiQOx2p1oaBLEeBbRPU tH6n7_z2VliYtTX2wbmWZHrC6dL3E1APjJPlfymPDwJEwu.LuBv27A_2UZSlf3Xx4yGUEXWtsSfc PRD2lKYS7mC92A521iVN7nC_AgfzjZV9PYIFtvl2JUl2SxlJSLwUqyQZ..ESAgiqrp.Ctb0I6Hoc 7bvKwLXf0vZkYphHJNp7VL2bb8FRHoWEzMSivy7lv5Mwfjar0Bnw1dpIMt9vM0ofFChvS__Q1Opu 1Js4sZexq9y3xbgY.XNkW2JbPhi1xm7tDkPk8l7xQ7WSWkcQ3A5VqLzz_PKCDO_frRHfbiLK.mBv gnQH0qV26s1sAhpH99oA14KBstqqFuXKOUlohJ6OC5sZWkBpiPqE_uxIa2wH9zP2A1HVtsxM4sBu wTrtAdIGVXxQqC8mAt0JbWRPYhHdw4zq0M91arnLWvgK02B2mxhj8_KClatvH31WA0xKKQWxn0mC IV9ULGbq3ysthlVi5trXzrrPT4xMvigVUsq3xUb5PZA.rOF9w9Pg6OMir0zYoFoSaOBgr6BB73sD ct1U1Vc6MmkPvPgv3Z1PLXh6Rrb2Efs1.uV31ZwujBlSr8OQ2FGbytfe8333V9A89LF8Sgkfd_H4 i5_52_5eSinZQN83TWvJO1hdqtSTGOuqZKUYmbwVEau8TrglsWX1MT60t_mGVp8hBK_E_o.ROQJ4 UFsysmoQbdd6P85aIXfOkapBnLD4F9RT4qthbwapWC3jSTrocE9Eqfhrwu5wb7HxkaMRbbFUK.3A A852uFvGkQGAgpHMd_k9TMe3md2_JRwUpzbSX8YOwvSPDAc2VhvV86Pu.6NcSBikuwHSvbXkWrRO 7YbubuNGSny442pP0ajgq31LQH5l0ilQbuDWwu5lQasYFPWDYctdBiqSW6nnE6yRostKr4j1WwTk toSXhLnbHIxDYvMGE7igM7qV4Rrq1w73d25Y26NEgD.8_mNews9wH.sLaz44qsS7kcSjU5h1ZOsI q3vdBWXPnu9nUeIIq8H2U6yu9CZ5J3ElSgdpmiJg1dm4X7FIPTlSIEQgc5zuRz9y3vz4CmoFV8ZL OUxuU0JTQO6R5Uz9zxHSFvY6W6ZLOWfERFwAOO2kvjWQIehApqzOzLm.jHS4ciKqAdxz84sLvlSa J8LwO3bIxOfJnrmMyTlw6SBRfOVZTvGf6yPgVTsAjpI8m_Wd89PlSciJboHqPug1jpf76qTK82f6 B82vzQFajW1wwfl3nQO9Y9t0aEqfkgbt.79ZZP2HoegkmJhEAfPUTZctNB4jnpxlhGPaYFF5Jl2C RF39VLnNYM_akrNyC.MpAIGpX6Hi1Ks84BpqL.9hF8DwUUe6uRmvo.uf19AFHsiMRW1Fwn3B0kDX iKn8BbS2cGdQDqO76KcU3ExsQTwh2bnIsQO0xocC_oSevtNpVnI6d79rzxTAiHxVTS3pJpGLCcZq vUpW9fpjpek6ZPL19s4U8PzeW2eBxYRQG2hISTX02WXSzhpHlb1g3n08JZXV7IK8ubmAm75ma8v0 Fy3xtwPeACKMWF37aLruk69NTmTzCxCu3rKOakapKmBgP1C2UrsTLWpmfJy5TFoHK5g4nCG_NJjg hxGNLFRPK2A8kpytoaE6E_UAJ3CEmzm2ZamzGGG1DZVinhlpYYVMewXQ1obhWq66xCx39QQ75KiM ocizq4EIT_Dj1Y5jc4eWWuuOduqsiR7U9idcFETchA_7cDoSDWPv0hno8rj07mw_EL7ei_u_CDBn VfBAIFSWeo6rr9yYnJ07a42V1TtJjavG1BVtKYSxXr9stGbNxY.flzUQ4VDv0w2ZNbEiCubgW1CB 6OCL7FN_klSjO6HJt9.Um2TB9lslEKoRPI1lyxxcSJYxG0nbbAmyx2p5l10ZNLKmcd8Ae4HAcxwx z1Mv9.gIl5jbwnKwy8UbhsTfQiw7ttOya.jBray_5D_Ttnyp8IlqDSpz0MVE8o61fG6JlpG8Eg8L 0_L68yh9UJQHa8JEJZqTzZelJh.LrsncZ3s9ZavBmDVDhpRS5jSFQnNscXpiJ1UG7xnnJY0vziDc 69h2U4Lgm7wWOCLiU5H8sDz1E.PY.75Ss9KrHwXCjg.vwVgL3gcTOlltm4o9AMUK3MUHzEqlLeNQ UZ7eTVJGHG4s8MY1OHoFwPFPp.Ns6QUwmERGXOIJE.Wut2AxncelFXE4Y6bIIv7lAH7p5p7CUMhc 8.pU- X-Sonic-MF: X-Sonic-ID: 25606138-0e5e-402f-916b-aa83f62199bb Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Fri, 21 Apr 2023 19:52:10 +0000 Received: by hermes--production-gq1-546798879c-9kfxl (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 48da965ea590d1257b1e96cba063fabe; Fri, 21 Apr 2023 19:52:06 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.400.51.1.1\)) Subject: Re: find(1): I18N gone wild ? Message-Id: Date: Fri, 21 Apr 2023 12:51:55 -0700 To: Yuri , Current FreeBSD X-Mailer: Apple Mail (2.3731.400.51.1.1) References: X-Spamd-Result: default: False [-2.49 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.995]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.147:from]; TO_DN_ALL(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org] X-Rspamd-Queue-Id: 4Q34tN4wjZz4CS9 X-Spamd-Bar: -- X-ThisMailContainsUnwantedMimeParts: N Yuri wrote on Date: Fri, 21 Apr 2023 18:18:21 UTC : > Yuri wrote: > > Mark Millard wrote: > >> Dimitry Andric wrote on > >> Date: Fri, 21 Apr 2023 10:38:05 UTC : > >> > >>> On 21 Apr 2023, at 12:01, Ronald Klop = wrote: > >>>> Van: Poul-Henning Kamp > >>>> Datum: maandag, 17 april 2023 23:06 > >>>> Aan: current@freebsd.org > >>>> Onderwerp: find(1): I18N gone wild ? > >>>> This surprised me: > >>>> > >>>> # mkdir /tmp/P > >>>> # cd /tmp/P > >>>> # touch FOO > >>>> # touch bar > >>>> # env LANG=3DC.UTF-8 find . -name '[A-Z]*' -print > >>>> ./FOO > >>>> # env LANG=3Den_US.UTF-8 find . -name '[A-Z]*' -print > >>>> ./FOO > >>>> ./bar > >>>> > >>>> Really ?! > >>> ... > >>>> My Mac and a Linux server only give ./FOO in both cases. Just a 2 = cents remark. > >>> > >>> Same here. However, I have read that with unicode, you should = *never* > >>> use [A-Z] or [0-9], but character classes instead. That seems to = give > >>> both files on macOS and Linux with [[:alpha:]]: > >>> > >>> $ LANG=3Den_US.UTF-8 find . -name '[[:alpha:]]*' -print > >>> ./BAR > >>> ./foo > >>> > >>> and only the lowercase file with [[:lower:]]: > >>> > >>> $ LANG=3Den_US.UTF-8 find . -name '[[:lower:]]*' -print > >>> ./foo > >>> > >>> But on FreeBSD, these don't work at all: > >>> > >>> $ LANG=3Den_US.UTF-8 find . -name '[[:alpha:]]*' -print > >>> > >>> > >>> $ LANG=3Den_US.UTF-8 find . -name '[[:lower:]]*' -print > >>> > >>> > >>> This is an interesting rabbit hole... :) > >> > >> FreeBSD: > >> > >> -name pattern > >> True if the last component of the pathname being examined matches > >> pattern. Special shell pattern matching characters (=E2=80=9C[=E2=80=9D= , =E2=80=9C]=E2=80=9D, > >> =E2=80=9C*=E2=80=9D, and =E2=80=9C?=E2=80=9D) may be used as part = of pattern. These characters > >> may be matched explicitly by escaping them with a backslash > >> (=E2=80=9C\=E2=80=9D). > >> > >> I conclude that [[:alpha:]] and [[:lower:]] were not > >> considered "Special shell pattern"s. "man glob" > >> indicates it is a shell specific builtin. > >> > >> macOS says similarly. Different shells, different > >> pattern notations and capabilities? Well, "man bash" > >> reports: > > [snip] > >> Seems like: pick your shell (as shown by echo $SHELL) and > >> that picks the pattern match rules used. (May be controllable > >> in the specific shell.) > >=20 > > No, the pattern is not passed to shell and shell used should not = matter > > (pattern should be properly escaped). The rules are here: > >=20 > > = https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#= tag_18_13 > >=20 > > ...which in turn refers to the following link for bracket = expressions: > >=20 > > = https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#t= ag_09_03_05 > >=20 > > Why we don't support all of that is different story. >=20 > A bit more on this; first link applies both to find(1) and fnmatch(3), > and find uses fnmatch() internally (which is good), but even the > function that processes bracket expressions is called rangematch() and > that's really all it does ignoring other bracket expression rules: >=20 > https://cgit.freebsd.org/src/tree/lib/libc/gen/fnmatch.c#n234 >=20 > So to "fix" find we just need to implement the bracket expressions > properly in fnmatch(). Too bad the -name documentation does not track this but points to shell notation. The following confirms that even for the IEEE Std 1003.1-2001 that FreeBSD's find is documented to be based on, the notations that you reference were indicated. FreeBSD's man page reports: STANDARDS The find utility syntax is a superset of the syntax specified by = the IEEE Std 1003.1-2001 (=E2=80=9CPOSIX.1=E2=80=9D) standard. All the single character options except -H and -L as well as -amin, -anewer, -cmin, -cnewer, -delete, -empty, -fstype, -iname, -inum, -iregex, -ls, -maxdepth, -mindepth, -mmin, -not, -path, -print0, = -regex, -sparse and all of the -B* birthtime related primaries are = extensions to IEEE Std 1003.1-2001 (=E2=80=9CPOSIX.1=E2=80=9D). . . . IEEE Std 1003.1-2001 find looks to be at: https://pubs.opengroup.org/onlinepubs/009604499/utilities/find.html -name pattern The primary shall evaluate as true if the basename of the = filename being examined matches pattern using the pattern matching = notation described in Pattern Matching Notation. = https://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html#= tag_02_13 [ The open bracket shall introduce a pattern bracket expression. The description of basic regular expression bracket expressions in the = Base Definitions volume of IEEE Std 1003.1-2001, Section 9.3.5, RE = Bracket Expression shall also apply to the pattern bracket expression, = https://pubs.opengroup.org/onlinepubs/009604499/basedefs/xbd_chap09.html#t= ag_09_03_05 =E2=80=A2 A character class expression shall represent the union of = two sets: =E2=80=A2 The set of single-character collating elements whose = characters belong to the character class, as defined in the LC_CTYPE = category in the current locale. =E2=80=A2 An unspecified set of multi-character collating = elements. All character classes specified in the current locale shall be = recognized. A character class expression is expressed as a character = class name enclosed within bracket-colon ( "[:" and ":]" ) delimiters. The following character class expressions shall be supported in all = locales: [:alnum:] [:cntrl:] [:lower:] [:space:] [:alpha:] [:digit:] [:print:] [:upper:] [:blank:] [:graph:] [:punct:] [:xdigit:] In addition, character class expressions of the form: [:name:] are recognized in those locales where the name keyword has been given a = charclass definition in the LC_CTYPE category. =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Fri Apr 21 20:05:55 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q35BH046yz46q3B for ; Fri, 21 Apr 2023 20:05:59 +0000 (UTC) (envelope-from yuri@aetern.org) Received: from wout3-smtp.messagingengine.com (wout3-smtp.messagingengine.com [64.147.123.19]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q35BG4DV5z3JRt for ; Fri, 21 Apr 2023 20:05:58 +0000 (UTC) (envelope-from yuri@aetern.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=aetern.org header.s=fm2 header.b=Q1OIa+6G; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="W /7nCeO"; spf=pass (mx1.freebsd.org: domain of yuri@aetern.org designates 64.147.123.19 as permitted sender) smtp.mailfrom=yuri@aetern.org; dmarc=none Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 7DA233200B45 for ; Fri, 21 Apr 2023 16:05:57 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Fri, 21 Apr 2023 16:05:57 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=aetern.org; h=cc :content-transfer-encoding:content-type:content-type:date:date :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:sender:subject:subject:to:to; s=fm2; t= 1682107557; x=1682193957; bh=kfjEBZW0PtxJPAH9rWjX/8d85E34Xqx3tNS Oqk9eZKg=; b=Q1OIa+6G1qPmIc2wdC50e2l1Zw9/s2/S7cGL+auyEY5bDdyS6Hr xBz7AztCtCJwScTuboWiKXTbgxvlf/o6MITFd5YlczKwePAI4EP0yhw4lpfXsEj6 Cf/2RyuZyQzoOt2ZkjJoV407lmGe1ScX+Zd9QnuU8+Y1PlhCLqWKOqVGu3oRlAAA e+nhOdeRGJdPei+vvpZl+4sK6lQOMyiP5Au13H87OIeL/kkxVqyC6b1pfmEFxB3I sUZ6kdtt6NyngsnWls0fzOfX7FjSQ/BBDkWQh1Bo+fFkMGPpv9aSpBNciLWgiSHe N+kJu4vzr59JOHKuFgGDh4jaK6sRgZGxA2g== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-transfer-encoding:content-type :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm3; t=1682107557; x= 1682193957; bh=kfjEBZW0PtxJPAH9rWjX/8d85E34Xqx3tNSOqk9eZKg=; b=W /7nCeOd9+EbxJkOdCV5Jb9jyjI4Oq6xsc+vFD6lKDzEx2h+8ZCiiDcLo5YFS4p1Y 5Crjr+4l4rvJyV1WqquWZxPC6ZtSHYscsCKajzzhYocGEFQx8vAaO1wF4dzumEUu BJsqZfTfUHtzyOpTQOZorrl9uB+7CnGtLUkirwjm/3ZVPgCXTn/2HKiT67ZlWtnz nNMtXp8nAJSSX+cEEG3yfGZzoJSbffM2cMrtEI5Agz6i/6HKNqlYxKfGBA1N5Qs/ dRxKkw4JYqUsA8Wd789R5litSaHzq7V4LWLxO5W2zM4n4756Kq66rHa7kf5csZus H++5cOvLNl9csgshQ1Gqg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvhedrfedtgedgudeggecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecunecujfgurhepkfffgggfuffhvfhfjggtgfesth ejredttdefjeenucfhrhhomhepjghurhhiuceohihurhhisegrvghtvghrnhdrohhrgheq necuggftrfgrthhtvghrnhepudejueefudfhfeeffeektdduheffgeegjeehveejleffhf ekfefhhfeiteeihfetnecuffhomhgrihhnpehfrhgvvggsshgurdhorhhgnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhephihurhhisegrvghtvg hrnhdrohhrgh X-ME-Proxy: Feedback-ID: i0d79475b:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA for ; Fri, 21 Apr 2023 16:05:56 -0400 (EDT) Message-ID: Date: Fri, 21 Apr 2023 22:05:55 +0200 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101 Thunderbird/102.10.0 Subject: Re: find(1): I18N gone wild? [[:alpha:]] not a substitute to refer 26 English letters A-Z Content-Language: en-US From: Yuri To: freebsd-current@freebsd.org References: <86efedcf-e3ed-be0c-79ab-03f0d4a743af@aetern.org> In-Reply-To: <86efedcf-e3ed-be0c-79ab-03f0d4a743af@aetern.org> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 4Q35BG4DV5z3JRt X-Spamd-Bar: - X-Spamd-Result: default: False [-1.50 / 15.00]; DWL_DNSWL_LOW(-1.00)[messagingengine.com:dkim]; R_SPF_ALLOW(-0.20)[+ip4:64.147.123.19:c]; R_DKIM_ALLOW(-0.20)[aetern.org:s=fm2,messagingengine.com:s=fm3]; RCVD_IN_DNSWL_LOW(-0.10)[64.147.123.19:from]; DKIM_TRACE(0.00)[aetern.org:+,messagingengine.com:+]; ARC_NA(0.00)[]; DMARC_NA(0.00)[aetern.org]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; ASN(0.00)[asn:29838, ipnet:64.147.123.0/24, country:US]; local_wl_from(0.00)[yuri@aetern.org]; SUBJECT_HAS_QUESTION(0.00)[] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: local_wl_from X-ThisMailContainsUnwantedMimeParts: N Yuri wrote: > parv/FreeBSD wrote: >> Wrote Dimitry Andric on Fri, 21 Apr 2023 10:38:05 UTC >> (via >> https://lists.freebsd.org/archives/freebsd-current/2023-April/003556.html ) >>> >>> ... However, I have read that with unicode, you should *never* >>> use [A-Z] or [0-9], but character classes instead. That seems to give >>> both files on macOS and Linux with [[:alpha:]]: >> ... >> >> Subject to the locale, problem with that is "[[:alpha:]]" will match >> more than 26 English letters "A" through "Z" (besides also matching >> lower case "a" through "z") even if none of 26 * 2 English alphabets >> appear in a string. > > (replying to random recent message) > > And there is a bit of quite recent history for fnmatch() related to > [a-z], same was done for regex with the same outcome -- attempt to make > [a-z] (guess [A-Z] as well) range non-collating failed. I am not aware > of the encountered failures, hopefully someone should remember: I just tried less intrusive change that seems to help with these ranges (but there's still a question what failed previously): diff --git a/lib/libc/gen/fnmatch.c b/lib/libc/gen/fnmatch.c index 40670545993..3234c1aaaa4 100644 --- a/lib/libc/gen/fnmatch.c +++ b/lib/libc/gen/fnmatch.c @@ -295,10 +295,11 @@ rangematch(const char *pattern, wchar_t test, int flags, char **newp, if (flags & FNM_CASEFOLD) c2 = towlower(c2); - if (table->__collate_load_error ? + if (table->__collate_load_error || + iswascii(test) ? c <= test && test <= c2 : - __wcollate_range_cmp(c, test) <= 0 - && __wcollate_range_cmp(test, c2) <= 0 + __wcollate_range_cmp(c, test) <= 0 && + __wcollate_range_cmp(test, c2) <= 0 ) ok = 1; } else if (c == test) $ LC_ALL=en_US.UTF-8 LD_PRELOAD=/usr/obj/home/yuri/ws/find/amd64.amd64/lib/libc/libc.so.7 find . -name '[a-z]*' ./bar $ LC_ALL=en_US.UTF-8 LD_PRELOAD=/usr/obj/home/yuri/ws/find/amd64.amd64/lib/libc/libc.so.7 find . -name '[A-Z]*' ./FOO > -------- > commit 5a5807dd4ca34467ac5fb458bc19f12bf62075a5 > Author: Andrey A. Chernov > Date: Sun Jul 10 03:49:38 2016 +0000 > > Remove broken support for collation in [a-z] type ranges. > Only first 256 wide chars are considered currently, all other are just > dropped from the range. Proper implementation require reverse tables > database lookup, since objects are really big as max UTF-8 (1114112 > code points), so just the same scanning as it was for 256 chars will > slow things down. > > POSIX does not require collation for [a-z] type ranges and does not > prohibit it for non-POSIX locales. POSIX require collation for ranges > only for POSIX (or C) locale which is equal to ASCII and binary for > other chars, so we already have it. > > No other *BSD implements collation for [a-z] type ranges. > > Restore ABI compatibility with unused now __collate_range_cmp() which > is visible from outside (will be removed later). > -------- > commit 1daad8f5ad767dfe7896b8d1959a329785c9a76b > Author: Andrey A. Chernov > Date: Thu Jul 14 08:18:12 2016 +0000 > > Back out non-collating [a-z] ranges. > Instead of changing whole course to another POSIX-permitted way > for consistency and uniformity I decide to completely ignore missing > regex fucntionality and concentrace on fixing bugs in what we have now, > too many small obstacles instead, counting ports. > -------- > commit 12eae8c8f346cb459a388259ca98faebdac47038 > Author: Andrey A. Chernov > Date: Thu Jul 14 09:07:25 2016 +0000 > > 1) Eliminate possibility to call __*collate_range_cmp() with inclomplete > locale (which cause core dump) by removing whole 'table' argument > by which it passed. > > 2) Restore __collate_range_cmp() in __sccl(). > > 3) Collating [a-z] range in regcomp() only for single bytes locales > (we can't do it now for other ones). In previous state only first 256 > wchars are considered and all others are just silently dropped from the > range. > -------- >