From nobody Mon Nov 6 00:34:32 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SNsmr1TSGz50GRn for ; Mon, 6 Nov 2023 00:34:36 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SNsmq4fj7z3RV8 for ; Mon, 6 Nov 2023 00:34:35 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; dkim=none; spf=softfail (mx1.freebsd.org: 213.133.107.7 is neither permitted nor denied by domain of mm@FreeBSD.org) smtp.mailfrom=mm@FreeBSD.org; dmarc=none Received: from sslproxy06.your-server.de ([78.46.172.3]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1qznZa-000Cdw-8q for freebsd-stable@freebsd.org; Mon, 06 Nov 2023 01:34:34 +0100 Received: from [188.167.171.2] (helo=[10.0.9.225]) by sslproxy06.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1qznZZ-000331-4e for freebsd-stable@freebsd.org; Mon, 06 Nov 2023 01:34:33 +0100 Message-ID: <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> Date: Mon, 6 Nov 2023 01:34:32 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Martin Matuska Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? To: FreeBSD-STABLE Mailing List References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> Content-Language: en-US In-Reply-To: <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.10/27083/Sun Nov 5 08:43:13 2023) X-Spamd-Result: default: False [-1.25 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.99)[-0.987]; NEURAL_HAM_SHORT(-0.17)[-0.171]; MIME_GOOD(-0.10)[text/plain]; XM_UA_NO_VERSION(0.01)[]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; TO_DN_ALL(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE]; ARC_NA(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; R_SPF_SOFTFAIL(0.00)[~all:c]; FREEFALL_USER(0.00)[mm]; RCVD_COUNT_TWO(0.00)[2]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_X_AS(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_TLS_ALL(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DMARC_NA(0.00)[freebsd.org]; TO_DOM_EQ_FROM_DOM(0.00)[] X-Rspamd-Queue-Id: 4SNsmq4fj7z3RV8 X-Spamd-Bar: - OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can work with pools that have feature@block_cloning enabled. The sysctl variable vfs.zfs.bclone_enabled affects the behavior of zfs_clone_range() which is called by copy_file_range(). When it is set to 0, zfs_clone_range() does not do block cloning. If it is set to anything else than 0, zfs_clone_range() does block cloning (if all conditions are met - same ZFS pool, correct data alignment, etc.). In FreeBSD-main, this tunable is enabled and I plan to enable it in stable/14 somewhere around December 11, 2023. As of today I personally use block cloning on all my systems. mm On 04/11/2023 13:35, Mark Millard wrote: > On Nov 4, 2023, at 04:38, Mike Karels wrote: > >> On 4 Nov 2023, at 4:01, Ronald Klop wrote: >> >>> On 11/4/23 02:39, Mark Millard wrote: >>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: >>>> >>>> int zfs_bclone_enabled; >>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, >>>> &zfs_bclone_enabled, 0, "Enable block cloning"); >>>> >>>> leaving block cloning effectively disabled by default, no >>>> matter what the pool has enabled. >>>> >>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: >>>> >>>> QUOTE >>>> OpenZFS has been upgraded to version 2.2. New features include: >>>> • >>>> block cloning, which allows shallow copies of blocks in file >>>> copies. This is optional, and disabled by default; it can be >>>> enabled with sysctl vfs.zfs.bclone_enabled=1. >>>> END QUOTE >>>> >>> >>> I think this answers your question in the subject. >> I think so too (and I wrote that text). > Thanks for the confirmation of the final intent. > > I believe this makes: > > QUOTE > author Brian Behlendorf 2023-05-25 20:53:08 +0000 > committer GitHub 2023-05-25 20:53:08 +0000 > commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) > tree dd01dfce6aeef357ade1775acf18aade535c6271 > . . . > Update compatibility.d files > > Add an openzfs-2.2 compatibility file for the next release. Edon-R > support has been enabled for FreeBSD removing the need for different > FreeBSD and Linux files. Symlinks for the -linux and -freebsd names > are created for any scripts expecting that convention. Additionally, a > symlink for ubunutu-22.04 was added. Signed-off-by: Brian Behlendorf > Closes #14833 > END QUOTE > > technically incorrect in that compatibility.d/openzfs-2.2-freebsd > should be distinct in content from compatibility.d/openzfs-2.2 so > that block cloning would not be enabled. > > >>>> Just curiousity on my part about the default completeness of >>>> openzfs-2.2 support, not an objection either way. >>>> >>> >>> I haven't seen new issues with block cloning in the last few weeks >>> mentioned on the mailing lists. All known issues are fixed AFAIK. >>> But I can imagine that the risk+effect ratio of data corruption is >>> seen as a bit too high for a 14.0 release for this particular >>> feature. That does not diminish the rest of the completeness of >>> openzfs-2.2. >>> >>> NB: I'm not involved in developing openzfs or the decision making in >>> the release. Just repeating what I read on the lists. >> There was another block cloning fix in 14.0-RC4; see the commit log. >> Maybe there will be no more issues, but it seems that corner cases were >> still being found recently. > Looks like I'll stay at openzfs-2.1 pool features until there is > a release that no longer has the default status: > > 0 for sysctl vfs.zfs.bclone_enabled > > I use main [so: 15 now] but only enable openzfs-2.* pool features > supported by default on some FreeBSD release, that has an accurate > compatibility.d/openzfs-2.*-freebsd file. > > === > Mark Millard > marklmi at yahoo.com > > From nobody Mon Nov 6 00:43:50 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SNszv04cwz50JbK for ; Mon, 6 Nov 2023 00:44:11 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SNszs3hHnz3TR0 for ; Mon, 6 Nov 2023 00:44:09 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=dPoAwVog; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699231447; bh=ElsRvfQntq/MGlS5AjsCel9uwEsnSlX+Suz6tXeRV6Y=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=dPoAwVog0uy7wQp/3I/lzve83ZNpw9aswfi9EfpeSa/IO//9ESETReyREwk2CCAcdguJ1wwr1B09cU148f9NLf82K1nfnbuZYgxygOSqnQBPw5ny+6U48c+TRZRbsqjBh+trgleyO3gvXGCp2JaZZGFFg6vukYVfPWSYVz9GRIyp9/u7441OXPfugNWglv34SysJGfuBXTX1/SoX9KUrd/BOB+WvNp56WXvqeBMYdPx7NqXbVqHAYF/V2rBKH0yG+EsV43uEcTqlCzJwJJjVWd7+kZELQu93ZVIMkXf4ihyxO8BGS7iwGMcxIOP9cq0hZF4PqfCy767AmWpoh0s5KA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699231447; bh=0edVvpF5HI2X8w2PhfMKQjamXwS6ikTkqP2P9tQK87f=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=eCVrmEcZXM4flFN3yQJO5ppUzpyFbr4JTg8L/WdTzbnB8dpYDI1vXic5OyaK7Y2fxE5k8qesugtmxXZskgPFz5BZCUUHlgFT6IExhbWqr+O9Ub4DTrt6GVjbZUef3AkiAGm07X5sGwAQUE27nDMRh9eNIbVISULaj6bYLlQh9lYjxBvK8772nn2mVRtJavdUykUXKTORJZ4zfTXR48eZa5JaLWT6UYaCT36/KOid9Tbaroxzn62FuU4+QczN8XtawofZjDx7ZMfY5qhLBC3+8SJ9gWXjgwPH2pD1bSxwvRctl0GKWQ91yzfgYXrfBI2R/FL2+PNTCPC3GzaPlGlEMw== X-YMail-OSG: yVNYL3kVM1m2ZgV9yti4jJtLW2TMIWeWbEYXsweXSUR.H_1koHbga4eFxT5UF_i nBv.g7w9tpsnWtfNH0zyDpDHzUS4_7bDXzy0kE.jHJOxjycuvsi_GT5cjgy0IEWbd5qlXWM8IT4W d3z9j_gwsN.p_0SMZKM2lIbnSE_eyYLjgch6u6pGjm51DTZitSwYFYmjSIY1ShFnBqoxkZRDqCEU ydSjm3ArN2EQWPFPDWIvgVKLIsiafZ6qpFpP3EqFPatl4baNHUBVH5kvAAb89Ye4wx.YApKcqmpT VWQZwrfWpWO36vgrNI0aYVAKiumpakhA5Y5ZD4hnD3qktvkg9UbtUYMdVJn3MZac7V9waQC0s_QR gm9pcoUJYmZXhpl1tFwPmo0LCEBDqEfGF_Qc0aUSK7YUxnJIA1SmVtVLW7K3aM5FJOCmrOLHgUYK qVWWcsgmYni8.MTfVHzl_lmDgcrLbA.p4e9YUgoVF3Ep6hzrO_cV_CSmsxUvYhyvo2hTUBBPykBC aby51kfQsg5nwGnLlj7kEoJu.8EBcKZox_JDzKydAs_M0tzu6bfdxwwp1i4UzgiyCueAUhSfhQK9 RjBFKOxKvSmjDwCNS3BnFhQE6syl0069DZVL3wSl8WVotGfDCIOTPVoU6hb7vRGXVwu39qS11vUF 1zPPpCgxY0XaPGct.o_fqDih6hKTZJgiqK9KM0i4Rr.wVZ5CVc7oVufZUiaERX0Sb00yot33umAX L_s47F0cgB9SYvEvE_IsmbIv2nBQbxKRc_i0GS.b_hbXLdiWBgtQUSIBdjUh86t3ajS2.2zuMKTG tobq5Ke2L50floCNDVUo1Z.F4W9EOnBQXSOh1d4mOPCPZnhMBWLCzl4Jr58A52CoexW0TLqYbBLi Y.V7ZVOmdINd8Jwpjd6v0uHlva4kA8SEg6Lge6VJeyEhOVq.fNj96XTSgzUDxKRVckTmVuvecs8Y jJQXNIiSW9X4PgLJuDDS3Teaob6NN7QACxu0KTXTQcuoUSOvid6M5EggwyvJLPOjBP7SGPA.SIkN PtUX_Bql2.191_yKDWrCGDUQmzLQj3pdC3EcIbXlz9DFVVcRgc.aTz4BT.w_WxIylXqEK9Vgj_.V vD2QC7vYx3DNVW0DVA5WhYqXI92ZKX0MPricPw7FKkAUAj4pxqi5wZUYguMHwDa6MP_qMPKCrevA buSy6VOSgcPmX0V7v5He2vwoayrMwanDljeh1IAsKuDAe4R7WRQd0Rcq8smgOaE8IWg_ayv0z3_v 4gTGjCTMustMyZSxetpyAYJxeMQh8K8n4fsAeBeS3WX5U6GC5hKoVJGJ5EYPEcjDm_1hWLcHaw4. AVRxLINmO.coA0u_vI6P5T.ptmurJ.WoOnqj3JUXa2H0A_mOvarmx3x39lQk57HcumXj.l9N_EaZ sWLQdX83o1PUURGh_D0x92CmwuU1dZR3o_GQnpSjkwX2PIMNaZ1FXvzM0b_1JkFNlejEjtVXc4kn nUFrj08rK69d.KYSaj7bK_8UiZeTdr.op8B5ML40bd7mWsVgzeh0XZUUEBbepoY4Qe3i8jz3uHuL DWd6jIM9MerMTz4BirPFpyb.6hdm3HyKHwHy3aSp7uVtJWM4BJcYL3fukJ80efHgga5h.1Dkpu2H 1Js7gqMWVxkRCnIXIWacyWsL0dkSZZWvV5lXrYGNnHzTy0aAxi21ShLsubKcrqneO1_rgXWbzF8U odvySrPh0aPGvoP2cW2xq8mu28IgBR9EPeeq1JZ0w_pjEWMkINaKJpH5XAkigdEr_FrNcJVyRlPI zBsRNYJVMM.X5neRochOlNjqLZkohvD1m4Ex6nNzAgdLLy1xvFSRr9afiq8Eqpu2LGY5P99w64Uq bWudSN1DA1Z0q2y26lbmCjQ9JLqmOgKfAODuRTgdmSrh5Auv_NPoowEkCUWtiVJFsqkYA2lYtkSx vXR2576SAXmOyuw2kg627ptWFuZ0c6YGq5SgSZDPxwlBS6e1gHhUpAYjxwFja_Pr2Uu0cKiesbFW 9g_vrDNeKcYHnqzuDhSKsg6wda4ayiuNxEcVP468GW5YF64Tl1LTVR5cj8vVFb6_dJXNyJvSo6d_ LojCk4to3HaV58ENUC_LyhO9YErs.82fB0LClcMkEdAZMoFAuDaOxoSon1r7K5I8ZPUb3L0812iQ DUPEHAzUwG0ObI5bIgE5X7qvDHyH4goFORmtoDd.koE4cTd1_3QRfTqeF77RlTm6FoRJz8ss4z82 J1Mo5Z5oMSweM5pqlxolvPqHuEZOkWkexvQTsbaEnfsX_VfQlEs_n9Rg1Hmc8i0GxERivGQhAu4M - X-Sonic-MF: X-Sonic-ID: da94b9b8-e415-49d4-a122-7e5ec7a500ea Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Mon, 6 Nov 2023 00:44:07 +0000 Received: by hermes--production-ne1-56df75844-pr5zc (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 5020e09f3d3d5d67ad8183454b828498; Mon, 06 Nov 2023 00:44:02 +0000 (UTC) Content-Type: text/plain; charset=utf-8 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\)) Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? From: Mark Millard In-Reply-To: Date: Sun, 5 Nov 2023 16:43:50 -0800 Cc: Mike Karels , Ronald Klop , FreeBSD-STABLE Mailing List Content-Transfer-Encoding: quoted-printable Message-Id: References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> To: =?utf-8?Q?Martin_Matu=C5=A1ka?= X-Mailer: Apple Mail (2.3774.200.91.1.1) X-Spamd-Result: default: False [-2.36 / 15.00]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_SHORT(-1.00)[-1.000]; NEURAL_HAM_LONG(-0.98)[-0.982]; NEURAL_HAM_MEDIUM(-0.87)[-0.873]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_THREE(0.00)[4]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.148:from]; BLOCKLISTDE_FAIL(0.00)[98.137.69.148:server fail]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; ARC_NA(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.148:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_DN_ALL(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FROM_EQ_ENVFROM(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SNszs3hHnz3TR0 X-Spamd-Bar: -- On Nov 5, 2023, at 16:27, Martin Matu=C5=A1ka wrote: > OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can work = with pools that have feature@block_cloning enabled. > The sysctl variable vfs.zfs.bclone_enabled affects the behavior of = zfs_clone_range() which is called by copy_file_range(). When it is set = to 0, zfs_clone_range() does not do block cloning. > If it is set to anything else than 0, zfs_clone_range() does block = cloning (if all conditions are met - same ZFS pool, correct data = alignment, etc.). Ahh. =46rom the naming and vague memories of the history, I did not = understand that vfs.zfs.bclone_enabled has a narrower set of consequences than the name = suggests and vfs.zfs.bclone_enabled=3D0 does not imply any lack of support for = pools that have block cloning active. May be the wording at, for example = https://www.freebsd.org/releases/14.0R/relnotes/ should be more explicit about the relationships involved when = vfs.zfs.bclone_enabled=3D0 since others may read in the same bad interpretation that I did. Thanks for the note. Very helpful. > In FreeBSD-main, this tunable is enabled and I plan to enable it in = stable/14 somewhere around December 11, 2023. >=20 > As of today I personally use block cloning on all my systems. >=20 > mm >=20 > On 04/11/2023 13:35, Mark Millard wrote: >> On Nov 4, 2023, at 04:38, Mike Karels wrote: >>=20 >>> On 4 Nov 2023, at 4:01, Ronald Klop wrote: >>>=20 >>>> On 11/4/23 02:39, Mark Millard wrote: >>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: >>>>>=20 >>>>> int zfs_bclone_enabled; >>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, >>>>> &zfs_bclone_enabled, 0, "Enable block cloning"); >>>>>=20 >>>>> leaving block cloning effectively disabled by default, no >>>>> matter what the pool has enabled. >>>>>=20 >>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: >>>>>=20 >>>>> QUOTE >>>>> OpenZFS has been upgraded to version 2.2. New features include: >>>>> =E2=80=A2 >>>>> block cloning, which allows shallow copies of blocks in file = copies. This is optional, and disabled by default; it can be enabled = with sysctl vfs.zfs.bclone_enabled=3D1. >>>>> END QUOTE >>>>>=20 >>>>=20 >>>> I think this answers your question in the subject. >>> I think so too (and I wrote that text). >> Thanks for the confirmation of the final intent. >>=20 >> I believe this makes: >>=20 >> QUOTE >> author Brian Behlendorf 2023-05-25 20:53:08 = +0000 >> committer GitHub 2023-05-25 20:53:08 +0000 >> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) >> tree dd01dfce6aeef357ade1775acf18aade535c6271 >> . . . >> Update compatibility.d files >>=20 >> Add an openzfs-2.2 compatibility file for the next release. Edon-R = support has been enabled for FreeBSD removing the need for different = FreeBSD and Linux files. Symlinks for the -linux and -freebsd names are = created for any scripts expecting that convention. Additionally, a = symlink for ubunutu-22.04 was added. Signed-off-by: Brian Behlendorf = Closes #14833 >> END QUOTE >>=20 >> technically incorrect in that compatibility.d/openzfs-2.2-freebsd >> should be distinct in content from compatibility.d/openzfs-2.2 so >> that block cloning would not be enabled. >>=20 >>=20 >>>>> Just curiousity on my part about the default completeness of >>>>> openzfs-2.2 support, not an objection either way. >>>>>=20 >>>>=20 >>>> I haven't seen new issues with block cloning in the last few weeks = mentioned on the mailing lists. All known issues are fixed AFAIK. >>>> But I can imagine that the risk+effect ratio of data corruption is = seen as a bit too high for a 14.0 release for this particular feature. = That does not diminish the rest of the completeness of openzfs-2.2. >>>>=20 >>>> NB: I'm not involved in developing openzfs or the decision making = in the release. Just repeating what I read on the lists. >>> There was another block cloning fix in 14.0-RC4; see the commit log. >>> Maybe there will be no more issues, but it seems that corner cases = were >>> still being found recently. >> Looks like I'll stay at openzfs-2.1 pool features until there is >> a release that no longer has the default status: >>=20 >> 0 for sysctl vfs.zfs.bclone_enabled >>=20 >> I use main [so: 15 now] but only enable openzfs-2.* pool features >> supported by default on some FreeBSD release, that has an accurate >> compatibility.d/openzfs-2.*-freebsd file. >=20 =3D=3D=3D Mark Millard marklmi at yahoo.com From nobody Mon Nov 6 07:43:05 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SP3HK4ZSrz50Yfn; Mon, 6 Nov 2023 07:43:09 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SP3HK4HJBz3RZT; Mon, 6 Nov 2023 07:43:09 +0000 (UTC) (envelope-from bapt@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699256589; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LklPFmP9vJGRNxzFEdUNXYYzqy9YbOiQubQNe5v4H4M=; b=pE8AAMB4boGpU0uAsVsBBDnU2zwD3CIUI5uHINFmi+8gEW61N3i2uPXvrcKjgqeO3LjO6l fmF0V0iMjBPfTPgmyRCNHtJCTWH0f5CXvPRp52VXGNMPUPDXCNScPsdjcO/3S76Xu/Ld59 Ox+llFnN85MD7GsbBVvUKUqN9+nS8kK6jOqO2TLQGugai2CW2we43GMdukTBI2bfAmpQ9F InFy9bYuMt+gGE0S5Rh+EcRSEJBPSu/S6xtFcMRHslvtPQhi/dyS/5GVUtd+g+ScvqOxxU bKXBavH9pMK3SC9/ttba7UkZn6K5wRlJupnAdrBUwsvL0JqS8Hg797hC51xniw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699256589; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=LklPFmP9vJGRNxzFEdUNXYYzqy9YbOiQubQNe5v4H4M=; b=DIYobyQMPYge8ggczH/TjAlHD4S+s3PHMXhQU78/f2+OhCvumxH5XWielxxq4Lz3/RYgdJ K4FSzBj9BfMCboUgDP2h8hOvxLwtFYjQFwIeUKn9d/HVpYA+NkHS8lgHCzq41MMdaBGo28 LcAupm3J8dhaycUdv7HqAJaxpWgFHcipAgDNbtcMcQsk/jV2BNpVhhc4ARyjRblN91kqxs AYwnSdcoLqhCfs+L4wbSTb3ePQfcuUnpm02dkPxE6Wx3gMKjwe6+Q59CPJTfe6umHsZhIY G66C6A+1lXh9Pcv4QWfsmP+us2/O2AMdf38mG5FsEudQaUYg3GS3bIdpMQxWeQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699256589; a=rsa-sha256; cv=none; b=D9uJsoEimi15LxHROf9/y/9No1pO+3wgFzkNEhWT3dXDdr2xe6x0c1oltZxwK+whSCx34w JKV7QIvbXtByClt61IUeiPtJ3247rMglE1otfsMYisZvbxi5hiYcapBtk9nUptBe0Mtplq 4HHRlaVSt4hrlqkUVmXxka4UGReMloOWBHXw2rN9NiwaumwfHjXmsbnCdYzI7gQkHqM+Lv aF6NbgwaW6gal4iyyd2G9SdYe6D5bT7A90CZy2qVGnnqOkQPlR6nqP4Zel9/BeY8+ymylS Abg82KmHQbQd25K8yM141u3Zgz7U2xlZHYsuWj1bf0KTUY15mU8gBi/x41N/rA== Received: from aniel.nours.eu (nours.eu [IPv6:2001:41d0:8:3a4d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: bapt) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SP3HK1BS5z17bF; Mon, 6 Nov 2023 07:43:09 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: by aniel.nours.eu (Postfix, from userid 1001) id E423319DB14; Mon, 6 Nov 2023 08:43:05 +0100 (CET) Date: Mon, 6 Nov 2023 08:43:05 +0100 From: Baptiste Daroussin To: The Doctor Cc: Glen Barber , freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org, FreeBSD Release Engineering Team Subject: Re: FreeBSD 14.0-RC4 Now Available Message-ID: <5p6sszthbhskientm4tki4pq4hyzws4fnmzgq4cagh7yvkrwyo@hhmf5cy4gmsu> References: <20231103234232.GK1289@FreeBSD.org> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Sun, Nov 05, 2023 at 12:11:06PM -0700, The Doctor wrote: > On Sun, Nov 05, 2023 at 10:32:44AM -0700, The Doctor wrote: > > On Fri, Nov 03, 2023 at 11:42:32PM +0000, Glen Barber wrote: > > > The fourth RC build of the 14.0-RELEASE release cycle is now available. > > > > > > Installation images are available for: > > > > > > o 14.0-RC4 amd64 GENERIC > > > o 14.0-RC4 i386 GENERIC > > > o 14.0-RC4 powerpc GENERIC > > > o 14.0-RC4 powerpc64 GENERIC64 > > > o 14.0-RC4 powerpc64le GENERIC64LE > > > o 14.0-RC4 powerpcspe MPC85XXSPE > > > o 14.0-RC4 armv7 GENERICSD > > > o 14.0-RC4 aarch64 GENERIC > > > o 14.0-RC4 aarch64 RPI > > > o 14.0-RC4 aarch64 PINE64 > > > o 14.0-RC4 aarch64 PINE64-LTS > > > o 14.0-RC4 aarch64 PINEBOOK > > > o 14.0-RC4 aarch64 ROCK64 > > > o 14.0-RC4 aarch64 ROCKPRO64 > > > o 14.0-RC4 riscv64 GENERIC > > > o 14.0-RC4 riscv64 GENERICSD > > > > > > Note regarding arm SD card images: For convenience for those without > > > console access to the system, a freebsd user with a password of > > > freebsd is available by default for ssh(1) access. Additionally, > > > the root user password is set to root. It is strongly recommended > > > to change the password for both users after gaining access to the > > > system. > > > > > > Installer images and memory stick images are available here: > > > > > > https://download.freebsd.org/releases/ISO-IMAGES/14.0/ > > > > > > The image checksums follow at the end of this e-mail. > > > > > > If you notice problems you can report them through the Bugzilla PR > > > system or on the -stable mailing list. > > > > > > If you would like to use Git to do a source based update of an existing > > > system, use the "releng/14.0" branch. > > > > > > A summary of changes since 14.0-RC3 includes: > > > > > > o ISA and GIANT-locked driver removal has been delayed to FreeBSD 15, > > > and system messages have been updated to reflect such. > > > > > > o An update to OpenZFS correcting block cloning between encrypted and > > > unencrypted datasets has been included. > > > > > > o A fix for Hyper-V emulation within QEMU resulting in system crashes > > > has been addressed. > > > > > > A list of changes since 13.2-RELEASE is available in the releng/14.0 > > > release notes: > > > > > > https://www.freebsd.org/releases/14.0R/relnotes/ > > > > > > Please note, the release notes page is not yet complete, and will be > > > updated on an ongoing basis as the 14.0-RELEASE cycle progresses. > > > > > > === Virtual Machine Disk Images === > > > > > > VM disk images are available for the amd64, i386, and aarch64 > > > architectures. Disk images may be downloaded from the following URL > > > (or any of the FreeBSD download mirrors): > > > > > > https://download.freebsd.org/releases/VM-IMAGES/14.0-RC4/ > > > > > > BASIC-CI images can be found at: > > > > > > https://download.freebsd.org/releases/CI-IMAGES/14.0-RC4/ > > > > > > The partition layout is: > > > > > > ~ 16 kB - freebsd-boot GPT partition type (bootfs GPT label) > > > ~ 1 GB - freebsd-swap GPT partition type (swapfs GPT label) > > > ~ 20 GB - freebsd-ufs GPT partition type (rootfs GPT label) > > > > > > The disk images are available in QCOW2, VHD, VMDK, and raw disk image > > > formats. The image download size is approximately 135 MB and 165 MB > > > respectively (amd64/i386), decompressing to a 21 GB sparse image. > > > > > > Note regarding arm64/aarch64 virtual machine images: a modified QEMU EFI > > > loader file is needed for qemu-system-aarch64 to be able to boot the > > > virtual machine images. See this page for more information: > > > > > > https://wiki.freebsd.org/arm64/QEMU > > > > > > To boot the VM image, run: > > > > > > % qemu-system-aarch64 -m 4096M -cpu cortex-a57 -M virt \ > > > -bios QEMU_EFI.fd -serial telnet::4444,server -nographic \ > > > -drive if=none,file=VMDISK,id=hd0 \ > > > -device virtio-blk-device,drive=hd0 \ > > > -device virtio-net-device,netdev=net0 \ > > > -netdev user,id=net0 > > > > > > Be sure to replace "VMDISK" with the path to the virtual machine image. > > > > > > === Amazon EC2 AMI Images === > > > > > > FreeBSD/amd64 EC2 AMI IDs can be retrieved from the Systems Manager > > > Parameter Store in each region using the keys: > > > > > > /aws/service/freebsd/amd64/base/ufs/14.0/RC4 > > > /aws/service/freebsd/amd64/base/zfs/14.0/RC4 > > > /aws/service/freebsd/amd64/cloud-init/ufs/14.0/RC4 > > > /aws/service/freebsd/amd64/cloud-init/zfs/14.0/RC4 > > > > > > FreeBSD/aarch64 EC2 AMI IDs can be retrieved from the Systems Manager > > > Parameter Store in each region using the keys: > > > > > > /aws/service/freebsd/aarch64/base/ufs/14.0/RC4 > > > /aws/service/freebsd/aarch64/base/zfs/14.0/RC4 > > > /aws/service/freebsd/aarch64/cloud-init/ufs/14.0/RC4 > > > /aws/service/freebsd/aarch64/cloud-init/zfs/14.0/RC4 > > > > > > === Vagrant Images === > > > > > > FreeBSD/amd64 images are available on the Hashicorp Atlas site, and can > > > be installed by running: > > > > > > % vagrant init freebsd/FreeBSD-14.0-RC4 > > > % vagrant up > > > > > > === Upgrading === > > > > > > The freebsd-update(8) utility supports binary upgrades of amd64, i386, > > > and aarch64 systems running earlier FreeBSD releases. Systems running > > > earlier FreeBSD releases can upgrade as follows: > > > > > > # freebsd-update upgrade -r 14.0-RC4 > > > > > > During this process, freebsd-update(8) may ask the user to help by > > > merging some configuration files or by confirming that the automatically > > > performed merging was done correctly. > > > > > > # freebsd-update install > > > > > > The system must be rebooted with the newly installed kernel before > > > continuing. > > > > > > # shutdown -r now > > > > > > After rebooting, freebsd-update needs to be run again to install the new > > > userland components: > > > > > > # freebsd-update install > > > > > > It is recommended to rebuild and install all applications if possible, > > > especially if upgrading from an earlier FreeBSD release, for example, > > > FreeBSD 12.x. Alternatively, the user can install misc/compat12x and > > > other compatibility libraries, afterwards the system must be rebooted > > > into the new userland: > > > > > > # shutdown -r now > > > > > > Finally, after rebooting, freebsd-update needs to be run again to remove > > > stale files: > > > > > > # freebsd-update install > > > > > > == ISO CHECKSUMS == > > > > > > o 14.0-RC4 amd64 GENERIC: > > > SHA512 (FreeBSD-14.0-RC4-amd64-bootonly.iso) = 2af53c6611eb67b184ba4c93c87ad5aba94911946da453428e38af385691c0a952126feff1be717bf67a3d8a227509449dd011364a9587eae41a204da7d18252 > > > SHA512 (FreeBSD-14.0-RC4-amd64-bootonly.iso.xz) = fbfe4be553dc78d69981646673b66f9dd8f83d9f5fca3b1e50b11a816f4d47f5e5733e655adf0bb0a78ea0bfb6579ebee0c48157d69b00ec8a669b292eb0c16e > > > SHA512 (FreeBSD-14.0-RC4-amd64-disc1.iso) = dac0f20ee98a2679647f1afdc8271b809d436182203e0b17601c69086e532ba73edf3872f67016d768e911164d7ecbc24770ce5b5546b2fe7a6fdabbc40fdc5d > > > SHA512 (FreeBSD-14.0-RC4-amd64-disc1.iso.xz) = 662a5cf5c9f1c547f19c974d157b91c600d8e94cd57ff683d3a97a78e10d7675c538c55ad631d7b0c8aed7b4860e4453fce48714debd0b7ded787c533860b5c3 > > > SHA512 (FreeBSD-14.0-RC4-amd64-dvd1.iso) = 6c25adce9d49bc9f168ef4048f4339ae70c1648c392b8f3f466335952f9eea5a05960b78bfee46acd38388195b12851e17d2cacf41be4b1fb22b10d5cb32404e > > > SHA512 (FreeBSD-14.0-RC4-amd64-dvd1.iso.xz) = cd6201e3432d0fc2889c1612612582436e61d322533a05552650fdff2a48583ff7d399d709d85ef279a377f4afb85894cb7544e7b984cdcf48561de412eb18cf > > > SHA512 (FreeBSD-14.0-RC4-amd64-memstick.img) = 02eb4fc41a595d9e78a1866b205d4301b502ec8deeb63414a01bbe61ff3ffecb503ad5a6cc75be4e0e5e11315f5875e141bead5ffe22fa69b111161d7c29c903 > > > SHA512 (FreeBSD-14.0-RC4-amd64-memstick.img.xz) = 705c41197a6d354c6c086aa2feca36704808db0df423cdef8bd2bc59fe3f12931a4cdb7257631602e59d49862e31edddf0b4dc974a61ceb9dd6a59209874ee88 > > > SHA512 (FreeBSD-14.0-RC4-amd64-mini-memstick.img) = 6988e9300baa0572d0801d9de289784c63b8f6aaab2c2cf37775458bc6828c79fcf3dc873b553346b0e470defc613a7ffdc4cf697e913e25f1cc8c3f95bd6c23 > > > SHA512 (FreeBSD-14.0-RC4-amd64-mini-memstick.img.xz) = 9d597049a794d5f6474414d0146107bf4f2886b25a6b48e8cbadf00868ad63afd219550257991509a3969a0462cfa655229d5e6aded9cbb601b6b3508868fa6d > > > > > > SHA256 (FreeBSD-14.0-RC4-amd64-bootonly.iso) = 28674417c8da59ffe4d1c4fc4f325f49d3f627787bda5dd485a82c6080816260 > > > SHA256 (FreeBSD-14.0-RC4-amd64-bootonly.iso.xz) = c9fe6bf7598446ddaaa181a503b559bb185e8f2acc63f5cd93d3caa77b49e762 > > > SHA256 (FreeBSD-14.0-RC4-amd64-disc1.iso) = 213d610088db85392ca8a8b8c916c93fff9dc8f8d1c5a07cd2bba9df55ca9105 > > > SHA256 (FreeBSD-14.0-RC4-amd64-disc1.iso.xz) = 74c460930aaedc994bb976579baee89df81b08dbacfe7668f004d9d45305b9e3 > > > SHA256 (FreeBSD-14.0-RC4-amd64-dvd1.iso) = c2feb7808e4e34a32c3c309c1adb0bf8ed5cd0fa4b4ee98a44b1ebc7e48dc04c > > > SHA256 (FreeBSD-14.0-RC4-amd64-dvd1.iso.xz) = a06045e90977e5d12c104daf7d96d9f0f452d3215d080dd27d072ab670ae981d > > > SHA256 (FreeBSD-14.0-RC4-amd64-memstick.img) = d315c05038065f984c178829deeda47ac682c74c1bbe66a177873cdda7bc25b4 > > > SHA256 (FreeBSD-14.0-RC4-amd64-memstick.img.xz) = a9068d9d04ac6b32f1d22b4ae803839a28f399fafe70e77d0a6aa7b3c89544e8 > > > SHA256 (FreeBSD-14.0-RC4-amd64-mini-memstick.img) = 0035dd4d97b5243b0b8ddf9636bb39071013941d48fea9be0b92015cf74d4d5d > > > SHA256 (FreeBSD-14.0-RC4-amd64-mini-memstick.img.xz) = d5e7b50fee0cfd5a8cce12fe0e86cc2539fe4d84345c5363b7ccfc08c59eb525 > > > > > > > > > o 14.0-RC4 i386 GENERIC: > > > SHA512 (FreeBSD-14.0-RC4-i386-bootonly.iso) = 8a9bc193a306328ec6fc17fe85a7333365a6f41560876ba6753623bff37a361b9e4e9417cf31f5913205c27e01b06f431553bff065efb4926d251229883e82f1 > > > SHA512 (FreeBSD-14.0-RC4-i386-bootonly.iso.xz) = d4a21ada61e8f7157191c022ab30f0f17fabc3828ec75628fe18946a7b52f44d9f684fa07d61ea01ffc0a64a4fd7cfd1927985a4be1d264c7840131999ed87a2 > > > SHA512 (FreeBSD-14.0-RC4-i386-disc1.iso) = 1b91762401259e9a0aab18499fa8270f8dad2b7651c5433d313850052af7e702a66f9408a7f274e379cbd86cfca184fb3e9181f33939a771dce1e683f0fb90e5 > > > SHA512 (FreeBSD-14.0-RC4-i386-disc1.iso.xz) = e535de2443335277f43fc8de8e93fab85b452d8c67596d4b87bd6830f0bf6a09e0014600d2e984c69bf5e5bb600683441b87cf580fa3994a626a098e10cb3272 > > > SHA512 (FreeBSD-14.0-RC4-i386-dvd1.iso) = 2b803000b10cb5d1fe324708193874dc52035ac6ed3442c6d6ce1668597da5f19c1bf920dc63a34b5cf599c52d2d423e5ea22402edaf084f219b755f7be99c24 > > > SHA512 (FreeBSD-14.0-RC4-i386-dvd1.iso.xz) = d269054590c2f66217a7d500c28886fd80f105e343052a69b55576c2aea981e9745d10b49778d18af13f8b2b2e884eaeb90b1d0324f196dfca0779032ff479bd > > > SHA512 (FreeBSD-14.0-RC4-i386-memstick.img) = 8a8c6488926099c04a78cf0ddeeececbf0c4d2372bf1ea00516d8bdad8ec31f0ac81333c06c3a78d1c883297fa584e53cc895f50a4cca620a9e20125c909d16f > > > SHA512 (FreeBSD-14.0-RC4-i386-memstick.img.xz) = 68211b0d63ffab63a76e04001d9ed0960095a1e4effd24d38f7fcb7e73d3d1ad99ec1a94d555e5610375bae534d01ef925a7d08ef542a9a352a6a7330712baa6 > > > SHA512 (FreeBSD-14.0-RC4-i386-mini-memstick.img) = 0571d8c32f493a908beda92bc92c597f9bd80a2474108ff743ad11dd463ef21c754294da5e993beb8944df03cd639d2d969318651d1fe1a1251b3f4f1f5eb66a > > > SHA512 (FreeBSD-14.0-RC4-i386-mini-memstick.img.xz) = 83f3422ee6935f8b6994a1fc40c2f750ea27bab1c1b324e4b3300f87de47e8a29e7f127201a4eb03f713b6fe1d47ea81a05cbf8ce063286960c3e28ab2f17694 > > > > > > SHA256 (FreeBSD-14.0-RC4-i386-bootonly.iso) = bfeacfe8bbf429cd88e8ed4145de24f747e75c4e97c6bfd023d88d4752e5100a > > > SHA256 (FreeBSD-14.0-RC4-i386-bootonly.iso.xz) = c11f09b11183105b5920753bbcf68427574d09ca7fb10d812b995999c04b2b0c > > > SHA256 (FreeBSD-14.0-RC4-i386-disc1.iso) = e032e4300210f9aed57e5c820fc3c07c4d0b9cbba417d61458dba74d188f0214 > > > SHA256 (FreeBSD-14.0-RC4-i386-disc1.iso.xz) = 53d6c96580d2a486b9da43169e4e4183cd125b06f3a1e7fa25e74487c20e50ea > > > SHA256 (FreeBSD-14.0-RC4-i386-dvd1.iso) = 7e756d0ae13e9798601f45740ab7381c0e319efa3691ba1565ff203a613ea77a > > > SHA256 (FreeBSD-14.0-RC4-i386-dvd1.iso.xz) = 90bbb28db964188988bbff402a677fe10cffaf9f620fc78a5883a1c381a969bb > > > SHA256 (FreeBSD-14.0-RC4-i386-memstick.img) = e4699f5e7ffe6d9f38b000e324c896955c005324702e30477b529786a49e00b9 > > > SHA256 (FreeBSD-14.0-RC4-i386-memstick.img.xz) = cd978403efd56248c655232b02dcf5e3b292faf691c98e72f1482707e2bc9db2 > > > SHA256 (FreeBSD-14.0-RC4-i386-mini-memstick.img) = b80b37c2aa16c4166426e2f6f3ec9fdccd765ca965f75b855ef3dda619c7a02e > > > SHA256 (FreeBSD-14.0-RC4-i386-mini-memstick.img.xz) = 5157bf384edbc8c4069e1360297a680e3540be85aad614f82c7bcc14d1ee1055 > > > > > > > > > o 14.0-RC4 powerpc GENERIC: > > > SHA512 (FreeBSD-14.0-RC4-powerpc-bootonly.iso) = 02ad34d6e7a3acdef46d3a6bb7abb44fe3e55fdf8f99b900a3aedd97520f409888db21e9ab359c641deabfebcce5360ec1d1d4ac90fe60c42b2ee8de27c9da89 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-bootonly.iso.xz) = 1a8b2b661cfbef4f1656c5185dbdbd37b751ff4862f951b30364d9d91bbe030e69c5ee98adac1be81a3a7fc0e209f530c57440c714c8e9e8b1a547e3cdbb6633 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-disc1.iso) = 783445a184562191cbaee132004de70502aac22001fc5aa1a10ae0d5d18eae5005dc630a2c3e72fe9b47e98c4625edef165fce21f1678deb1f8ad3e991ca8f3e > > > SHA512 (FreeBSD-14.0-RC4-powerpc-disc1.iso.xz) = c28086587265cc80ecff0233ac6eb8b3e4a51232e5ab662c04df513e78fe996f051aefb668038190658805d18539c86f49e992540308ff35e4c47fd24a72e292 > > > > > > SHA256 (FreeBSD-14.0-RC4-powerpc-bootonly.iso) = a6c330740401487ca15097612acf08a1b8c933e94f6a1a104527062250f5d5f9 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-bootonly.iso.xz) = e7e5bfcd06bce96a10785bfe0b2aaba21f7cf0963af0744e1b064bc198c23811 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-disc1.iso) = d31ac53ca89a915c2a613343bd4bd5bf128d2e8f9f38afa789e232097c25ab5c > > > SHA256 (FreeBSD-14.0-RC4-powerpc-disc1.iso.xz) = c85cbf09b76f66b6795b18214d06329694f392e06ffa9df52447cede5c0a9aec > > > > > > > > > o 14.0-RC4 powerpc64 GENERIC64: > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64-bootonly.iso) = 6dab28bf1bb33d4f0b2f90eb3dd45c473e2ee446125f3597a43c5a69d5731a3bf52c402ea8d5df0871dc53e0689bd01dce279e36e9ebff64bd4a9a7988083795 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64-bootonly.iso.xz) = 3022076a2c7bd3c0b27775320267e232f61efc4469d84045f2732f0864a780641e8833de89b6a591a2a5302884588b956b64cf85c538467998f7e005af72be0d > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64-disc1.iso) = bf3c2a10a513ff281e77aa0cd41d0a5c8bee4d5be94936615e612dc3044dbb4e8077b0e5e1fa236479a0d48b72649871e5b1f5cb8d64cb484d09037ce3407f65 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64-disc1.iso.xz) = ac4ba53584e65b9ce94a6fe50139c4df10ba527647c170b4a37077e7bdd0d6abebb0954ba7b06f4b1d0aa27d74fec6f8cde2f214646d9c0ed4523790378bac96 > > > > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64-bootonly.iso) = 73c31144955ad8a63772dec8ee4be107ec9c290e966faf95f354c63c1b41ca14 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64-bootonly.iso.xz) = de0a4127dd7b7ea2bd4bbfb809094199e1f9135f1c9fd12fcc6e06fe7c818861 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64-disc1.iso) = 34a72ea31ae01ad57ced7e4ab6eac3b9714179003ecd0995996284c74a88ebb8 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64-disc1.iso.xz) = e3f82a17d9dddb20c6ca195340ddcf2d63eb82bbc6c381b4adcd864c1845386f > > > > > > > > > o 14.0-RC4 powerpc64le GENERIC64LE: > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64le-bootonly.iso) = 14338fe5d76356713ee94f6b3ca293abbdaf380d0cd2b74ada14a4ef475f6c34c68e67dd13fcb69dbd9cbab1a951d4e71c81d5add7f3133d792e94311c8f262e > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64le-bootonly.iso.xz) = 399fdb443265566bd5e6ce1a3dc11ca41f005968a529a7535dd72622a3ae0fb2a93e66b7753c953fc5d35c63b58022ab4c223f914d3720a19490332b379156e3 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64le-disc1.iso) = acc729d7abd0f8474e1fdf3f22804aaa486c98e479dad813b83116036e0db16dce9ab956c1fd6fb80d69f462f43200f3220864e1901ae62903b717682838e5fe > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpc64le-disc1.iso.xz) = cb341d0eb64847a50ccb33170bbfca9c8855377e6883a4cc0c181967182700afc669b6112edb37275519cac5992765700f9eb16aef6d0bcfbc25cdb399839cab > > > > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64le-bootonly.iso) = 2bff70b162f8f4711f9f2eb67fe10716166f4b8c062e4cbaca34fb8d521bfb09 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64le-bootonly.iso.xz) = 03c7bd799e52bb6e95428ce7809ba863f2c98e2ab71dbb45fcd67eddf79677ef > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64le-disc1.iso) = e2387661bb279c3cf6f904a858c2d87aed1ede49a29f1d1f8424d65bad38ccd6 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpc64le-disc1.iso.xz) = a3b4ca6af55b099100677685033328868d28cdd9322b1bcd12b4d5c0919d6652 > > > > > > > > > o 14.0-RC4 powerpcspe MPC85XXSPE: > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpcspe-bootonly.iso) = 4417492d5d6ae001661252a465090d9d6daa50429cda5c069284c95095533e35e6b9bb94c4c5a24d8c51768a9732b960b3659f1e66577c4e0baf41c37cdb01e7 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpcspe-bootonly.iso.xz) = e9fb2333c5dea7048fbf289209bab16a8854de08847782acbcfe869d32d77aa5dc4089020baf53c4e88aa95c8dc696ff7bb770f8d14a823d2c0248c477cf78d1 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpcspe-disc1.iso) = 25ebf8167e090dd09647eff3f26c8313097de84b3038165018afa05f5dc5893b63286042a524fb53feeb323ee337c934616f47c6739e65cd89ffc3bc0e5076b7 > > > SHA512 (FreeBSD-14.0-RC4-powerpc-powerpcspe-disc1.iso.xz) = f7199bda837bd15ad1c201962fcc31c88ef0134605092436f09fa1ee7027f440cd808deef20c34f63c6a80de1411d1eababfa0ed529af091dad7baef8c676a77 > > > > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpcspe-bootonly.iso) = b9b930c2e2d3a320befc46cd93eeeb8bb10d8c87db8864d00fc5438def022391 > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpcspe-bootonly.iso.xz) = 9281bceb101f97b9741bb0ca3407998db598a35c6117f472409d7dbc88c78f4b > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpcspe-disc1.iso) = 01b59d130bb608dfb012306472d24d9a46a25c329a2d86bb9396c5b26c9233be > > > SHA256 (FreeBSD-14.0-RC4-powerpc-powerpcspe-disc1.iso.xz) = 721a3a28b3cebb358d5d93ba7cf7345325ea421d6fb1c6b624ba8d8a8244d23d > > > > > > > > > o 14.0-RC4 armv7 GENERICSD: > > > SHA512 (FreeBSD-14.0-RC4-arm-armv7-GENERICSD.img.xz) = 5d0592accd64318e81ab36b3f15cd828998734a6c781d9a267928550a5ae60ab3a2e36ddfdc6749b6851f96e06f57778aabb861c6f55cdc2476402eee1c8f8d2 > > > > > > SHA256 (FreeBSD-14.0-RC4-arm-armv7-GENERICSD.img.xz) = 858f153be5aa6312e14ae8b4109bd7bf3b221668cfacada991bcd22beec95342 > > > > > > > > > o 14.0-RC4 aarch64 GENERIC: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-bootonly.iso) = 47b116b596167cbabed0cbe238b0265c79cfc9cd75be74ab690c465756145552e6c8d40f8a803d56afb4d7ef0106cce14581d342e31c92cf230b12ff5fe04324 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-bootonly.iso.xz) = 379bdb38548995866a5325b3fd4ff71b90887d2b97d9c5f41561d41c953cca271990a66b541b5625897ef4ce1359deae8fcf42f7efa4f19aa1b582793fb4e513 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-disc1.iso) = 105a6fe2f4b4b0669219adba0aecc4f056e27d3f380d1591a8bdb1cb60917b52242bd5ac9127a461bf3cdc85d323402977e617220c37129511149810e166c48e > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-disc1.iso.xz) = 413a2ce8523ab59acd70bc8e52c93ce65558cf0b57627683534267d5695977b1bcea85d754528251e1375b469f8f69b15e22476fb3cd8916cb59c4ca10a6217a > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-dvd1.iso) = ad536c842136232eafa8f27b57641212c3b13bbb8ad560cdc7d964dedbfa878b6f050876792c94a54c8caccc687c49fad1bfe5586e51f9001c2db5fc1e24651a > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-dvd1.iso.xz) = 74d78034d36f19f918e3c807ecb675b5d95bbe3d6fa517d6d2ceb10a29fcf7d89c9067ec1c78638af6cc8c5fba8eb91bb5445a4af1a66a447c03000433ee7ac4 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-memstick.img) = adea1b24641b05b1caccd806f224397421cf94d1105861d3d6485065027461f20809edee7f258fbf45a69cd82a1beae7afd0c48d025a8e4646232d9a634abf1d > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-memstick.img.xz) = 76c6733aea25b1c7130b0dd87056a40310caf5868afd592fb0239a882cd45f61c8975b3819a958d0dd42385d2c52dbe61dc910d7814a0c8514c804b52944da09 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-mini-memstick.img) = dab27511105faeeaccfaabb498682906ff2d61561e9f422eb26b20abc4ce2bbfa0a6d3cbfc5e26e06a737fffbb799559ebd3a659ad5a37f361937cdbf0f3c312 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-mini-memstick.img.xz) = 918d3452b5a4bc5873e781596d5bc119cde82b89f782094efc5e6913bc58774bf368ee178287fce22ff146526cbe17171e7db45acafdb94bc2818d45c475fcaf > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-bootonly.iso) = 84cedc4fcf0da7186f1289815276b7d6eb1cb9fd0058a8057bf130e00b7596e5 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-bootonly.iso.xz) = 080dc3e9d965ef4b5180b376e5c6623fb5d2fdb6b6e3ed151ca981eef6965675 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-disc1.iso) = 41a9163f4adaec848319f914f23879c608fe73ef3d498f1b402494d594e86fa8 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-disc1.iso.xz) = 621897f6d6f3631b0b830d08fa5b969a1a556e18aac637a05a6d9876885d440a > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-dvd1.iso) = f252763cecb39b3fa6f3d2a75dd593bd891f01c3dcd68f90436a53610b77fcb2 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-dvd1.iso.xz) = 7e9e9085ef8c4ec7f1bf7e493fb16f2101edd59342b05de7bd52c4e142aea812 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-memstick.img) = 145eda64f7a3e91f3ba253478a5d290fe141a0f6c2f8050e04625ec759f1e043 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-memstick.img.xz) = 7afc16842df83f0722e667eca07cdfcfbc9b2c35f8437b3397c195e6a9636c27 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-mini-memstick.img) = 9baa46356ae16a828e3e8ad3d9f1217bf4dd2d826392f18797d885600f7e814c > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-mini-memstick.img.xz) = 000ee91e812d04886d04d6e4f77516835caf1c372bb37eebbcd81a56f5dd83e3 > > > > > > > > > o 14.0-RC4 aarch64 RPI: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-RPI.img.xz) = 7291b077b0338c5400af32eb6822d0139b953ad971cd9f44c5646ab5e3e502833fcda35005412a7b24203dd8dc9e3b2fe68bc3c737812b2255be624094013c47 > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-RPI.img.xz) = 26d84974d64fc0fb10cc8f5f7afd9fd49a84b42392b3537fd8766bdcc0e0cd41 > > > > > > > > > o 14.0-RC4 aarch64 PINE64: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-PINE64.img.xz) = 6aaa42b2f515e3847235001382683484a57790e9ba1da8b19fc318668ea6b0640f76e00d6a12b3daa1ea9ea09bd0afd0db1fa322ed236fa9e181c1ebaa19906f > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-PINE64.img.xz) = d242d0337fd76684ebfce4ff84641303da07502d51f33089219aa70c615eb4b2 > > > > > > > > > o 14.0-RC4 aarch64 PINE64-LTS: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-PINE64-LTS.img.xz) = ce3deff3d02244359303331b0b3ed966bc503a42f1da0eee54f62984d2e8e54dc61f184c699cfb11aa3465fe31a89028799c0059185915d151da3be10301b8cd > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-PINE64-LTS.img.xz) = a6d398d616d2f9db3692f2c8118d9a5cde3960cdf2797497a7c155b9e8139b37 > > > > > > > > > o 14.0-RC4 aarch64 PINEBOOK: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-PINEBOOK.img.xz) = f0a50d83fee5d113a95ab09c64c757243bb8874d8384c05f4bc14d6f5bd33075989ae0a73c13a46b9107f32c2834a74cde5481e970fb47d064544664a21f6abf > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-PINEBOOK.img.xz) = 42e38931bd4d527ca0753cf861f90e91d3df2f9e108c632656a77bcf16bf5068 > > > > > > > > > o 14.0-RC4 aarch64 ROCK64: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-ROCK64.img.xz) = 14888fe2af5cd4f1787b86cd49c61f353619761cbe43436beb12ca1207b5c15e682ec0446257dd0dc4876c9e0a9843b318e38f20f4b8f96f8aecfb2c7d8ee6d2 > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-ROCK64.img.xz) = 720687a8f2f7639d661dc21bd2176bad14c27f51d6fcdf6aa45fa976352332e6 > > > > > > > > > o 14.0-RC4 aarch64 ROCKPRO64: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-ROCKPRO64.img.xz) = 476f9bf3cf86aec65823eeeffe8b3caf3f8df3b599672137bc6cddacde16b5db28575e768206cb5d29baee09236a658a03f75efb5fdfb766adf49abcd698bee3 > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-ROCKPRO64.img.xz) = 66d44eb1c4745b80c928dc71fe4822b9adc213cd66dbb61f78f440d715c9cd82 > > > > > > > > > o 14.0-RC4 riscv64 GENERIC: > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-bootonly.iso) = 524773593f3187ef7bacb7bcd49ad3422121b819ae34f51d79d6ce91f2754df7e52dd28df62c335bccfd987bd7af073a00d04beb866cfc83d710e6cdc8d33624 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-bootonly.iso.xz) = b1fa47bfe88a85dda0a7a92e76d65fd620a5c62a1c8fb5baa21183ff75614bf80c63424a9efd4a4a646e50c03e1dfb17c49c589dbdf7c8c78f915835115d1ce3 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-disc1.iso) = 2627889757b073413dfdf94118ae7d47a173a4a6f148d9552c7f77357ffdda54298a4ae59b0c5a36159076784a2f50ca1c24cd127616ade0945a953604d62935 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-disc1.iso.xz) = 756e79801584cf2366d8eb0365facd53e6fcd3ae1c753073f6389d4e16c4f839ddf8e5a26b27daf767b5b727dac3d2976e14b02231d52498de7ee4bc4dc34541 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-memstick.img) = d511a815a8ceef7bb3197dabecd4e59ba61078d008f96ec6c8990b9d0583569ac896d8739133feb7f13ea91635232d6dc3e6f9a8e5d546047ecef953b087cfd8 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-memstick.img.xz) = b33caeb29f10dda4bbffba3ab1cdd914777e1b0639a2264859741522f313f38bde8a9bfa24e6cdb64c02746257e7dabaa56c39b3824a66480063e8be91c79fb6 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-mini-memstick.img) = 68c638927e0e070e8ec7a704e763db9561a23b46d39288e55faa1ac8fde0b39853ebf9c36e2f51752a965d6940cbdbdcafdff2bffc33fad9866363988dabe337 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-mini-memstick.img.xz) = af0b304c13a6bb69eebc5e106caf60419a8d41d2da90ef935b76e7b39cacceba98c37a8237d7cd3d012fcd4658d71a36d7197f7e4ee2b9fa67c8d7bd4c33c830 > > > > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-bootonly.iso) = 3605436a2601aa992ab8f6b0ac238e13fd8fe14c11c42c6fe5ffd2395f0b5669 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-bootonly.iso.xz) = e9423f5452213d821720bb6497416741f678187d4c7eb885437661c7b13af955 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-disc1.iso) = 5f81e6deb3c98feca33ae5f5b409bb25b425b62279bcc1b40a36e756ad1ae1a9 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-disc1.iso.xz) = 8c6e348f874ef42aab7527fd118902862529e37a50e33e73ba700758262501c8 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-memstick.img) = 69153d6dc19fe08af410f23af3c2f95bd33500e9152e2f00a9489d47fa308faa > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-memstick.img.xz) = a1b97f38dd5f16c2c0fca710a6c296e5420388d2df887c66c31c7b152449cf47 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-mini-memstick.img) = e9a370a815ce652fa9851242b4beea28a41ee27ca3ccab87c2605fc491dc8fbb > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-mini-memstick.img.xz) = 93075f74008d544c843332ef792dbcae4cf26ba57b50842c39f299e8137d1da3 > > > > > > > > > o 14.0-RC4 riscv64 GENERICSD: > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-GENERICSD.img.xz) = e61212d3bcb42858f080c0260fac4d0fcee80819de0ca6504bb0c9113cf598b43d11134ac26a9dc3b5fc142431370c3bec86debd94e02a2fa3114c7c7d9cc34f > > > > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-GENERICSD.img.xz) = e1bb1522ebaeb9a8b0d218af007ba76542d87e6b0fd146fecc07bff7d2bbed90 > > > > > > > > > == VM IMAGE CHECKSUMS == > > > > > > o 14.0-RC4 amd64: > > > SHA512 (FreeBSD-14.0-RC4-amd64-ufs.qcow2.xz) = 166b0709efc63262b964c604ea746e1ac6743b2bead2efab5d5c4d6c9491a336e39d1a793b465eb751f6085ef7fa0988840e1dbcb47f67f0f5dd7ae3d48b57c4 > > > SHA512 (FreeBSD-14.0-RC4-amd64-ufs.raw.xz) = b158c9369ecbb0178cc048990f855a71e6ffbff32f0bdbf2d5ff0255659b95e062ccc886cfdd7f91ad25fdb1d8d0f4bb78d3d097202bc6b8bd27cf13fde3220c > > > SHA512 (FreeBSD-14.0-RC4-amd64-ufs.vhd.xz) = f860eeb1cd405bd074cca746542cff147c9331a5b64a50c43c95d686686db81b5d2e4b1f5ea8957f403b4425dfa156bf39ede7a98807d25828d7d9dc0232dd69 > > > SHA512 (FreeBSD-14.0-RC4-amd64-ufs.vmdk.xz) = 3787cf9de0a5f4b8ae358f2316ddf62493aa6b30d70f6af2108a1730e9d79e07ef847932d7fa42b11f6f5b8b9eb3f33f41c701a67f2995005afdea053d6ef330 > > > SHA512 (FreeBSD-14.0-RC4-amd64-zfs.qcow2.xz) = c3e7e36e5b19f3ee80011a035b047063deb64acc1409fccd3732f5639d0cc1ce0c0b1da0f60a99b3cd7c696243917ec580ef6d5319b52c1763676def92d34e86 > > > SHA512 (FreeBSD-14.0-RC4-amd64-zfs.raw.xz) = 50c9a08d10d96f42c1669abf25a75d4115c6eb6b71241f1b8e321dcf2ac313627e6b449b941604150ecef652a7786dcd0b0be7c1b4221ae459c08e0bd5ceee91 > > > SHA512 (FreeBSD-14.0-RC4-amd64-zfs.vhd.xz) = be44fe7f97ac044b00cc39554df4eca8a19dcacc03e936747822acbc091f0baa5b6f6ebd219fd34a1f3e86f8031cf132cdd7e12f9b498bef3b9a4c79cb60d3a8 > > > SHA512 (FreeBSD-14.0-RC4-amd64-zfs.vmdk.xz) = 25a399e1165f7c77ab96daff57270cc5013fc3f2a1d777583fbde100a396211f46661f3fd9e675367e519b7be3cc29276529aaab062a821e8763496e8a628597 > > > SHA512 (FreeBSD-14.0-RC4-amd64.qcow2.xz) = 166b0709efc63262b964c604ea746e1ac6743b2bead2efab5d5c4d6c9491a336e39d1a793b465eb751f6085ef7fa0988840e1dbcb47f67f0f5dd7ae3d48b57c4 > > > SHA512 (FreeBSD-14.0-RC4-amd64.raw.xz) = b158c9369ecbb0178cc048990f855a71e6ffbff32f0bdbf2d5ff0255659b95e062ccc886cfdd7f91ad25fdb1d8d0f4bb78d3d097202bc6b8bd27cf13fde3220c > > > SHA512 (FreeBSD-14.0-RC4-amd64.vhd.xz) = f860eeb1cd405bd074cca746542cff147c9331a5b64a50c43c95d686686db81b5d2e4b1f5ea8957f403b4425dfa156bf39ede7a98807d25828d7d9dc0232dd69 > > > SHA512 (FreeBSD-14.0-RC4-amd64.vmdk.xz) = 3787cf9de0a5f4b8ae358f2316ddf62493aa6b30d70f6af2108a1730e9d79e07ef847932d7fa42b11f6f5b8b9eb3f33f41c701a67f2995005afdea053d6ef330 > > > > > > SHA256 (FreeBSD-14.0-RC4-amd64-ufs.qcow2.xz) = b79d9e82e3bf5081efb8159b66b3b3fa0472fefab9389f0ca8e18a469720a5eb > > > SHA256 (FreeBSD-14.0-RC4-amd64-ufs.raw.xz) = c1718eff5c455afc9e689b97751ba577eeff880c1e02ae0ac758dd8452f18977 > > > SHA256 (FreeBSD-14.0-RC4-amd64-ufs.vhd.xz) = f75b814422bfe88ae352d4342a6fce7a7059ae83210c20f3af30ef410bbc7a56 > > > SHA256 (FreeBSD-14.0-RC4-amd64-ufs.vmdk.xz) = 4abcfc77bda9f04fdb83af19d66a3db908dc2eac3983f3169e01b59c7cce3055 > > > SHA256 (FreeBSD-14.0-RC4-amd64-zfs.qcow2.xz) = f99a66864585598496cc43f73deed54f1a4c05552e5f02e8505c71bca743d573 > > > SHA256 (FreeBSD-14.0-RC4-amd64-zfs.raw.xz) = bfcb87a82a02ed1f7ee3bd25885f3d8d51c1c384f56e4ab9ff959fbac600e6ae > > > SHA256 (FreeBSD-14.0-RC4-amd64-zfs.vhd.xz) = 7c3c47e9abf98e406bb3c68dba55cab34a54e50fae5df79afdf9605dc768f0f1 > > > SHA256 (FreeBSD-14.0-RC4-amd64-zfs.vmdk.xz) = 5b0c3e8941ab57d18a9c3bf611158812d413cc9f1191bafdc0874e3b858448d3 > > > SHA256 (FreeBSD-14.0-RC4-amd64.qcow2.xz) = b79d9e82e3bf5081efb8159b66b3b3fa0472fefab9389f0ca8e18a469720a5eb > > > SHA256 (FreeBSD-14.0-RC4-amd64.raw.xz) = c1718eff5c455afc9e689b97751ba577eeff880c1e02ae0ac758dd8452f18977 > > > SHA256 (FreeBSD-14.0-RC4-amd64.vhd.xz) = f75b814422bfe88ae352d4342a6fce7a7059ae83210c20f3af30ef410bbc7a56 > > > SHA256 (FreeBSD-14.0-RC4-amd64.vmdk.xz) = 4abcfc77bda9f04fdb83af19d66a3db908dc2eac3983f3169e01b59c7cce3055 > > > > > > > > > o 14.0-RC4 i386: > > > SHA512 (FreeBSD-14.0-RC4-i386-ufs.qcow2.xz) = 83e9052c4e7bb574ea9763309ddc952a9632cc8b2b8693cf84c3d405e91a2d5c0ce8c9b25947982661537a9fb64a3325449cfb9f18699741328f5f8f446d1407 > > > SHA512 (FreeBSD-14.0-RC4-i386-ufs.raw.xz) = 66f9dca74d1e24869ea85f6ac10868b9197bdb508956168319f25db9ab1e7b13723b48cd544d0b25dc01d898eec127decb06329ca9c3ca9f3b333fd106eed49d > > > SHA512 (FreeBSD-14.0-RC4-i386-ufs.vhd.xz) = 8b0152123fffd4ea9e38b7658eeddf7e6fb69525e68312f3ca0a57efdcc3716113703e94b03da0f6607ef87782599a1da74b588b3b17a934c4299f9b9d62f99a > > > SHA512 (FreeBSD-14.0-RC4-i386-ufs.vmdk.xz) = a894383f188ad274130eb3c6d27d2e8810a502cf4f43544ecead17a61880b7b5a24f3a2709b6251c4139613a9d7e231f6d13b75b2c1413136e76b9ada1b04897 > > > SHA512 (FreeBSD-14.0-RC4-i386-zfs.qcow2.xz) = b02e65d34149f2b21ea8f4d546764f16ad0c96c067f7c7afb685301cf4e4b1d5953f9fbaefdb6b8b2e1ab275d5c201ef1f1e3a2fdb4ce9a60078c133092f6911 > > > SHA512 (FreeBSD-14.0-RC4-i386-zfs.raw.xz) = cfbe7aa6c8339daee268202e95d2c0c385f65debe53a22710c8314beae5a7f6e9cde02920920febf062a6613c84498e6659aa0878b112fb06af7c1a27843fd65 > > > SHA512 (FreeBSD-14.0-RC4-i386-zfs.vhd.xz) = d72284dad778a11cf40be9a91ba1537f32118567e3dcefe40449d225f6f6280746a89b0d15e689db7ed60b54caa4cd69067b303791ab33b47b58ccc854e0c035 > > > SHA512 (FreeBSD-14.0-RC4-i386-zfs.vmdk.xz) = 0e61517e4ff20edb0154bd68a7bff76f6095a4ef63bd1699f4665c39cf4a00d7eea1f9d562e4620c75c1ab982811409032033900e8fdcf892d228fcf49f3a335 > > > SHA512 (FreeBSD-14.0-RC4-i386.qcow2.xz) = 83e9052c4e7bb574ea9763309ddc952a9632cc8b2b8693cf84c3d405e91a2d5c0ce8c9b25947982661537a9fb64a3325449cfb9f18699741328f5f8f446d1407 > > > SHA512 (FreeBSD-14.0-RC4-i386.raw.xz) = 66f9dca74d1e24869ea85f6ac10868b9197bdb508956168319f25db9ab1e7b13723b48cd544d0b25dc01d898eec127decb06329ca9c3ca9f3b333fd106eed49d > > > SHA512 (FreeBSD-14.0-RC4-i386.vhd.xz) = 8b0152123fffd4ea9e38b7658eeddf7e6fb69525e68312f3ca0a57efdcc3716113703e94b03da0f6607ef87782599a1da74b588b3b17a934c4299f9b9d62f99a > > > SHA512 (FreeBSD-14.0-RC4-i386.vmdk.xz) = a894383f188ad274130eb3c6d27d2e8810a502cf4f43544ecead17a61880b7b5a24f3a2709b6251c4139613a9d7e231f6d13b75b2c1413136e76b9ada1b04897 > > > > > > SHA256 (FreeBSD-14.0-RC4-i386-ufs.qcow2.xz) = 6ef1dfcea80aa0687141c7917f0e8ab81c4241a62bee8043ac1dc23c82be05ed > > > SHA256 (FreeBSD-14.0-RC4-i386-ufs.raw.xz) = 5eca8b0ffe97f7e6a485f496a61fad9c93a261c6c4e0d083f9fd1c500c2d4db4 > > > SHA256 (FreeBSD-14.0-RC4-i386-ufs.vhd.xz) = 3997fd89984392f2ca2fc3b7df864dfd0dfc17be750aa78d61a31233e947fc0b > > > SHA256 (FreeBSD-14.0-RC4-i386-ufs.vmdk.xz) = dda9a1962d1cd910465c1c91d36c2df5c923fe7353fffb172c04adaebb368505 > > > SHA256 (FreeBSD-14.0-RC4-i386-zfs.qcow2.xz) = ab840c119d46ab49e2e33f7024f453ef872df908b47a8d9fdca0a8740b49c478 > > > SHA256 (FreeBSD-14.0-RC4-i386-zfs.raw.xz) = 1d2635935bdd73de8e76b8815392a2f0399782aa9e41296bc4dfb9e2073a35aa > > > SHA256 (FreeBSD-14.0-RC4-i386-zfs.vhd.xz) = 998a040e0849ed03a1cbf6da446672525c5ddee79ed225ff7cd8283164b484bf > > > SHA256 (FreeBSD-14.0-RC4-i386-zfs.vmdk.xz) = db11801f993246135e3cd4b28b6c7457853668180ff7d11c4b0c7018e04a58c2 > > > SHA256 (FreeBSD-14.0-RC4-i386.qcow2.xz) = 6ef1dfcea80aa0687141c7917f0e8ab81c4241a62bee8043ac1dc23c82be05ed > > > SHA256 (FreeBSD-14.0-RC4-i386.raw.xz) = 5eca8b0ffe97f7e6a485f496a61fad9c93a261c6c4e0d083f9fd1c500c2d4db4 > > > SHA256 (FreeBSD-14.0-RC4-i386.vhd.xz) = 3997fd89984392f2ca2fc3b7df864dfd0dfc17be750aa78d61a31233e947fc0b > > > SHA256 (FreeBSD-14.0-RC4-i386.vmdk.xz) = dda9a1962d1cd910465c1c91d36c2df5c923fe7353fffb172c04adaebb368505 > > > > > > > > > o 14.0-RC4 aarch64: > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.qcow2.xz) = 6f67f49ddf46bd4468394c985a2b34bc48f975f659de560a8ca1bd6935c3f27d02bc143632c088580aa440fae5f7f18932b5b335247c3aa328090ddbbc9c70b5 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.raw.xz) = ef0af5d2877eaa8e2a6c3ca9df9ac3b9a0e9f30035170eca3ceef1984d35b08ea422254583c08b45cda9453a3e09d7209d85b2093d69768e8c892be67ae5f975 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.vhd.xz) = 4b6bc513892b5ebd1cf2206014da0b548ddfbedb1a5fa8349fa5ddfd6363afb68f104bdc6d5447476f70ad61b27202a390cf6f5d7c5cf90c415edfe3ce0bc26d > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.vmdk.xz) = fece4ca62f30260beee2de4265f3c6559ba0a948e6dcf65658016ee10aa961136fbacc7238a6ea3b6e9afe334879cdfbcfe23f31d012a9abfca86c6783ea6a10 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.qcow2.xz) = cb21c41ad4e1e09e58bc00cc55c6e5decdf2d01e29ef7031e4497532bb33ced39473390141292d8819eb59f9f49b880c8d6d269d879c4f3b671e646968c83d32 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.raw.xz) = c33a58f30eb6f798cb353e11ad821ac9927852f6262617bb7340b963448ad83e4b4fe039676e1e48a5618db4e00ee6bc209b32809e3e70395114935189493e24 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.vhd.xz) = 4bb8cc4d313d14247701b7fa0bc920c2c5fd467126b961a5ecbc791c928730440c87dc452ecbc653db26fda482d86b0be87cd0f6ba272c2036ee3146096c70fc > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.vmdk.xz) = 5a49668bebd971fdc1145adb69edb5dd4aa521ca1a86d1532f43b58c1b3aed23c4187ce23366cc182fae296cc17a31b00858660522bb2897746f519ce5ddf3eb > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64.qcow2.xz) = 6f67f49ddf46bd4468394c985a2b34bc48f975f659de560a8ca1bd6935c3f27d02bc143632c088580aa440fae5f7f18932b5b335247c3aa328090ddbbc9c70b5 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64.raw.xz) = ef0af5d2877eaa8e2a6c3ca9df9ac3b9a0e9f30035170eca3ceef1984d35b08ea422254583c08b45cda9453a3e09d7209d85b2093d69768e8c892be67ae5f975 > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64.vhd.xz) = 4b6bc513892b5ebd1cf2206014da0b548ddfbedb1a5fa8349fa5ddfd6363afb68f104bdc6d5447476f70ad61b27202a390cf6f5d7c5cf90c415edfe3ce0bc26d > > > SHA512 (FreeBSD-14.0-RC4-arm64-aarch64.vmdk.xz) = fece4ca62f30260beee2de4265f3c6559ba0a948e6dcf65658016ee10aa961136fbacc7238a6ea3b6e9afe334879cdfbcfe23f31d012a9abfca86c6783ea6a10 > > > > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.qcow2.xz) = afbc7e756418a70a28c77d3a50022fedbce08a2dc3861a1faf19273d75d8d893 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.raw.xz) = 100ddb13b6952293e9ea6678b6b4dacb350642036d95b3c6aa2b6d03f653e3b2 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.vhd.xz) = 3c6c44ffd9dbacce38e41ee2a3dbea86c2a1630c4e0f1e4157d6a9ae35045330 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-ufs.vmdk.xz) = 725b3e4f834f3aa296efef28524dca72e6e3a366afe9a70f66f872446d620417 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.qcow2.xz) = b064cb8feeee25caf7b87634ec2d2111dc8f264dbc433e41249d2620b06f5c6b > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.raw.xz) = ebb407772250fdfb179bad97483233378fc63cc72ccd3dbcf01163968bfbd68e > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.vhd.xz) = 406a87f32a192a00077c071d295c749c166b8ad110d28f509a1f163d8b1c9358 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64-zfs.vmdk.xz) = c8be970e5337d282b45c097bb30b2d2a9eb469402cc9ffa6b7f2ee9c3bbfdb65 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64.qcow2.xz) = afbc7e756418a70a28c77d3a50022fedbce08a2dc3861a1faf19273d75d8d893 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64.raw.xz) = 100ddb13b6952293e9ea6678b6b4dacb350642036d95b3c6aa2b6d03f653e3b2 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64.vhd.xz) = 3c6c44ffd9dbacce38e41ee2a3dbea86c2a1630c4e0f1e4157d6a9ae35045330 > > > SHA256 (FreeBSD-14.0-RC4-arm64-aarch64.vmdk.xz) = 725b3e4f834f3aa296efef28524dca72e6e3a366afe9a70f66f872446d620417 > > > > > > > > > o 14.0-RC4 riscv64: > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.qcow2.xz) = 2538e1fa6d9c975d0ab2b680e634441209f8c2fdc2ad95f3431ead256185397cf7f639e2523753d4cee1b4f677499815538486b20d3684f9210e492dd0675b1a > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.raw.xz) = 96a49721ba1428b8cddf65abed849ff3567e31773fc2446f021dd67f61b92a231939a1d5abea28995d1ec8af9d8ba891b881f7ec8b6530a2182e570cb5e78042 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.vhd.xz) = d2aacdcefecee3cd8456d7f371a2f5254ddebbd22277d978b00baa9c23304239df1fcec89531644db575f0481b19ccdbfeaa66aa85f17e8651ec164afc4842a2 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.vmdk.xz) = e6bacea877352f5ff3fc16f5717e3944927753a5c64e7d237bf5c6c253f85dfe3011314d286198ae7c62d921cc2d7b00398767e207793c4b968f7071ea706858 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.qcow2.xz) = 8681d7bee0b60bee27e113c9b0c6601b640a8d49218280511d55aa38b6bcf1d873911de3beedae61be1c2e11244c384239c9d06bd8ebf7f081c0975621af44aa > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.raw.xz) = 2812d912f583dc574074c4c49f2d6005d60dbf0515c310a49858788bdea2b3f016091937acf67a80d1bcc123133f1f4585fd9a90ebd10b3fd2cfa2954d2a7b0b > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.vhd.xz) = 8cc13a0ccc40262c99e0dd8b71965cf7150e5ddab6fb754a8e46bb448ed7f8fda63aa79b132502cf890be0bc1141dddfebf77c3cb2d20e2edee050428cf138e8 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.vmdk.xz) = e71bb6d4f17e709289f6cc853a72e18a5b61c99e747064bae0e356ab9b1893cb97183ff971892eba1658bcf50bbea40c55827c579643ef88b2799b4f13d5d364 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64.qcow2.xz) = 2538e1fa6d9c975d0ab2b680e634441209f8c2fdc2ad95f3431ead256185397cf7f639e2523753d4cee1b4f677499815538486b20d3684f9210e492dd0675b1a > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64.raw.xz) = 96a49721ba1428b8cddf65abed849ff3567e31773fc2446f021dd67f61b92a231939a1d5abea28995d1ec8af9d8ba891b881f7ec8b6530a2182e570cb5e78042 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64.vhd.xz) = d2aacdcefecee3cd8456d7f371a2f5254ddebbd22277d978b00baa9c23304239df1fcec89531644db575f0481b19ccdbfeaa66aa85f17e8651ec164afc4842a2 > > > SHA512 (FreeBSD-14.0-RC4-riscv-riscv64.vmdk.xz) = e6bacea877352f5ff3fc16f5717e3944927753a5c64e7d237bf5c6c253f85dfe3011314d286198ae7c62d921cc2d7b00398767e207793c4b968f7071ea706858 > > > > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.qcow2.xz) = a20bb128dfcf977a8d8fcbe711287cde37204fed4656b015b55da484825207e3 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.raw.xz) = 9b8ceb27d0e67144c3e498f3c0d29857746147acf65262fbea319c5a8c7d038c > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.vhd.xz) = ee5dc591d890b717d5fef43081d0d9c7050a9d8ae8ce255d68b2a34ff123340b > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-ufs.vmdk.xz) = c37a5c0527a086fc52a81c4699ae0aea4b5a963fc71ce5836b0dc0cbe94fa152 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.qcow2.xz) = 23627ae8e397f0de495a0cbe75cec3d98053a0d450c1d8f95c400984f52d9add > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.raw.xz) = a38a35541f621e59404c32c8ec1f73f1f6a54f554a39230fcc82f963f365543f > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.vhd.xz) = be9c521b31c7443b46f99f24dab640b6c37da4cf2edee48fc6ecf815be0b8252 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64-zfs.vmdk.xz) = 43506386c53bb76ce5b5b67684a1f36da948fd31a92caa252fa492b3a94e9004 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64.qcow2.xz) = a20bb128dfcf977a8d8fcbe711287cde37204fed4656b015b55da484825207e3 > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64.raw.xz) = 9b8ceb27d0e67144c3e498f3c0d29857746147acf65262fbea319c5a8c7d038c > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64.vhd.xz) = ee5dc591d890b717d5fef43081d0d9c7050a9d8ae8ce255d68b2a34ff123340b > > > SHA256 (FreeBSD-14.0-RC4-riscv-riscv64.vmdk.xz) = c37a5c0527a086fc52a81c4699ae0aea4b5a963fc71ce5836b0dc0cbe94fa152 > > > > > > > > > o 14.0-RC4 amd64 BASIC-CI: > > > SHA512 (FreeBSD-14.0-RC4-amd64-BASIC-CI.raw.xz) = f553ef24fe92025eee845e54f25c1410883367beba2293c3244516218d9d9f5e3957f1d95643c57e4f97d7a6dc46d6534354ba1284f6a909a532d80e0fc260c4 > > > > > > SHA256 (FreeBSD-14.0-RC4-amd64-BASIC-CI.raw.xz) = 3d0eea50499d49020b9510a26d8df36d0f63ad66b8befb35e42b32943a44a7fe > > > > > > Regards, > > > > > > Glen > > > > > > Please consider donating to help support my FreeBSD work: > > > https://www.gofundme.com/f/gjbbsd > > > https://paypal.me/gjbbsd > > > > > > Love FreeBSD? Support this and future releases with a donation to > > > the FreeBSD Foundation! https://www.freebsdfoundation.org/donate/ > > > > > > > > > > I am having a problem witb Freeradius and GNU radius. > > > > Anyone else? > > > > > > I found the problem. Replace /usr/lib/libncursesw.so as a symbolic > link to /lib/libncursesw.so.9 can you provide more inputs here, please where do you get your freeradious or gnu radius implementation from, what is failing an so on? Best regards, Bapt From nobody Mon Nov 6 08:04:14 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SP3lh5vCnz50dKT; Mon, 6 Nov 2023 08:04:16 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SP3lh580Pz3Xnc; Mon, 6 Nov 2023 08:04:16 +0000 (UTC) (envelope-from bapt@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699257856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=j+DWhyZnNu4kqVbqAa5gCdhNSnfSod9vgUE0NMHqsUs=; b=adk+FVed3PmMHS53dDKLQD7WVz6lCrtJitlDrhu4Z+iJNZFlbvsaJcfVKYTvSdGNh5QWLt bPjwNqlAWeDyADQO3dyLZeNZr9zAb6Uh/EEDH3AswhJZLZYfbKdXpAXF8Tydz1cSDxQkNl GNosPVmBJmt9ETcUL6slfjZvcijbahCmzmpywlsLx07h4cV1L9X3Nb6lLdxOiI3UnD/ek/ FwciBfGJFJfn21f14gW9YacNsnRCOOGaWpHUZqyDGqAwB5pOVDXueFjyVVbsN7vjNtr/Gk bM5BbKZ8sTT3tKuGVJ/xuQ0BtxDnXSvrPXYIsz6SdHhFK5jTfwgPqmepDCflow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699257856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=j+DWhyZnNu4kqVbqAa5gCdhNSnfSod9vgUE0NMHqsUs=; b=YCPokmduqvbcQ9fPr1c9aUoHmSyiriLTxruoAHHCRpCgVteAgs2fkjVSVB84EmS94uN1kQ dKaIhKgcD1PFbkoAy0IcS9PN4yczUBkRi6SS9wOAnaqvZU2XEbkGlf0/pTwqpHqnDCe/Xi 1947Zwam1JnMWtTP3kg9qpYUhnvL+6xBC7J8nK0rjlc9MzherUW2uvPJ67wKe9q9WhBGcT 8c4vHm53WgNLlICgQYZBiF1QI6Okax+zfnFPoUOBSZVcyQMHNt94EyCVkVl03P7JLk4KOY f0UbTo4+xKrJZvBOQC9N0lpDn7DyMEuf9vPWs2+Hr56rFY/v9JcPPPAo5sWuWQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699257856; a=rsa-sha256; cv=none; b=qC6ypEBbdlIRskFumxSWLnUPdfHZVRCuZ3MtAv3bVT42qYUI4IaFVSvgRwfBrr5/r/V16H 0Zsq9auq0n9bkKV1sBK/oNoHmpYl7UkVqB95CKcJTgZMSivuEkiRp9qn4f8BXPGyhkmeG+ Ipd0TmIv4uQzlbiRpbsAdgzwYZkxa9cR3Cq/vLBd7nfU4W0ULGQpZnBHJtwlkKfHvhyfl1 nnsIy2NQlWzvrywumgiWsaNPkJUl4CimbuAH10j7Y4GA/5WFgMd1pWSVATf1t5OZ5r1RfE BZf9mIJJxnlAOPZgik9Vx++bJ4kqKlf2DeOKPe5wyQ4bQeNto1YY1AtVkuVTLA== Received: from aniel.nours.eu (nours.eu [176.31.115.77]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: bapt) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SP3lh3W00z17Tb; Mon, 6 Nov 2023 08:04:16 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: by aniel.nours.eu (Postfix, from userid 1001) id E89D819DB9E; Mon, 6 Nov 2023 09:04:14 +0100 (CET) Date: Mon, 6 Nov 2023 09:04:14 +0100 From: Baptiste Daroussin To: The Doctor Cc: Glen Barber , freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org, FreeBSD Release Engineering Team Subject: Re: Radius challenges in FreeBSD 14.0 Re: FreeBSD 14.0-RC4 Now Available Message-ID: References: <20231103234232.GK1289@FreeBSD.org> <5p6sszthbhskientm4tki4pq4hyzws4fnmzgq4cagh7yvkrwyo@hhmf5cy4gmsu> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Mon, Nov 06, 2023 at 12:50:43AM -0700, The Doctor wrote: > On Mon, Nov 06, 2023 at 08:43:05AM +0100, Baptiste Daroussin wrote: > > On Sun, Nov 05, 2023 at 12:11:06PM -0700, The Doctor wrote: > > > On Sun, Nov 05, 2023 at 10:32:44AM -0700, The Doctor wrote: > > > > On Fri, Nov 03, 2023 at 11:42:32PM +0000, Glen Barber wrote: > > > > > The fourth RC build of the 14.0-RELEASE release cycle is now available. > > > > > > > > > > Installation images are available for: > > > > > > > > > > o 14.0-RC4 amd64 GENERIC > > > > > > > > > > > > > > > > > > I am having a problem witb Freeradius and GNU radius. > > > > > > > > Anyone else? > > > > > > > > > > > > > > I found the problem. Replace /usr/lib/libncursesw.so as a symbolic > > > link to /lib/libncursesw.so.9 > > > > can you provide more inputs here, please where do you get your freeradious or > > gnu radius implementation from, what is failing an so on? > > > > Best regards, > > Bapt > > freeradius looks like a TLS issue with openssl 3 > > . > > gnu Radius was biuld by me and not a port. > > Trying to recode is a challenge. > > What is happening is that > > the so file is not being recognised so > I have to symlink in order to get GNU radius 1.6.X to work . > > It will work with /lib/libncursesw.so.9 if the so file is found in > /usr/lib . OK I will dig into gnu-radius, and fix the port if needed! thank you Best regards, Bapt From nobody Mon Nov 6 08:50:18 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SP4mr2Hlgz4ypsT; Mon, 6 Nov 2023 08:50:20 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SP4mr1rKcz4F4Q; Mon, 6 Nov 2023 08:50:20 +0000 (UTC) (envelope-from bapt@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699260620; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=99cbVRInoTXI6COvgeS8ZG85p+Srwd36eoKciMbT2f4=; b=JG+eA1+JWmamvEvRNdcwe9eDZ7mVjsD5A6QVVuufnMMk7Y4K+KDtCxrQm4eYBWi711daot X1lGM1UA95Mfn6rK/EafMcwOemk0v7J+H2ERE3+byWv/AkRxk+hSPo7M6YfS6zaLsgwRP3 wIiRdHmdrzuk8Z032tVdVPkWrJgqCQVzD9cJZpqW1YwMDIsMX3CiCRtw6bgxh5QvQ6PWTw NHQUJkAykcck3wKh93TDJrhLQJjGcjDCvogFEW6+Trd7inxG8+DxbiBCVObxSkn2N2ypWF Jk5yW8zQPjtS8dNA97HItxKUTCATdWl2MqCLMEQeI/rgO1EPYvqnF2oJeVcThA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699260620; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=99cbVRInoTXI6COvgeS8ZG85p+Srwd36eoKciMbT2f4=; b=PgkUQw525ZmXFchtIdyJrtnsD39qhEmJX8e0xhLuQEN74FsZ5OUHyI8XJoaABqWvUPnQvS /m7WCaKkzt9Zg3ecNP2KUuYPA0lcrsludFQB81YKRRvGqpeL4U4PEYb8KLJGWcc9plZtuw lTcKP7TuHvIgATFcmLya2ddT4NiIISbxZtHvrK6qHGOedBFmnb1M12oEhUXtZ1SNwggTfU pWu2F+1iQ6JrNcb7taqXq48Jd9MQXyH0g3iqlANpEPQn+KgnsOfjV03nfb1M27jIiz0SzZ MZs7Jf121jLcUDS4HmS+E+G3rWG010En9tgfTkqyAs0uRrvfy6WFPDV3Rn1qXA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699260620; a=rsa-sha256; cv=none; b=ig9SVzN8IS36/Quhxb5DUJBoLNhuxs5QpotZagHDPT4HtqO1yVeuwA5r0lZYI2L7wSe212 zvpAGs15ia2mfWWZLaYksFSEfkEi71tb4UT78KUnPxSpBixGuia0TBhbdC+7NCJh4fBrR1 FibTTRoMfAQl4RTtF1doNQHNmOl+e++gDNBnzvaDj3ymbE0Tm1DtecVOQj356So9qdBLnk r+KMPxZRLNGyoXIAlX5S7FlRO2LfOFNvr72MFEp1R7/bR7384MXk6L+CyNZSHDXkIpqIxQ pWsMdxGqTHL7se9Yd7J+j8rMB9ekIFt1Mb6GNxWt0z0Wt83yPyfSBTtzaFnY4w== Received: from aniel.nours.eu (nours.eu [176.31.115.77]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) (Authenticated sender: bapt) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SP4mr0Cg4z18Hh; Mon, 6 Nov 2023 08:50:20 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: by aniel.nours.eu (Postfix, from userid 1001) id 1150919DC16; Mon, 6 Nov 2023 09:50:18 +0100 (CET) Date: Mon, 6 Nov 2023 09:50:18 +0100 From: Baptiste Daroussin To: The Doctor Cc: Glen Barber , freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org, FreeBSD Release Engineering Team Subject: Re: Radius challenges in FreeBSD 14.0 Re: FreeBSD 14.0-RC4 Now Available Message-ID: References: <20231103234232.GK1289@FreeBSD.org> <5p6sszthbhskientm4tki4pq4hyzws4fnmzgq4cagh7yvkrwyo@hhmf5cy4gmsu> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Mon, Nov 06, 2023 at 09:04:15AM +0100, Baptiste Daroussin wrote: > On Mon, Nov 06, 2023 at 12:50:43AM -0700, The Doctor wrote: > > On Mon, Nov 06, 2023 at 08:43:05AM +0100, Baptiste Daroussin wrote: > > > On Sun, Nov 05, 2023 at 12:11:06PM -0700, The Doctor wrote: > > > > On Sun, Nov 05, 2023 at 10:32:44AM -0700, The Doctor wrote: > > > > > On Fri, Nov 03, 2023 at 11:42:32PM +0000, Glen Barber wrote: > > > > > > The fourth RC build of the 14.0-RELEASE release cycle is now available. > > > > > > > > > > > > Installation images are available for: > > > > > > > > > > > > o 14.0-RC4 amd64 GENERIC > > > > > > > > > > > > > > > > > > > > > > I am having a problem witb Freeradius and GNU radius. > > > > > > > > > > Anyone else? > > > > > > > > > > > > > > > > > > I found the problem. Replace /usr/lib/libncursesw.so as a symbolic > > > > link to /lib/libncursesw.so.9 > > > > > > can you provide more inputs here, please where do you get your freeradious or > > > gnu radius implementation from, what is failing an so on? > > > > > > Best regards, > > > Bapt > > > > freeradius looks like a TLS issue with openssl 3 > > > > . > > > > gnu Radius was biuld by me and not a port. > > > > Trying to recode is a challenge. > > > > What is happening is that > > > > the so file is not being recognised so > > I have to symlink in order to get GNU radius 1.6.X to work . > > > > It will work with /lib/libncursesw.so.9 if the so file is found in > > /usr/lib . > > OK I will dig into gnu-radius, and fix the port if needed! thank you > > Best regards, > Bapt I just checked here, and I built gnu-radius on a vanilla freebsd 14.0 rc4 and a vanilla 15 current, and in both case it perfectly links to libncursesw.so.9 and does not require any change of the .so, at least from all the binary analysis that I have done, do you have a specific command that will expose the issue? Can you provide me you gnu-radius package as created by pkg create gnu-radius ? Best regards, Bapt From nobody Mon Nov 6 14:09:22 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPCsG3d46z4yjJg for ; Mon, 6 Nov 2023 14:09:38 +0000 (UTC) (envelope-from jon@xyinn.org) Received: from mail-4317.proton.ch (mail-4317.proton.ch [185.70.43.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPCsC5ZJqz3Cmp for ; Mon, 6 Nov 2023 14:09:34 +0000 (UTC) (envelope-from jon@xyinn.org) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=xyinn.org header.s=protonmail2 header.b=MTEXrbtf; spf=pass (mx1.freebsd.org: domain of jon@xyinn.org designates 185.70.43.17 as permitted sender) smtp.mailfrom=jon@xyinn.org; dmarc=pass (policy=none) header.from=xyinn.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=xyinn.org; s=protonmail2; t=1699279771; x=1699538971; bh=QWlWAtc8Sc7ShBcQDVnPYrEhhj9iRcLQvdbhlZji2hY=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=MTEXrbtf7Wie7ue5LbMAEai+FNCerW644nSwJ0r2g1Ie6VWJLJa7wjEyL9TM31AqJ vwySs/8pqlLhaal9j/WvSxG2NECpB7HLo7ivzOFiN7vTuQRrog+mQdK+PhmyyfBlBf pwB02VNAYyExUFhTYsmxC+bwuotX0L+uwBvQ5yiXWNNeAA4zmVJ0L+b0/803qYygRy CVa6ppd6A21UIrLfMzKMLAGZo5Oy5FwE3oG0g5LflBLRympGjbezh3dSQE/aVVplAs CeI6kd4LJiHuFdaYuluwgDFa0hmEJd0/8wTJpldJWMdGrTpqCpHXae1xO2a28O2DgV HPexws4AIcGtA== Date: Mon, 06 Nov 2023 14:09:22 +0000 To: freebsd-stable@freebsd.org, FreeBSD Release Engineering Team From: Jonathan Vasquez Subject: Re: FreeBSD 14.0-RC4 Now Available Message-ID: In-Reply-To: <5p6sszthbhskientm4tki4pq4hyzws4fnmzgq4cagh7yvkrwyo@hhmf5cy4gmsu> References: <20231103234232.GK1289@FreeBSD.org> <5p6sszthbhskientm4tki4pq4hyzws4fnmzgq4cagh7yvkrwyo@hhmf5cy4gmsu> Feedback-ID: 12351801:user:proton List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Result: default: False [-4.40 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-1.00)[-0.999]; DMARC_POLICY_ALLOW(-0.50)[xyinn.org,none]; RWL_MAILSPIKE_EXCELLENT(-0.40)[185.70.43.17:from]; R_SPF_ALLOW(-0.20)[+ip4:185.70.43.0/24]; R_DKIM_ALLOW(-0.20)[xyinn.org:s=protonmail2]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[xyinn.org:+]; FREEFALL_USER(0.00)[jon]; RCPT_COUNT_TWO(0.00)[2]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Rspamd-Queue-Id: 4SPCsC5ZJqz3Cmp X-Spamd-Bar: ---- Upgrade from RC3 -> RC4 seems to have gone smoothly on Framework Laptop (20= 21). From nobody Mon Nov 6 15:31:04 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPFgG4lYDz501T3; Mon, 6 Nov 2023 15:31:06 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPFgG33Lgz3XxY; Mon, 6 Nov 2023 15:31:06 +0000 (UTC) (envelope-from bapt@freebsd.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699284666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=f2ms3vFo8VAm83zMqKX4t4o8vd/n1g1rUSI+F+a3iEE=; b=bAmnUS7B5Uk9msTQolBgjVD1Rr/kLQHSm76lQgZUDZMQ+LIrijxzDTPjVlj8kIMrUGUGXW AvLBioq3Es6sr+26aVFxRBZzwhpAbhZycVE3Y9HTyzmH+wSVh/GR5m3TmdNZ+CUF2rY+OJ XF6ia16iYM3gmdPLTxrYtA0VR+zogd9lJlUJXSBOEieI6LPnxdLC1GCKJ8i+Myuqk9s9LL Vq5nHArbWF3Pg8JLQZesvyv40O7JjIkfY4aUk58a8YMI+awxd8FHYbVb+PKO90nrGFFvxF vzSZierkNuj125w1aVJMtBuKYOIr31iJgQt0vrxQWK2jp7N9e8upKGTwVjawdA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699284666; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=f2ms3vFo8VAm83zMqKX4t4o8vd/n1g1rUSI+F+a3iEE=; b=t3Mkmf5vBNykIbZXwUi0FPPjLL0+fVj8XAPn9y1P2ecRYRCEwdZEmb0pQN47qO+7CZtkBm AK6wUjYbmmNjUWBuX+FpUX/ZqwmNcNd4cV8nxesbqvXMzMFuS8QhuXWX8yFgokwdbFrsDu zsizf/fWZv5dbWsfgs38kEb++tomaRd9plZ9L1jG18lSU4qY4nYjEpQOSximxzWi3TxUnL V0CeMCMEak0OsWyn8XcM3d/xQ1NjyuGnfksncd5F4BZcOV1bizVlpPh8amZq/Wfezs4slL sh+NmuVTch/MC3dXY66RWEhyTQ0yJvUIlloRbEKk+QPRjILJjyGu24Au/SAmIg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699284666; a=rsa-sha256; cv=none; b=XHoQiKHEpsnxpSSwB/YjGvFyHqcrVdteGfrw+4+HrbgNl/odMYzm0PI2pyI6mgZxScc75I CH+gsf5IYcSkyJ1prpfnmgGv1wUGv2lkgqiGBnxQ1vyIVzoi9pZvfjHJ/JavGuC8wCK4/P ZQYnSUAZtKpgaNIM3uvLh21I154MFn9/9q40oaQqF80Iqsj9a4AJfpYEUjztC62Sje807x lJcSWNtTYUYg3xtn861BdlP4osYUhUZ852yfuEyB8GvGblnF5GcsWT3v80hhoSCZiW/IuK Yl7hXh39Cxp3Pt+3pAdXfJGJKAolAuY89fg4GFgYrYCnIhBfGytZDXLiLDXtuA== Received: from aniel.nours.eu (nours.eu [IPv6:2001:41d0:8:3a4d::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: bapt) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SPFgG0mJbz1Gf1; Mon, 6 Nov 2023 15:31:06 +0000 (UTC) (envelope-from bapt@freebsd.org) Received: by aniel.nours.eu (Postfix, from userid 1001) id CAFDB19EFD3; Mon, 6 Nov 2023 16:31:04 +0100 (CET) Date: Mon, 6 Nov 2023 16:31:04 +0100 From: Baptiste Daroussin To: The Doctor Cc: Glen Barber , freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-snapshots@freebsd.org, FreeBSD Release Engineering Team Subject: Re: Radius challenges in FreeBSD 14.0 Re: FreeBSD 14.0-RC4 Now Available Message-ID: <4k7sqgsiolglrfq35lesmb4qr7l7lcbnmebcdaxs4n7so2vubn@56xwzstht5dy> References: <20231103234232.GK1289@FreeBSD.org> <5p6sszthbhskientm4tki4pq4hyzws4fnmzgq4cagh7yvkrwyo@hhmf5cy4gmsu> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: On Mon, Nov 06, 2023 at 08:27:34AM -0700, The Doctor wrote: > On Mon, Nov 06, 2023 at 09:50:18AM +0100, Baptiste Daroussin wrote: > > On Mon, Nov 06, 2023 at 09:04:15AM +0100, Baptiste Daroussin wrote: > > > On Mon, Nov 06, 2023 at 12:50:43AM -0700, The Doctor wrote: > > > > On Mon, Nov 06, 2023 at 08:43:05AM +0100, Baptiste Daroussin wrote: > > > > > On Sun, Nov 05, 2023 at 12:11:06PM -0700, The Doctor wrote: > > > > > > On Sun, Nov 05, 2023 at 10:32:44AM -0700, The Doctor wrote: > > > > > > > On Fri, Nov 03, 2023 at 11:42:32PM +0000, Glen Barber wrote: > > > > > > > > The fourth RC build of the 14.0-RELEASE release cycle is now available. > > > > > > > > > > > > > > > > Installation images are available for: > > > > > > > > > > > > > > > > o 14.0-RC4 amd64 GENERIC > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I am having a problem witb Freeradius and GNU radius. > > > > > > > > > > > > > > Anyone else? > > > > > > > > > > > > > > > > > > > > > > > > > > I found the problem. Replace /usr/lib/libncursesw.so as a symbolic > > > > > > link to /lib/libncursesw.so.9 > > > > > > > > > > can you provide more inputs here, please where do you get your freeradious or > > > > > gnu radius implementation from, what is failing an so on? > > > > > > > > > > Best regards, > > > > > Bapt > > > > > > > > freeradius looks like a TLS issue with openssl 3 > > > > > > > > . > > > > > > > > gnu Radius was biuld by me and not a port. > > > > > > > > Trying to recode is a challenge. > > > > > > > > What is happening is that > > > > > > > > the so file is not being recognised so > > > > I have to symlink in order to get GNU radius 1.6.X to work . > > > > > > > > It will work with /lib/libncursesw.so.9 if the so file is found in > > > > /usr/lib . > > > > > > OK I will dig into gnu-radius, and fix the port if needed! thank you > > > > > > Best regards, > > > Bapt > > > > I just checked here, and I built gnu-radius on a vanilla freebsd 14.0 rc4 and a > > vanilla 15 current, and in both case it perfectly links to libncursesw.so.9 and > > does not require any change of the .so, at least from all the binary analysis > > that I have done, do you have a specific command that will expose the issue? > > > > Can you provide me you gnu-radius package as created by pkg create gnu-radius ? > > > > Best regards, > > Bapt > > Try to build from ports, here is what I get > > cc -DHAVE_CONFIG_H -I. -I.. -I.. -I../include -I../include -I../include/radius -I../include/radius -I../lib -I../gnu -I../gnu -I/usr/local/include/guile/1.8 -I/usr/local/include -isystem /usr/local/include -D_THREAD_SAFE -DSYSCONFDIR=\"/usr/local/etc\" -DRADPID_DIR=\"/var/run\" -DRADLOG_DIR=\"/var/log\" -DRADIUS_DATADIR=\"/usr/local/share/radius/1.6.1\" -DRADIUS_LIBDIR=\"/usr/local/lib/radius/1.6.1\" -I/usr/local/include/ -I/usr/local/include -fno-strict-aliasing -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -MT builddbm.o -MD -MP -MF .deps/builddbm.Tpo -c -o builddbm.o builddbm.c > builddbm.c:101:13: warning: call to undeclared function 'grad_dbm_insert'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] > if (grad_dbm_insert(closure->dbmfile, named, contentd)) { > ^ > builddbm.c:141:13: warning: call to undeclared function 'grad_dbm_create'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] > if (grad_dbm_create(db_file, &closure.dbmfile)) { > ^ > builddbm.c:147:39: error: incompatible function pointer types passing 'int (DBM_closure *, User_symbol *)' (aka 'int (DBM_closure *, struct user_symbol *)') to parameter of type 'int (*)(void *, grad_symbol_t *)' (aka 'int (*)(void *, struct symbol *)') [-Wincompatible-function-pointer-types] > grad_symtab_iterate(user_tab, append_symbol, &closure); > ^~~~~~~~~~~~~ > ../include/radius/symtab.h:48:11: note: passing argument to parameter 'fn' here > int (*fn)(void *, grad_symbol_t *), > ^ > builddbm.c:243:13: warning: call to undeclared function 'paircmp'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] > if (paircmp(req, check_tmp) == 0) { > ^ > builddbm.c:252:23: warning: using the result of an assignment as a condition without parentheses [-Wparentheses] > if (p = grad_avl_find(check_tmp, DA_MATCH_PROFILE)) { > ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > builddbm.c:252:23: note: place parentheses around the assignment to silence this warning > if (p = grad_avl_find(check_tmp, DA_MATCH_PROFILE)) { > ^ > ( ) > builddbm.c:252:23: note: use '==' to turn this assignment into an equality comparison > if (p = grad_avl_find(check_tmp, DA_MATCH_PROFILE)) { > ^ > == > builddbm.c:308:35: warning: passing arguments to a function without a prototype is deprecated in all versions of C and is not supported in C2x [-Wdeprecated-non-prototype] > (*fn)(buffer, sizeof(buffer), name, i), > ^ > builddbm.c:323:23: warning: using the result of an assignment as a condition without parentheses [-Wparentheses] > if (p = grad_avl_find(*reply_pairs, DA_MATCH_PROFILE)) { > ~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > builddbm.c:323:23: note: place parentheses around the assignment to silence this warning > if (p = grad_avl_find(*reply_pairs, DA_MATCH_PROFILE)) { > ^ > ( ) > builddbm.c:323:23: note: use '==' to turn this assignment into an equality comparison > if (p = grad_avl_find(*reply_pairs, DA_MATCH_PROFILE)) { > ^ > == > builddbm.c:336:22: warning: call to undeclared function 'fallthrough'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration] > if (!fallthrough(*reply_pairs)) > ^ > 7 warnings and 1 error generated. > gmake[6]: *** [Makefile:743: builddbm.o] Error 1 > gmake[6]: Leaving directory '/usr/ports/net/gnu-radius/work/radius-1.6.1/radiusd' > gmake[5]: *** [Makefile:641: all] Error 2 > gmake[5]: Leaving directory '/usr/ports/net/gnu-radius/work/radius-1.6.1/radiusd' > gmake[4]: *** [Makefile:745: all-recursive] Error 1 > gmake[4]: Leaving directory '/usr/ports/net/gnu-radius/work/radius-1.6.1' > gmake[3]: *** [Makefile:621: all] Error 2 > gmake[3]: Leaving directory '/usr/ports/net/gnu-radius/work/radius-1.6.1' > ===> Compilation failed unexpectedly. > Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to > the maintainer. > *** Error code 1 > > Stop. > make[2]: stopped in /usr/ports/net/gnu-radius > *** Error code 1 > > Stop. > make[1]: stopped in /usr/ports/net/gnu-radius > *** Error code 1 > > Stop. > make: stopped in /usr/ports/net/gnu-radius > This is not a ncurses issue. Bapt From nobody Mon Nov 6 18:00:49 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPK055BMSz50TF9 for ; Mon, 6 Nov 2023 18:00:53 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic309-21.consmr.mail.ne1.yahoo.com (sonic309-21.consmr.mail.ne1.yahoo.com [66.163.184.147]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPK04519Yz4SxZ for ; Mon, 6 Nov 2023 18:00:52 +0000 (UTC) (envelope-from filippomore@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=hPoA02f5; spf=pass (mx1.freebsd.org: domain of filippomore@yahoo.com designates 66.163.184.147 as permitted sender) smtp.mailfrom=filippomore@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699293650; bh=x5p3fe0PxHIvz2Nx4cLRuAnHzcXILJyoXwVWlKrIm8Y=; h=Date:From:To:Subject:References:From:Subject:Reply-To; b=hPoA02f5q7nvKsmr5MsTOCDopW0PwPcV2s9O8ZIVDpOQgraBNOCI07+Il+pLqZcvAY0FkJivc7ggLQJK3eNle3U1ormW2/qo4DPwQgk3hflSqehzTX+/5vna1BMMproBIrW2r9zcUS/Z3JCAvbzwKxfRFwRyvnNiaeGneZUTZOBAjUH/MIZ/g7VEZ0unv9902s6kxvozbrOzLmSvjNn9mZIqNhsa1DXujHr1NuAx9GqXQTHLAYCdmpjjaAG+L01chahtxQZRpOgqN6/ZiGKv0V803LC+p4+tKSnOr2frLivd5dPOWbg1CiGLHV65gj4n0A/o8w30c7AkJht/T6qqzQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699293650; bh=9sR9gdrUg7Qs5RMJXcdVeguUDQ2YFUdvYkAobyyfM6G=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=bOAiZ4axT7lYKPIp14hpU/eVsPAw9VW8ndPCcay3ABHK2uptjuhf/VaunzSH2ZMxXdUrEC5US+6Zc6vfcuWw8EXWalAAKCG4UPuDuArUQsJyt62dNYGVxtCAKYGegqkJ8Pg6ZI+3UqmtjeMz6bOTZJrqzzta2MYX/z6wplDEhaoiq1QfKCBLnDMfTi7M6CAhyudSehly/Bd1Q+YTdtyCU5pka0iNQUbLFL1ctsfh3LeCkVRgwujvQJM4VCjaNvGcbE3TwxBDwPYBCc/8hK9XiOXqC7KNcVMJ1Sl9tn6WmaxGVEteaqq9P5aiTUHdg9yuGhzlK6PV5xjdBETATLhY1g== X-YMail-OSG: T2AjBqwVM1kFasf38GjDOmNvLWxTW1o0ZbRudFNytidZigvdMbJAHo7fxP6QiBj WznehFz5DS6I.TVkOZP.6QiowU.wajhFWHk.J1rQCd2dTiYVdVE3Twei7VligH0.hJr2Uq1NH0Te .VTq1yhADh5M9RnyUwTUA6JKObN9TPWMai5styv7oW.2lQ65OCrp3W72L_BCz9YQ.anho_dHP1dm VwBF_.b6RgRE6CeoTwKMWkVe9SgUfrq1B_zG7ogNS9PGnuRJn.1i.E1nNUl5O0CXT.SyE5hORKa0 gTRat1Af2xx6_eqDItXtnmOsL3VlGZ97sZCbQ3VbaNSln42LXOuLrtr_pJGH0vJxZRL4hC5aozuP So45M_iO6uFwoPNHy5OtmV1nqGicuXOLMQWgwlwrwP.tr2F2FjQuYcQf8dkbmTroUFLwsB8C34EN WvSG51jd9sc7a85qxoix7VFWGM9B.Uv0hoi23NXmZxwFHbPNvQyOFQdS5vtOVShdhV.GFLGViLyN l_EYCnaB_xasl2UC.gqOlIczG853gkJQk7PEAe3oQykliRf8j1jalikvHBWo5kdwoOpHqpt2Z6Xg 08zPHxJRlrzdyWu0WK5J0mH4VU8yMB7YM4tJrNdHL9y9H4iGa9m.owqZt5PLCw83igUw5akTtbNb WLSWXqB_W4p9eGE3JQ68J.INeXwm2czabUj1f93LsQLKxfWgRPDmv2Zy_X1eG5FGJQYZA_Vtk_BI NPqDn52tASzO1tNU7pbl1z0j.2BdoVleIaDSTV7rDQZsN7G6fpiujbUIsJh5zeXtyQwmOEA68WMD Jra_Mv3NtJyegpw7H20W9sfmoa9Rls2Ow1gw7y7MVzuBuZX.5d8PPDdDadDJ9L3Zo_s91vKQ_Hya HPM8dOk7iwMFad.4_svL82qqfDbP4tb6HPswzzsp7FFP8g_FDmQM1mcEMmoYhlDEYYauYD59HC5m 9RSrQ.SYAdVXddFgSndQ.nupo8vFWrSUokEYhv3NjEPEmaYgmKFaozOamehhKBrUHDZgnWGtFRWR zQool78DKRoz7kl7leQjaYJ0Q8_oKxCbY12hdD7AqO7j01sV8gAFSY6dJ.WcGe.9oT0UEQ.8gh6A 8Q7GdzXJ6IRwHGDZ768T3tOdhnSDhhRRsp3xA7B45d8rMGOusN.ORAAE6VbvGYUKX5DxdxoB6zd7 1uEq.n0Rlc0Qqx4KKRsM2hQFRgOT.s7SdBc1VsAT4pgS2OHJXwCGkAkDbGhdEKDHGwLOIm2OI4ZI oG3_4HcxdvZKvy9cBHXFAAKz_5wny_q2Q7KT9LjFRfi_XHx_pEPcDU6cPw8XhJS48fNeQ_QLCHQi BbuA_QdirTUo2r37xOGN09m77mrZMrmiq0Uif.6xSZAo.Tjae2ffBh2jqt0IFf96HftbOilU01QK 9GztILoWFFcz4ukTh239H0yhsZd_MI2CIu0200CPB4uIoqo1G.x7Qn59ZMq_uoTX2Y6TxmO4rzX1 rsLIAd0UKoGMvKnUYRJWRxmGXjwkwaj5fBFQBIp4HeWcwpj4fc.TApDsfOsZ6CqbTbB1ME5z_fAY 6kmB0n7Cfm3WezTbnK0H_XZXdyOjLRVpZm4gbS4XqsnRARsfhVzLF9ZcEtha1uPCzLNGqzIT3vPx UQQuTaQWVyS0reT3e7ErcZLpVyQ15kXlS3qcYSaCH7I221fqKQ_sBivCHlI5nLtTjVHvC.5fGfQ1 UqoLMloAz6TXepvSAebbDAROAlGdpJ.dDQKjDCNMP8xApgdJ8yyXVKRFL2EBssYX2QeWGYVqImr_ npzewsizaX0n2TTPl60qg1tz0H5Q5tXiRk1TMVIarbe7FNGoD3Hx6Ucj1ZvvvCDl65VldevzsUrh Yzdo832dfaCOJnbsXuy59u7yItvfCWjRrpHskQSrmWABctyoFPi3B_wXc4dqFpxju55tr99Xz62t r5.aqxUV6g7jInWITAJ_deznpYPWxmIuRL2_N364rrfCI4eSmwts5v4N52YlebNly60HQm7t09cC p0JLsntXRPTL6B39yAH3izJNDiTHtGntelQBsgndFBmcmyRmY_Jl3gacMEuHDBnyAfQm1DDrjAlA QaSfGigU4Jy2DqtUbLQrMdsf8foAMqjXcBno.ZXPK97Hv5erbk1fAwt25UJ_FoHCZOjU7EEESnoS dCgfc3bYP8OrLw3OAQiEg X-Sonic-MF: X-Sonic-ID: 0640893a-31ff-4660-bdea-29ce38e4180f Received: from sonic.gate.mail.ne1.yahoo.com by sonic309.consmr.mail.ne1.yahoo.com with HTTP; Mon, 6 Nov 2023 18:00:50 +0000 Date: Mon, 6 Nov 2023 18:00:49 +0000 (UTC) From: Filippo Moretti To: Freebsd-stable List Message-ID: <17452825.3912917.1699293649529@mail.yahoo.com> Subject: Problem with iridium List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_3912916_2091313255.1699293649528" References: <17452825.3912917.1699293649529.ref@mail.yahoo.com> X-Mailer: WebService/1.1.21890 YMailNorrin X-Spamd-Result: default: False [-1.46 / 15.00]; NEURAL_HAM_SHORT(-1.00)[-0.998]; NEURAL_SPAM_MEDIUM(0.88)[0.877]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_LONG(-0.34)[-0.340]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[stable@freebsd.org]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_IN_DNSWL_NONE(0.00)[66.163.184.147:from]; ARC_NA(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FROM_HAS_DN(0.00)[]; RWL_MAILSPIKE_POSSIBLE(0.00)[66.163.184.147:from]; DKIM_TRACE(0.00)[yahoo.com:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; FREEMAIL_FROM(0.00)[yahoo.com]; MID_RHS_MATCH_FROMTLD(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-Rspamd-Queue-Id: 4SPK04519Yz4SxZ X-Spamd-Bar: - ------=_Part_3912916_2091313255.1699293649528 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable FreeBSD STING 14.0-STABLE FreeBSD 14.0-STABLE #2 stable/14-n265685-52415bfc= 0adf-dirty: Mon Nov=C2=A0 6 17:32:23 CET 2023=C2=A0=C2=A0=C2=A0=C2=A0 root@= STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64 When starting iridium I get Trace/BPT trap.There are no core dumps and noth= ing in /var/crashIt was compiled from ports without changing defaults.Filip= po ------=_Part_3912916_2091313255.1699293649528 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit
FreeBSD STING 14.0-STABLE FreeBSD 14.0-STABLE #2 stable/14-n265685-52415bfc0adf-dirty: Mon Nov  6 17:32:23 CET 2023     root@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64
When starting iridium I get Trace/BPT trap.
There are no core dumps and nothing in /var/crash
It was compiled from ports without changing defaults.
Filippo

------=_Part_3912916_2091313255.1699293649528-- From nobody Mon Nov 6 21:52:46 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPQ802mzfz50K3g for ; Mon, 6 Nov 2023 21:53:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPQ802GNcz3fgl; Mon, 6 Nov 2023 21:53:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699307584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZF2NYaXabNuJdWFxbvs0OccHVDqv0n48DL4hHYTxrMc=; b=f59IIZpEDGQ0QD4S25Q0yQc3/gRSrYZSDFe/K3Q/RCa+H1rmUiHDbRiyyHpXqm7i7nIWq+ 8YFFRuMEpEsr/X3dGa0/G/IeQU+XslkGvj7HgfO745pvjABoXhLSvwFZRYP9YXfnmqva2V tY7ZSWteK7PrI09fGaJeg5Dhl0uGekKEYppoedrW41RntIGGHlh/tEWHHqUIKntHzQmldZ CG/TNx8fLfFHycu0DRW+5cYA+OZ0f5FXesEu2BZ2q/vk0nkvVySFiYC8ua2J1vp9hgDnCG dYxbB/TxEDCV9ZQVhxkNTrCBCevioHFwfj5SdTtur2ne7BmAtNClA6ig7NwEUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699307584; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZF2NYaXabNuJdWFxbvs0OccHVDqv0n48DL4hHYTxrMc=; b=kOFJi9RAjmDCfxLOlsE7/+ULc1tH3m/LcWJ2DhuRZlypJaN4fhHfaWlh+njFQlKUyWPWkA bz0UF1D6C3zqF3EaG72xFWTPrT6Dw+5o/MQ9ElCBcXhQfyKnm1AUFPZJjh61eQ+eDpfQL5 E7lNo+4gRN5z7avx25jzXC8bNcjwbACC7otY33RQ28Gbjh11EPq4mUXpaKTQ63xZCPOsOh 85bcHYloFm4sBZnJiEPa82R/QeqCBmD9R7s6uxiph7TsAdyHscEpejmqsUn/dmDCW3Zgum PyFngNfT/O1MBuyXqKOfrTexOPz0Bma8xD5tNfRx2ucXksLkaIbwp8VhClNHnw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699307584; a=rsa-sha256; cv=none; b=rCovkT08OOUeK3hUcOPcsUHJ0k+9r+zVYPmp4dIYXXuqOllDqEdQM4n5BZ4MGE8S2fevoZ CIrvJks/yT4QnwjfEeHMP3B6D/szlakvEkjF2PfdhnNKsf5UjJwyRa2855oB8ZdaMg4ouB fwxPwd7SwhmedbosH1sn2gK9FGA1fy18GnTMLtvu4VT/gRmHL1IVTA1LMnXBT5kcmZKqt8 Fcus6dBecDeDBUDtC7tdFlMGC+wx/yeznPTnUZDttRwByiL4oy0vMINx++egWjy/0j2i1J 642B7R9KatyO+SUTBTnzRH3nmadfKYRvYbbzPBpGFVapODMhzVo7E4gGoradnA== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SPQ800Y4Mz1NBZ; Mon, 6 Nov 2023 21:53:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (longrow.home.andric.com [192.168.0.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id D111F785A2; Mon, 6 Nov 2023 22:53:01 +0100 (CET) Content-Type: multipart/signed; boundary="Apple-Mail=_5363D21A-5219-4053-BD7F-C40788599CE2"; protocol="application/pgp-signature"; micalg=pgp-sha1 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Problem with iridium From: Dimitry Andric In-Reply-To: <17452825.3912917.1699293649529@mail.yahoo.com> Date: Mon, 6 Nov 2023 22:52:46 +0100 Cc: Freebsd-stable List Message-Id: References: <17452825.3912917.1699293649529.ref@mail.yahoo.com> <17452825.3912917.1699293649529@mail.yahoo.com> To: Filippo Moretti X-Mailer: Apple Mail (2.3731.700.6) --Apple-Mail=_5363D21A-5219-4053-BD7F-C40788599CE2 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii On 6 Nov 2023, at 19:00, Filippo Moretti wrote: >=20 > FreeBSD STING 14.0-STABLE FreeBSD 14.0-STABLE #2 = stable/14-n265685-52415bfc0adf-dirty: Mon Nov 6 17:32:23 CET 2023 = root@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64 > When starting iridium I get Trace/BPT trap. > There are no core dumps and nothing in /var/crash > It was compiled from ports without changing defaults. Could be that it is running into an "unreachable" instruction? Can you = run it under lldb or gdb? -Dimitry --Apple-Mail=_5363D21A-5219-4053-BD7F-C40788599CE2 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCZUlgLgAKCRCwXqMKLiCW o7O8AKCK0QYVV57eoUAAZTrOOSG1QKwTjQCgq1kaP4EYZtB8txKxs+vk0KipZJE= =IUqM -----END PGP SIGNATURE----- --Apple-Mail=_5363D21A-5219-4053-BD7F-C40788599CE2-- From nobody Tue Nov 7 13:46:02 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPqHj5bSwz507Yl for ; Tue, 7 Nov 2023 13:46:09 +0000 (UTC) (envelope-from filippomore@yahoo.com) Received: from sonic320-20.consmr.mail.ne1.yahoo.com (sonic320-20.consmr.mail.ne1.yahoo.com [66.163.191.82]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPqHj3bQ7z4Tg6 for ; Tue, 7 Nov 2023 13:46:09 +0000 (UTC) (envelope-from filippomore@yahoo.com) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699364767; bh=iQEx8EVvWYzf2+EM670FWbVqJIPT2c7NmY1ULSYLw6U=; h=Date:From:To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To; b=LevbaBXvvrKE72j9kvIAnjvJ5rnsgr7e2rpZksj8nuqTYTPzBWWWgBobYfdYM/30OHy5wwIW1EJUcogCyYP40FGoFk/GQ4lsIH4nA4SPXobKGOJj1iwk/97Wzn1GH06YPqebLeo6RJDSLKEImHJOdPVZ7iKkAf5xKm8DZV7H4sCh6uN6a6QcYle5ApsDJ46tqUowJqALtulFCznJV0wA3KB7WciRjeze5FXXLdysg+v4Zez295WeNVDXHUMnpH4YPk+pDQLQ4u7mwRLMhIvYJ9Yo+Uyl6pKPyKmJ+MvmxLu8x3k35nelpZfwG24BvUBTZbKwKufToR3WBpTasL5wWw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1699364767; bh=eMtwt2LLK/gbO0g6R8CQnp8BuXHn70IZA+yvvTbzB2V=; h=X-Sonic-MF:Date:From:To:Subject:From:Subject; b=GnGs8PW6Bxipf4/2NfcFlj9cqpnwNpEd8NjWGa8rIRTXCDqFv2d9SnCU/xTvHVHyzAMU+5JGsmmhtB4Np6sOcnVYVyhcE8PpnqWMLflBR5G9AFgWGDFxsUBGZUh2BeckunOuXsbpx3JM0xYIv6wtEOgCHVp9slwYn7qazpUgY8jii8pyBY6kikty+hOpyGTrTVvPbQmfrodEuu2tne/RDLsDpBWMdTPPKkrvVACAH+H+U8Nj98e2lti46NHzD7VWeY6bxo5jns/MzNqwRz2Hbhoq+Au1yqC/KH7FVdyD1Q7GmLPraFz36jtSvTfIRPzDMWitaB2PdH6KZdPfrac0+A== X-YMail-OSG: Sp6io0gVM1n5tvZOu3vMXhs4g6DRBJ8o..QlNJn8h4ZWnm6LKPGQfx83zc18fz_ LI9wws8cKI3gvdt.GX4HdgKZ1_AP4wlSm037LM5sCpiasNAzMs1WaZBSff4MTvTy0DPUN6S9tap. qzaZlAD6.ywuL36E7bU3Vr4J1_0IrEbNum6xAc0Rhk0sirzdg9eX4guaBZgxrcxdJT6Gi9zigaEb tV.jCEetp1EZyPWZ91ecVWt37mS8lFtA7I3grbf6IaeCFmadYZKEbCWUNHt2wPxufLEGbzvK6_Xd 2lmH3vfFuuBII4IajszeaXxeY4P9oFiEw1pc1yFxqSxWJF3K2ODSY77jTVs6rXaN22D8JtK6TH1c KjF2H2OhTJFb_M7o18C7Oj76ycLi452adg5F37yWYnoo2a2wDvw8tuslkdkgQqlw4kaT4.3H0G3D 2SD9YP0jRRHJzdb6xYzhTmT4obZJ_hE2kE48xxGLmKmURHC1CvM1kVEnc8ZBWBa9Hw3V1TZOIC_B tq5QCx3oOUzYJzMGaQKSzZab60E0pi2FwmJlDNz882L0V0GH0Lm..zjvg.bLlAyBgDYXC_.e91y0 V.x0GOdh_Dau9SDvwvZPgDDJDM90sTztXdbWb7T1iA7MEEK94hmZwYdtCH5nGlKjzWGkFwFvcVKZ ZP3dQcGYZ7eGKTSsaqy32sEVH._pRKkrMUvveQXP7qnySWIA2xyv3Yb_CLJEHZipHZEXESNojzms clT7SYmQFqXi79JAC.RWeIoc8aUkgf2NhDFRKvexC0IbVuZcVn91le2qxQB5Odn3TsO12wZshu9n dAUnar8XKUh6dU7TTn9L4Qpi6M7m5NrnS90uR93bOpKPIOI4Sf75_nMUDrLcfsU02HJ4aDXOYK8m k2xRjbAgf89y5Zo0YQKWD_.cT8IFOt_hSc3rm7u2CKlH7kW4YXPzyghJB.o8CgGVpakLHyUyW_Qe iJQaP2Z.0VKfOcziAKJS0YjkszMeSk1Vs43OJ_N0jascnw_QxsS8LeuP5bd6LxvgJeSxWkLJ9kMc 8wmhoV51JBJCqfXfBHY2xwJwqvKFTPZYt10O7YInZ42cqGLUyJWGNctiJgW1jp30CqZdBLarvr4q 7RAh44b44unH055_30nJBwbSSB8VC1.66JJr2wMAD2tJlueYztLeoHuYM4NdDdWjpe7ZgU2DgE.G _ygmFXnXpC6eaaCQ9DcQyrDPlSwi0jpplyQfM5SybbgVYVaKa4teiDbDGhc7VU.WYa8c3begFlY8 EbkJrD0Hp_hW_kJz403O0neha7HMF60AtIr1W01jaZjCLYPlWI5qGUfPsuq8JjgF3U0jweIW3S.Y C3D6Pe9xnf4Ju.a3ZFoB5WjtLyqXxC78xLV9z.HLmWWRPPzJFTngqQBnZB2THT60qfx0UkyCmuUs IM9D8LdogOHUILaDIQCbnv2PwH44x2ljTuXIfa2r.l.szXjYPcAy82wxRa.4jzW6z7MbIVtLk9QP .IF9cfvqj5GxZ8WvoHSLgsUtIwlJFeuXcrv8VJiZ.EM0P.F6ojuwPnWhnfwmzCG3HpsFI_Y6NtfJ oYtsobT_6Id3t5JR42UNSsFElmemBO9.lZSudOu0tmz0wIj44O1LD4p94q.bOt1Gc24vDp8HN9H7 wMJoRPw9HmCVMgThvxvQs_jkMvc1_mFbHX_4wWp4LXHU4p4xPtg4mGeWupN81pfS3A5tN1CHOnq3 Dkdx2XcSVxagVsSN94L9LMhOiATXE_k0dBs1_fyLsEbpw5Twz2cI4xQ4Gn6ESSiXW3DEAtlvND9v OHFH.rMvOQFO75_QqMTqykQEG9dNNfEuSSsS3B8iy0FGznP4OASm_zjwQanf3jpN_2SrZzVpndub lEPdgm_HHLsay3XXwEcEW024zf3XzLTBPkRgfqKZ8bEoeqSMcgAqON_yPy5pTiFXIC7_eaQ7yMpm GJQ9SF70hWGj8G1YzQgeSdSNBanP0sp5JYF.hxEgOXcmVw0XiwBKC4PEukTRhVSPB3FuLGpuaY1H RQgraLhIhWkp38We3W3dbdvwDF8ns4FLcxlmgc0o1YdqLDT3xHJvpgRwdZTSdiBXQsmAhqv5sSWk BwAJjnZBhmybRKzZvmBkBxC4qRu.z9QnrBiExBuVbL0zrEDauX62takluIXyLkfo.nRT6z1_buwc nOFDrsVe5dYSfksxnayAfPw2Fzjh13BX3lwENU_YXEbILMPm1ZsVEPaYPRNuBPkGyf.2u2Xk- X-Sonic-MF: X-Sonic-ID: 2062e5ce-9243-44c0-9828-7c06251ab4c7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic320.consmr.mail.ne1.yahoo.com with HTTP; Tue, 7 Nov 2023 13:46:07 +0000 Date: Tue, 7 Nov 2023 13:46:02 +0000 (UTC) From: Filippo Moretti To: Dimitry Andric Cc: Freebsd-stable List Message-ID: <2104142015.375896.1699364762980@mail.yahoo.com> In-Reply-To: References: <17452825.3912917.1699293649529.ref@mail.yahoo.com> <17452825.3912917.1699293649529@mail.yahoo.com> Subject: Re: Problem with iridium List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_375895_362768625.1699364762979" X-Mailer: WebService/1.1.21896 YMailNorrin X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US] X-Rspamd-Queue-Id: 4SPqHj3bQ7z4Tg6 ------=_Part_375895_362768625.1699364762979 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Thank you for your prompt answer:(lldb)=C2=A0 target create "iridium"error= "/usr/local/bin/iridium" doesn't contain any=C2=A0 'host' platform archite= ctures:x86_64,i386Filippo On Monday, November 6, 2023 at 10:53:10 PM GMT+1, Dimitry Andric wrote: =20 =20 On 6 Nov 2023, at 19:00, Filippo Moretti wrote: >=20 > FreeBSD STING 14.0-STABLE FreeBSD 14.0-STABLE #2 stable/14-n265685-52415b= fc0adf-dirty: Mon Nov=C2=A0 6 17:32:23 CET 2023=C2=A0 =C2=A0 root@STING:/us= r/obj/usr/src/amd64.amd64/sys/STING amd64 > When starting iridium I get Trace/BPT trap. > There are no core dumps and nothing in /var/crash > It was compiled from ports without changing defaults. Could be that it is running into an "unreachable" instruction? Can you run = it under lldb or gdb? -Dimitry =20 ------=_Part_375895_362768625.1699364762979 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 7bit
Thank you for your prompt answer:
(lldb)  target create "iridium"
error "/usr/local/bin/iridium" doesn't contain any  'host' platform architectures:x86_64,i386
Filippo

On Monday, November 6, 2023 at 10:53:10 PM GMT+1, Dimitry Andric <dim@freebsd.org> wrote:


On 6 Nov 2023, at 19:00, Filippo Moretti <filippomore@yahoo.com> wrote:

>
> FreeBSD STING 14.0-STABLE FreeBSD 14.0-STABLE #2 stable/14-n265685-52415bfc0adf-dirty: Mon Nov  6 17:32:23 CET 2023    root@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64
> When starting iridium I get Trace/BPT trap.
> There are no core dumps and nothing in /var/crash
> It was compiled from ports without changing defaults.


Could be that it is running into an "unreachable" instruction? Can you run it under lldb or gdb?

-Dimitry


------=_Part_375895_362768625.1699364762979-- From nobody Tue Nov 7 14:54:42 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPrqD4sJqz50LmJ for ; Tue, 7 Nov 2023 14:55:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPrqD40T3z4fqt; Tue, 7 Nov 2023 14:55:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699368904; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Hs2zWesUoJAGnt5bvam5HdFoNGrYV9Y1CsJw0bcRqp0=; b=PJdG+eDvfpmzBmNoMJcvyPvWdCJUVfo25ra1qtmi9UB9Z8xE7kwwBG0vJZCtsIeX6y7L0b K0Vn2ZrOfxOn2FLMRCqURHC12xOsZEBrlJm5aBDbZqyWNt8yo9ijmHDm8gRj4AxFxmDxx6 1/m0E6vG7pbv0h5m3VEyq5YkOuag9zALwIL7H0de/GALtT55onhtlVST6lwRF+P9XHJfY2 EdY/jDkkE7huUZlYhTZTODL2oriG+2TFx34BHCkSXddftyOBMIVmJfeFnw3gDLqtqM0EuH MBOpHT6Jk9s0/jE6R2dBr0rslEotgkulwG62bM7CJwvAjRwnQu6IZqmHVNEYqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1699368904; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Hs2zWesUoJAGnt5bvam5HdFoNGrYV9Y1CsJw0bcRqp0=; b=u56AJaItBOZ/Yq7imba8pVq2gYoFAL47FkTeaW1OOidArlgCChf4ellOSBnWm/nWEFhDFR Wq9iinve1bsUc6Wg5KFOIwcgnfEPBphDG9hnfrIu8yfBeepPMhnGWbwqg2CPFyPMfbTrFA 5N42QRo+NalnRSVZ0ijPFiLCdZIvo0KsGDDJHumbnfr1voUFM6+uoPNMB+Si4BIwx1jMv9 kHlTMp6XuhGEpebNrcjxiactZPBR1jBn/HQ1ZTTAeFqKw+Zm+juDJsswrQ1B68PivFnr2B ysPo9ZokrztQ+QPfXGUTJk2XvtxnvarXCp9AOqACoZtnH9MA0ldURgMx9mH3rg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1699368904; a=rsa-sha256; cv=none; b=N151N+VAAxHTpXPdei87k6uMJ4IOnYsWw/Mekd+pMm2RVT0jMMo7zMnXn+xtIzE/Mu64yk 0kDniKTfL3x+cxFS8QbdMFZ68hB0Wurzc0MepNhgtrH1H/DHep+Lp0/jmorlvjwsENem5P +2bds8qk7YJ8hLHHdaEPhAmk+JiY/WbSMZbsh5kfz4+2IMRhq2w8npcNpsUxsJ1xfr6Kda pyr8CCsGJ/ih/kyBRfeaxhyMRdo6szfwaHUcsRY2mIvT3bRkoDI0VSqSemlMdw7J1RI1NX rNmxNBzF6aFzK9Np8oDgijDBLuIMhaplCpw5iYHJrjFUZq09MGfMy+QQiObDmw== Received: from tensor.andric.com (tensor.andric.com [87.251.56.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "tensor.andric.com", Issuer "R3" (verified OK)) (Authenticated sender: dim) by smtp.freebsd.org (Postfix) with ESMTPSA id 4SPrqD2M6wzl2Z; Tue, 7 Nov 2023 14:55:04 +0000 (UTC) (envelope-from dim@FreeBSD.org) Received: from smtpclient.apple (92-111-45-100.static.v4.ziggozakelijk.nl [92.111.45.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by tensor.andric.com (Postfix) with ESMTPSA id 0FD207C1F6; Tue, 7 Nov 2023 15:55:03 +0100 (CET) Content-Type: multipart/signed; boundary="Apple-Mail=_5BA93A83-1D9D-4809-BF28-7141C29EE6AB"; protocol="application/pgp-signature"; micalg=pgp-sha1 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\)) Subject: Re: Problem with iridium From: Dimitry Andric In-Reply-To: <2104142015.375896.1699364762980@mail.yahoo.com> Date: Tue, 7 Nov 2023 15:54:42 +0100 Cc: Freebsd-stable List Message-Id: <7FEF7D6F-7CD6-4E2F-95A8-F09B6C569621@FreeBSD.org> References: <17452825.3912917.1699293649529.ref@mail.yahoo.com> <17452825.3912917.1699293649529@mail.yahoo.com> <2104142015.375896.1699364762980@mail.yahoo.com> To: Filippo Moretti X-Mailer: Apple Mail (2.3731.700.6) --Apple-Mail=_5BA93A83-1D9D-4809-BF28-7141C29EE6AB Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii My guess would be that /usr/local/bin/iridium is a wrapper script, shell = or python, that invokes the "real" binary. You have to find that binary = and see if you can load it into the debugger. -Dimitry > On 7 Nov 2023, at 14:46, Filippo Moretti = wrote: >=20 > Thank you for your prompt answer: > (lldb) target create "iridium" > error "/usr/local/bin/iridium" doesn't contain any 'host' platform = architectures:x86_64,i386 > Filippo >=20 > On Monday, November 6, 2023 at 10:53:10 PM GMT+1, Dimitry Andric = wrote: >=20 >=20 > On 6 Nov 2023, at 19:00, Filippo Moretti = wrote: >=20 > > > > FreeBSD STING 14.0-STABLE FreeBSD 14.0-STABLE #2 = stable/14-n265685-52415bfc0adf-dirty: Mon Nov 6 17:32:23 CET 2023 = root@STING:/usr/obj/usr/src/amd64.amd64/sys/STING amd64 > > When starting iridium I get Trace/BPT trap. > > There are no core dumps and nothing in /var/crash > > It was compiled from ports without changing defaults. >=20 >=20 > Could be that it is running into an "unreachable" instruction? Can you = run it under lldb or gdb? >=20 > -Dimitry >=20 >=20 --Apple-Mail=_5BA93A83-1D9D-4809-BF28-7141C29EE6AB Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.2 iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCZUpPsgAKCRCwXqMKLiCW o07HAJoDcMRi6a7HX1uFlRFljZlLGdmwUACfUSEwVAzeB8mElyOS5OVZdfk4m0c= =FJcv -----END PGP SIGNATURE----- --Apple-Mail=_5BA93A83-1D9D-4809-BF28-7141C29EE6AB-- From nobody Tue Nov 7 19:52:05 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SPzQ66rZ0z50BrR for ; Tue, 7 Nov 2023 19:52:14 +0000 (UTC) (envelope-from SRS0=PhM4=GU=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int-backup.realworks.nl (smtp-relay-int-backup.realworks.nl [87.255.56.188]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SPzQ60Xb6z4Xmg for ; Tue, 7 Nov 2023 19:52:13 +0000 (UTC) (envelope-from SRS0=PhM4=GU=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=klop.ws header.s=rw2 header.b=U7QPmDNt; spf=pass (mx1.freebsd.org: domain of "SRS0=PhM4=GU=klop.ws=ronald-lists@realworks.nl" designates 87.255.56.188 as permitted sender) smtp.mailfrom="SRS0=PhM4=GU=klop.ws=ronald-lists@realworks.nl"; dmarc=pass (policy=quarantine) header.from=klop.ws Received: from rwvirtual84.colo.realworks.nl (rwvirtual84.colo.realworks.nl [10.0.10.103]) by mailrelayint1.colo2.realworks.nl (Postfix) with ESMTP id 4SPzPx4vVQz1cs for ; Tue, 7 Nov 2023 20:52:05 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1699386725; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type; bh=NFvm02GEpYWT5jnLYXzWy0pTlfVtA/gmwFg+GlfdG7g=; b=U7QPmDNtycPKQxTTFU5dPl/URP0XYW5/Nl2mmMgdvdZLRkzgzqZU/HvqO2nYZgCMZGwsqE RLd//T7V5vyEDCnE62nHIADLjbhe8fYVBH9+jgN2qnOvGL1xbhvLJPJ5y0Ef3MifGBBcXG s/TiRhBJNndtp/COJJsEKQTQvwSS2EVQSXgZOrTyBYOFSVdpcJd6fadomJ68kaCpLBmybP rqkPsueDDrnHfsvKM420qobkmlaV02CXecs4G6z1oINDQSp7acLYwXanle8pv8nga7XJVP BxvlpI/1BgBWzvJF7KDBRz+V1T18CdkdZ7OBCgIIdoza4Wm4MBky3zrVazkB2w== Received: from rwvirtual84.colo.realworks.nl (localhost [127.0.0.1]) by rwvirtual84.colo.realworks.nl (Postfix) with ESMTP id 6DB43C066E for ; Tue, 7 Nov 2023 20:52:05 +0100 (CET) Date: Tue, 7 Nov 2023 20:52:05 +0100 (CET) From: Ronald Klop To: Freebsd-stable List Message-ID: <1964125924.4915.1699386725365@localhost> Subject: failed 13.2 -> 14-RC4 upgrade on RPI3 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_4914_174502750.1699386725272" X-Mailer: Realworks (679.137) X-Originating-Host: from (84-105-120-103.cable.dynamic.v4.ziggo.nl [84.105.120.103]) by rwvirtual84 [10.0.10.103] with HTTP; Tue, 07 Nov 2023 20:52:05 +0100 Importance: Normal X-Priority: 3 (Normal) X-Originating-User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/119.0 X-Spamd-Result: default: False [-3.19 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.990]; MID_RHS_NOT_FQDN(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[klop.ws,quarantine]; FORGED_SENDER(0.30)[ronald-lists@klop.ws,SRS0=PhM4=GU=klop.ws=ronald-lists@realworks.nl]; R_SPF_ALLOW(-0.20)[+ip4:87.255.56.128/26]; R_DKIM_ALLOW(-0.20)[klop.ws:s=rw2]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; ASN(0.00)[asn:38930, ipnet:87.255.32.0/19, country:NL]; HAS_X_PRIO_THREE(0.00)[3]; MLMMJ_DEST(0.00)[stable@freebsd.org]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; TO_DN_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[ronald-lists@klop.ws,SRS0=PhM4=GU=klop.ws=ronald-lists@realworks.nl]; FROM_HAS_DN(0.00)[]; ARC_NA(0.00)[]; DKIM_TRACE(0.00)[klop.ws:+]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; PREVIOUSLY_DELIVERED(0.00)[stable@freebsd.org]; RCVD_COUNT_TWO(0.00)[2] X-Rspamd-Queue-Id: 4SPzQ60Xb6z4Xmg X-Spamd-Bar: --- ------=_Part_4914_174502750.1699386725272 Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Hi, Tried to update my RPI3-on-UFS from 13.2-RELEASE to 14-RC4. Everything looked ok until the end: ========================================== The following changes, which occurred between FreeBSD 13.2-RELEASE and FreeBSD 14.0-RC4 have been merged into /etc/ttys: --- current version +++ new version @@ -1,7 +1,6 @@ # -# $FreeBSD$ # @(#)ttys 5.1 (Berkeley) 4/17/89 # # This file specifies various information about terminals on the system. # It is used by several different programs. Common entries for the # various columns include: @@ -44,10 +43,10 @@ ttyu0 "/usr/libexec/getty 3wire" vt100 onifconsole secure ttyu1 "/usr/libexec/getty 3wire" vt100 onifconsole secure ttyu2 "/usr/libexec/getty 3wire" vt100 onifconsole secure ttyu3 "/usr/libexec/getty 3wire" vt100 onifconsole secure # Dumb console -dcons "/usr/libexec/getty std.9600" vt100 off secure +dcons "/usr/libexec/getty std.115200" vt100 off secure # Xen Virtual console xc0 "/usr/libexec/getty Pc" xterm onifconsole secure # RISC-V HTIF console -rcons "/usr/libexec/getty std.9600" vt100 onifconsole secure +rcons "/usr/libexec/getty std.115200" vt100 onifconsole secure Does this look reasonable (y/n)? y look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory look: INDEX-NEW: No such file or directory comm: INDEX-NEW: No such file or directory rm: modifiedfiles: No such file or directory rm: INDEX-PRESENT: No such file or directory No updates needed to update system to 14.0-RC4-p0. touch: f465c3739385890c221dff1a05e578c6cae0d0430e46996d319db7439f884336-install/kernelfirst: No such file or directory To install the downloaded upgrades, run "/usr/sbin/freebsd-update install". [root@rpi3 ~]# freebsd-update install No updates are available to install. Run '/usr/sbin/freebsd-update fetch' first. [root@rpi3 ~]# ======================================== Does this make sense to anybody? Do I have old cruft lying around in some directory which conflicts with the upgrade? It didn't give an error while downloading all 55000+ patches. Regards, Ronald. ------=_Part_4914_174502750.1699386725272 Content-Type: text/html; charset=us-ascii Content-Transfer-Encoding: 7bit Hi,

Tried to update my RPI3-on-UFS from 13.2-RELEASE to 14-RC4.

Everything looked ok until the end:
==========================================
The following changes, which occurred between FreeBSD 13.2-RELEASE and
FreeBSD 14.0-RC4 have been merged into /etc/ttys:
--- current version
+++ new version
@@ -1,7 +1,6 @@
 #
-# $FreeBSD$
 #      @(#)ttys        5.1 (Berkeley) 4/17/89
 #
 # This file specifies various information about terminals on the system.
 # It is used by several different programs.  Common entries for the
 # various columns include:
@@ -44,10 +43,10 @@
 ttyu0  "/usr/libexec/getty 3wire"      vt100   onifconsole secure
 ttyu1  "/usr/libexec/getty 3wire"      vt100   onifconsole secure
 ttyu2  "/usr/libexec/getty 3wire"      vt100   onifconsole secure
 ttyu3  "/usr/libexec/getty 3wire"      vt100   onifconsole secure
 # Dumb console
-dcons  "/usr/libexec/getty std.9600"   vt100   off secure
+dcons  "/usr/libexec/getty std.115200" vt100   off secure
 # Xen Virtual console
 xc0    "/usr/libexec/getty Pc"         xterm   onifconsole secure
 # RISC-V HTIF console
-rcons  "/usr/libexec/getty std.9600"   vt100   onifconsole secure
+rcons  "/usr/libexec/getty std.115200" vt100   onifconsole secure
Does this look reasonable (y/n)? y
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
look: INDEX-NEW: No such file or directory
comm: INDEX-NEW: No such file or directory
rm: modifiedfiles: No such file or directory
rm: INDEX-PRESENT: No such file or directory

No updates needed to update system to 14.0-RC4-p0.
touch: f465c3739385890c221dff1a05e578c6cae0d0430e46996d319db7439f884336-install/kernelfirst: No such file or directory
To install the downloaded upgrades, run "/usr/sbin/freebsd-update install".
[root@rpi3 ~]# freebsd-update install
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
[root@rpi3 ~]#

========================================

Does this make sense to anybody?
Do I have old cruft lying around in some directory which conflicts with the upgrade?
It didn't give an error while downloading all 55000+ patches.

Regards,
Ronald.
  ------=_Part_4914_174502750.1699386725272-- From nobody Tue Nov 7 21:34:23 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SQ1hG42ncz50WWF for ; Tue, 7 Nov 2023 21:34:38 +0000 (UTC) (envelope-from jason@tubnor.net) Received: from mail.tubnor.net (mail.tubnor.net [103.236.162.16]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SQ1hF35HDz4l2f for ; Tue, 7 Nov 2023 21:34:37 +0000 (UTC) (envelope-from jason@tubnor.net) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=tubnor.net; s=20220915; t=1699392864; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=74IB0V/vPeSL7eKb70wdf5lH0VMd1c73kqXHHrVdV7U=; b=M1YNPoGTxQd3WKboB7+PuLnb4VB++okk5BILmiClXdeqbjGNxWHbMc6qbH/r7BOjFuAWun S9tUhC7E964+/dTELtffGL1V6ml/S6EirOKmfrg2r/Y4wRgvpqQrXT/iGWPU0ipS2387ht ehK2AtJaBDJZsi/HggVhzr7qu+qx698= Received: from drive.tubnor.net ( [2403:5812:73e6:2::22:1]) by mel01.ar18.net (OpenSMTPD) with ESMTPSA id 81333820 (TLSv1.3:TLS_AES_256_GCM_SHA384:256:NO); Wed, 8 Nov 2023 08:34:23 +1100 (AEDT) From: Jason Tubnor To: Freebsd-stable List , Ronald Klop Cc: Subject: Re: failed 13.2 -> 14-RC4 upgrade on RPI3 References: <1964125924.4915.1699386725365@localhost> In-Reply-To: <1964125924.4915.1699386725365@localhost> Message-ID: <20231107213423.Horde.vBiRSZqTTo69qkWRAQkCTCG@drive.tubnor.net> User-Agent: Horde Application Framework 5 Date: Tue, 07 Nov 2023 21:34:23 +0000 Content-Type: multipart/alternative; boundary="=_bRRKTAQdqZvnpriBdNyWdmj" List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:133159, ipnet:103.236.162.0/23, country:AU] X-Rspamd-Queue-Id: 4SQ1hF35HDz4l2f This message is in MIME format. --=_bRRKTAQdqZvnpriBdNyWdmj Content-Type: text/plain; charset=utf-8 Content-Description: Plaintext Version of Message Content-Transfer-Encoding: 8bit     "Ronald Klop" ronald-lists@klop.ws – 8 November 2023 6:52 AM  > Hi,Tried to update my RPI3-on-UFS from 13.2-RELEASE to 14-RC4.comm: > INDEX-NEW: No such file or directoryrm: modifiedfiles: No such file or > directoryrm: INDEX-PRESENT: No such file or directoryNo updates needed > to update system to 14.0-RC4-p0.touch: > f465c3739385890c221dff1a05e578c6cae0d0430e46996d319db7439f884336-install/kernelfirst: No such file or directoryTo install the downloaded upgrades, run "/usr/sbin/freebsd-update install".[root@rpi3 ~]# freebsd-update installNo updates are available to install.Run '/usr/sbin/freebsd-update fetch' first.[root@rpi3 ~]#========================================Does this make sense to anybody?Do I have old cruft lying around in some directory which conflicts with the upgrade?It didn't give an error while downloading all 55000+ > patches.Regards,Ronald.    Did you:   freebsd-update fetch install && shutdown -r now    Before you ran the 14 upgrade process? There were some recent errata that corrected issues in preparation of 14 upgrades.   Cheers,   Jason.   --=_bRRKTAQdqZvnpriBdNyWdmj Content-Type: text/html; charset=utf-8 Content-Description: HTML Version of Message Content-Transfer-Encoding: quoted-printable

=C2=A0

=C2=A0

"Ronald Klop" ronald-lists@klo= p.ws =E2=80=93 8 November 2023 6:52 AM
=C2=A0

Hi,

Tried to update my RPI3-on-UFS from 13.2-RELEASE to = 14-RC4.


comm: INDEX-NEW: No such file or directory
rm: modifi= edfiles: No such file or directory
rm: INDEX-PRESENT: No such file or di= rectory

No updates needed to update system to 14.0-RC4-p0.
touch:= f465c3739385890c221dff1a05e578c6cae0d0430e46996d319db7439f884336-install/k= ernelfirst: No such file or directory
To install the downloaded upgrades= , run "/usr/sbin/freebsd-update install".
[root@rpi3 ~]# freebsd-update = install
No updates are available to install.
Run '/usr/sbin/freebsd-u= pdate fetch' first.
[root@rpi3 ~]#

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D

Does this make sense to anybody?
Do I have old cr= uft lying around in some directory which conflicts with the upgrade?
It = didn't give an error while downloading all 55000+ patches.

Regards,<= br>Ronald.
=C2=A0

=C2=A0

Did you:

=C2=A0

freebsd-update fetch install && shutdown -r now=C2=A0

=C2=A0

Before you ran the= 14 upgrade process? There were some recent errata that corrected issues in= preparation of 14 upgrades.

=C2=A0

Cheers,

=C2=A0

Jason.

=C2=A0

--=_bRRKTAQdqZvnpriBdNyWdmj-- From nobody Wed Nov 8 07:25:36 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SQGpK2Rbqz50GV4 for ; Wed, 8 Nov 2023 07:25:45 +0000 (UTC) (envelope-from SRS0=284H=GV=klop.ws=ronald-lists@realworks.nl) Received: from smtp-relay-int.realworks.nl (smtp-relay-int.realworks.nl [194.109.157.24]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SQGpK0Ffhz4JLk for ; Wed, 8 Nov 2023 07:25:44 +0000 (UTC) (envelope-from SRS0=284H=GV=klop.ws=ronald-lists@realworks.nl) Authentication-Results: mx1.freebsd.org; none Date: Wed, 8 Nov 2023 08:25:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=klop.ws; s=rw2; t=1699428337; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZKrVwhkIRykhkO4eFzJOjNDtZY6gCAhzYXAzy1Zv/k4=; b=mwCC68RaJY3x2LM5yccgOXzH1vz0XCgPaLsLCKgiGeFZipAj9qdVZj/zhHQmUISZRM9bPB U4plPteGAtVFQS/V4HucntF8eFMAsg0SdL3R0rrr9XwU/n9ltSMy8mU1heninMWX4pZ2Eh Z2ghzjKsAxeUtMK6pZFwop6o5Bz4/Fsk46/diQsps1lLbKB8uU2SHyjEyQIVogtLWYoku4 SPvMGTS+phIlKuIwrqsk0c8rNEjO4i5qBPX/W0Fo0zhlLLr63EjGBP1t06YSFk4VJg4Bz8 5SovaMtwSKsb41ySzJNq5iEIk8Ea/EP7yOtwa6UhOhhbYc6tchi3K07tPRYrsQ== From: Ronald Klop To: Jason Tubnor Cc: Freebsd-stable List Message-ID: <1625115570.9056.1699428336994@localhost> In-Reply-To: <20231107213423.Horde.vBiRSZqTTo69qkWRAQkCTCG@drive.tubnor.net> References: <1964125924.4915.1699386725365@localhost> <20231107213423.Horde.vBiRSZqTTo69qkWRAQkCTCG@drive.tubnor.net> Subject: Re: failed 13.2 -> 14-RC4 upgrade on RPI3 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_9055_987433704.1699428336925" X-Mailer: Realworks (679.137) Importance: Normal X-Priority: 3 (Normal) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:3265, ipnet:194.109.0.0/16, country:NL] X-Rspamd-Queue-Id: 4SQGpK0Ffhz4JLk ------=_Part_9055_987433704.1699428336925 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable Van: Jason Tubnor Datum: dinsdag, 7 november 2023 22:34 Aan: Freebsd-stable List , Ronald Klop Onderwerp: Re: failed 13.2 -> 14-RC4 upgrade on RPI3 >=20 > =20 >=20 > =20 >=20 > "Ronald Klop" ronald-lists@klop.ws =E2=80=93 8 November 2023 6:52 AM > =20 >>=20 >> Hi, >>=20 >> Tried to update my RPI3-on-UFS from 13.2-RELEASE to 14-RC4. >>=20 >>=20 >> comm: INDEX-NEW: No such file or directory >> rm: modifiedfiles: No such file or directory >> rm: INDEX-PRESENT: No such file or directory >>=20 >> No updates needed to update system to 14.0-RC4-p0. >> touch: f465c3739385890c221dff1a05e578c6cae0d0430e46996d319db7439f884336-= install/kernelfirst: No such file or directory >> To install the downloaded upgrades, run "/usr/sbin/freebsd-update instal= l". >> [root@rpi3 ~]# freebsd-update install >> No updates are available to install. >> Run '/usr/sbin/freebsd-update fetch' first. >> [root@rpi3 ~]# >>=20 >> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>=20 >> Does this make sense to anybody? >> Do I have old cruft lying around in some directory which conflicts with = the upgrade? >> It didn't give an error while downloading all 55000+ patches. >>=20 >> Regards, >> Ronald. >> =20 >=20 > =20 >=20 > Did you: >=20 > =20 >=20 > freebsd-update fetch install && shutdown -r now=20 >=20 > =20 >=20 > Before you ran the 14 upgrade process? There were some recent errata that= corrected issues in preparation of 14 upgrades. >=20 > =20 >=20 > Cheers, >=20 > =20 >=20 > Jason. >=20 > =20 >=20 Hi, I think I know what happened. I have this line in cron which I forgot to di= sable: "@daily freebsd-update cron && freebsd-update updatesready > /dev/null && f= reebsd-update install" On my RPI3 a full upgrade takes a while so I let it run overnight. Last nig= ht I tried again and the cron job now gave me error mails. That is how I fo= und out. Now running an upgrade with this line commented out. Regards, Ronald. =20 ------=_Part_9055_987433704.1699428336925 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable

Van: Jason Tubnor <jason@tubnor.net>
Datum: dinsdag, 7 november 2023 22:34
Aan: Freebsd-stable List <stable@freebsd.org>, Ronal= d Klop <ronald-lists@klop.ws>
Onderwerp: Re: failed 13.2 -> 14-RC4 upgrade on RPI3

 

 

"Ronald Klop" ronald-lists@klop.ws =E2=80=93 8 Novem= ber 2023 6:52 AM
 

Hi,

Tried to update my RPI3-on-UFS from 13.2-RELEASE to 14-RC4.


comm: INDEX-NEW: No such file or directory
rm: modifiedfiles: No such file or directory
rm: INDEX-PRESENT: No such file or directory

No updates needed to update system to 14.0-RC4-p0.
touch: f465c3739385890c221dff1a05e578c6cae0d0430e46996d319db7439f884336-ins= tall/kernelfirst: No such file or directory
To install the downloaded upgrades, run "/usr/sbin/freebsd-update install".=
[root@rpi3 ~]# freebsd-update install
No updates are available to install.
Run '/usr/sbin/freebsd-update fetch' first.
[root@rpi3 ~]#

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Does this make sense to anybody?
Do I have old cruft lying around in some directory which conflicts with the= upgrade?
It didn't give an error while downloading all 55000+ patches.

Regards,
Ronald.
 

 

Did you:

 

freebsd-update fetch install && shutdown -r = now 

 

Before you ran the 14 upgrade process? There were so= me recent errata that corrected issues in preparation of 14 upgrades.

 

Cheers,

 

Jason.

 



Hi,

I think I know what happened. I have this line in cron which I forgot to di= sable:
"@daily freebsd-update cron && freebsd-update updatesready > /de= v/null && freebsd-update install"

On my RPI3 a full upgrade takes a while so I let it run overnight. Last nig= ht I tried again and the cron job now gave me error mails. That is how I fo= und out.

Now running an upgrade with this line commented out.

Regards,
Ronald.
  ------=_Part_9055_987433704.1699428336925-- From nobody Thu Nov 9 11:05:01 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SQzd46gD2z50KlL for ; Thu, 9 Nov 2023 11:05:12 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SQzd25Gkkz4M5y for ; Thu, 9 Nov 2023 11:05:10 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=zxy.spb.ru header.s=dkim header.b=LBQ9OTEY; spf=pass (mx1.freebsd.org: domain of slw@zxy.spb.ru designates 195.70.199.98 as permitted sender) smtp.mailfrom=slw@zxy.spb.ru; dmarc=pass (policy=none) header.from=zxy.spb.ru DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=zxy.spb.ru; s=dkim; h=Content-Type:MIME-Version:Message-ID:Subject:To:From:Date; bh=YtQdEmyV/TY4K0eIpLQFxUzOF//mJyDgVz9o88NXF/4=; b=LBQ9OTEYITbA6EhPjYYCRFhRpb 3lFws5mF01UvRmwwc5APpD6H7BSyF+9Mw6M7tOeVDFv/cFezGUisYYQzkzESP5O32qe/Qks3dk7OQ PpIvZ+iI3P6YBnBmfyeev4jfAYrmhfv2shkWAQg0YBUJf13Ew6Vw2tr6nFhrbzneSwFDS2/CuAGPs 5+4IRakReMcleycRTSKPiyMzlYppp3kYEM69tRG1HMMo38kqoWxWGn8v7zLrpPvYb6qDaILcVWp7u 9XQdxKbkPdGJLtr4Zramvhy1RkUi4yG7krX0Nv+uOuLgBITBj95TZyuxpC5jsCs3bzzQ+VeNFN8t3 T7T9GK8Q==; Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1r12qL-000DJR-Im for stable@freebsd.org; Thu, 09 Nov 2023 14:05:01 +0300 Date: Thu, 9 Nov 2023 14:05:01 +0300 From: Slawa Olhovchenkov To: stable@freebsd.org Subject: Dell R750 NVMe switch Message-ID: <20231109110501.GA50969@zxy.spb.ru> List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Spamd-Result: default: False [-3.89 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.99)[-0.994]; DMARC_POLICY_ALLOW(-0.50)[zxy.spb.ru,none]; R_DKIM_ALLOW(-0.20)[zxy.spb.ru:s=dkim]; R_SPF_ALLOW(-0.20)[+ip4:195.70.199.98/32]; ONCE_RECEIVED(0.10)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:5495, ipnet:195.70.192.0/19, country:RU]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_ONE(0.00)[1]; FROM_EQ_ENVFROM(0.00)[]; MLMMJ_DEST(0.00)[stable@freebsd.org]; DKIM_TRACE(0.00)[zxy.spb.ru:+]; RCPT_COUNT_ONE(0.00)[1]; FROM_HAS_DN(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; TO_DN_NONE(0.00)[]; ARC_NA(0.00)[] X-Rspamd-Queue-Id: 4SQzd25Gkkz4M5y X-Spamd-Bar: --- Is FreeBSD (TrueNAS) compatible and support NVMe switch in Dell Poweredge R750? From nobody Fri Nov 10 07:04:20 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRVDw0Wg4z50ZxY for ; Fri, 10 Nov 2023 07:04:32 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [IPv6:2001:470:1:117::25]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "anubis.delphij.net", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRVDv0sXfz4XyM; Fri, 10 Nov 2023 07:04:31 +0000 (UTC) (envelope-from delphij@delphij.net) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=delphij.net header.s=y07n header.b=SG4dW297; dkim=pass header.d=delphij.net header.s=w44o header.b=pd0ClVEv; spf=pass (mx1.freebsd.org: domain of delphij@delphij.net designates 2001:470:1:117::25 as permitted sender) smtp.mailfrom=delphij@delphij.net; dmarc=pass (policy=reject) header.from=delphij.net DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/simple; d=delphij.net; i=@delphij.net; q=dns/txt; s=y07n; t=1699599862; h=message-id : date : mime-version : reply-to : subject : to : references : from : in-reply-to : content-type : from; bh=YmxDNEZZv8KyW/JRoS2ayGNAGzVHL//lUtmcXPae58c=; b=SG4dW297mB2Jv4AIZxCW6zv+Bfmn2DZWe6fRukm7j3jyCmuy1hyOA9tGcmgVvRSc3xnOG BIubpiWj5JYdr6bAw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=delphij.net; i=@delphij.net; q=dns/txt; s=w44o; t=1699599862; h=message-id : date : mime-version : reply-to : subject : to : references : from : in-reply-to : content-type : from; bh=YmxDNEZZv8KyW/JRoS2ayGNAGzVHL//lUtmcXPae58c=; b=pd0ClVEv9WUtw2tOLnEaQa8x+vYz53Zl2bg01ETWED8mLxi5gqxUTAOC5uvhmxyMRFpta ajRyuHEB97urDi8zq1Sf1ql7RNXwLlEMJyXa78tcGQmqp2Fn7dvz5rNnigGY0LI0BdXxEwY iOZ5s6gsMH5FelSiyOBieyciZhQ63QIuC6fKjCy2VW1OOJyPFoGj4bvngWwzJi3BpsKzpVD w5QXptAh4SPJoU6qf4DOJq8hps5q4FPOanwArOwN40XT8ifJ1A/Dw4C3KlJ+9MYUlqxPwzP b+rIiHxQmDG2DbL+BQSwH/nkZ0yIDkwXuE/ZcWMfM7+RIhYRTAfW95e/c0tA== Received: from xins-laptop (unknown [IPv6:2601:646:9a00:3b0a:b42d:de04:6faf:3927]) by anubis.delphij.net (Postfix) with ESMTPSA id AC1B038331; Thu, 9 Nov 2023 23:04:22 -0800 (PST) Message-ID: <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> Date: Thu, 9 Nov 2023 23:04:20 -0800 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Reply-To: d@delphij.net Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? Content-Language: en-US To: Martin Matuska , FreeBSD-STABLE Mailing List , pjd@FreeBSD.org References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> From: Xin Li Autocrypt: addr=delphij@delphij.net; keydata= xjMEZPbDoRYJKwYBBAHaRw8BAQdAsUNmxEWz6QiGdFbBrVVEpjNpgQV9FXjDWsLsY0UwRPvN HFhpbiBMSSA8ZGVscGhpakBkZWxwaGlqLm5ldD7ClgQTFgoAPhYhBLskk2pXNatsapeNzxED 4uuXWeTFBQJk9sRMAhsDBQkKBDXmBQsJCAcDBRUKCQgLBRYCAwEAAh4FAheAAAoJEBED4uuX WeTF6yIA/2Ls3Rb/qC8mQZ6D2S0UO5vblPghJfboFJLNJFw3i4GYAQCsTmQg3ahgbNEJu/vU xgtro2kTxa6kKnZ35IbqPqPcCc44BGT2w6ESCisGAQQBl1UBBQEBB0Cxji+sQgVPajLNA/Lw yHx0ogSalPQszdkfVgeg3iR3FAMBCAfCeAQYFgoAIBYhBLskk2pXNatsapeNzxED4uuXWeTF BQJk9sOhAhsMAAoJEBED4uuXWeTF3BQBAIx/gPCTFN2DPBrKLkE3oC/+j9EkmNLMUCGidlP/ Zb6HAP4nL1kStTsOldIGhi/3m1LvU7r3Kel3MnlIK8/9BlLPAg== In-Reply-To: <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="------------tZbYpB30DpTzqxB0ZbCDEv0z" X-Spamd-Result: default: False [-4.87 / 15.00]; SIGNED_PGP(-2.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.981]; DMARC_POLICY_ALLOW(-0.50)[delphij.net,reject]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,multipart/mixed,text/plain]; R_DKIM_ALLOW(-0.20)[delphij.net:s=y07n,delphij.net:s=w44o]; ONCE_RECEIVED(0.10)[]; MIME_BASE64_TEXT(0.10)[]; XM_UA_NO_VERSION(0.01)[]; FREEFALL_USER(0.00)[delphij]; HAS_REPLYTO(0.00)[d@delphij.net]; RCPT_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; REPLYTO_DOM_EQ_FROM_DOM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; ARC_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:+,3:~]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; HAS_ATTACHMENT(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_TRACE(0.00)[delphij.net:+]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; MLMMJ_DEST(0.00)[freebsd-stable@freebsd.org]; RCVD_TLS_ALL(0.00)[] X-Rspamd-Queue-Id: 4SRVDv0sXfz4XyM X-Spamd-Bar: ---- This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --------------tZbYpB30DpTzqxB0ZbCDEv0z Content-Type: multipart/mixed; boundary="------------jvk50lwEhNzmRy3Of6INPY0V"; protected-headers="v1" From: Xin Li Reply-To: d@delphij.net To: Martin Matuska , FreeBSD-STABLE Mailing List , pjd@FreeBSD.org Message-ID: <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> In-Reply-To: <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> --------------jvk50lwEhNzmRy3Of6INPY0V Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: base64 T24gMjAyMy0xMS0wNSAxNjozNCwgTWFydGluIE1hdHVza2Egd3JvdGU6DQo+IE9wZW5aRlMg Mi4yLjAgaW4gRnJlZUJTRCAxNCBmdWxseSBzdXBwb3J0cyBibG9jayBjbG9uaW5nLiBZb3Ug Y2FuIHdvcmsgDQo+IHdpdGggcG9vbHMgdGhhdCBoYXZlIGZlYXR1cmVAYmxvY2tfY2xvbmlu ZyBlbmFibGVkLg0KPiBUaGUgc3lzY3RsIHZhcmlhYmxlIHZmcy56ZnMuYmNsb25lX2VuYWJs ZWQgYWZmZWN0cyB0aGUgYmVoYXZpb3Igb2YgDQo+IHpmc19jbG9uZV9yYW5nZSgpIHdoaWNo IGlzIGNhbGxlZCBieSBjb3B5X2ZpbGVfcmFuZ2UoKS4gV2hlbiBpdCBpcyBzZXQgDQo+IHRv IDAsIHpmc19jbG9uZV9yYW5nZSgpIGRvZXMgbm90IGRvIGJsb2NrIGNsb25pbmcuDQo+IElm IGl0IGlzIHNldCB0byBhbnl0aGluZyBlbHNlIHRoYW4gMCwgemZzX2Nsb25lX3JhbmdlKCkg ZG9lcyBibG9jayANCj4gY2xvbmluZyAoaWYgYWxsIGNvbmRpdGlvbnMgYXJlIG1ldCAtIHNh bWUgWkZTIHBvb2wsIGNvcnJlY3QgZGF0YSANCj4gYWxpZ25tZW50LCBldGMuKS4NCj4gDQo+ IEluIEZyZWVCU0QtbWFpbiwgdGhpcyB0dW5hYmxlIGlzIGVuYWJsZWQgYW5kIEkgcGxhbiB0 byBlbmFibGUgaXQgaW4gDQo+IHN0YWJsZS8xNCBzb21ld2hlcmUgYXJvdW5kIERlY2VtYmVy IDExLCAyMDIzLg0KPiANCj4gQXMgb2YgdG9kYXkgSSBwZXJzb25hbGx5IHVzZSBibG9jayBj bG9uaW5nIG9uIGFsbCBteSBzeXN0ZW1zLg0KDQpJJ2QgbGlrZSB0byBzaGFyZSBhIGRpZmZl cmVudCBkYXRhIHBvaW50LiAgSXQgc3RpbGwgcGFuaWNzIG9uIG15IHN0b3JhZ2UgDQoocnVu bmluZyAtQ1VSUkVOVCBhYm91dCBhIHdlZWsgYWdvKSB3aGVuIGVuYWJsZWQgYW5kIGNhbiBi ZSB0cmlnZ2VyZWQgYnkgDQoibWFrZSBidWlsZHdvcmxkIGJ1aWxka2VybmVsIi4gIEkgd2Fz bid0IGFibGUgdG8gY2FwdHVyZSBlYXJsaWVyIA0KY29yZWR1bXAgdW50aWwgdGhlIG1vc3Qg cmVjZW50IG9uZSwgd2hpY2ggcGFuaWNrZWQgd2l0aDoNCg0KDQpjcHVpZCA9IDINCnRpbWUg PSAxNjk5NTkzNDU2DQpLREI6IHN0YWNrIGJhY2t0cmFjZToNCmRiX3RyYWNlX3NlbGZfd3Jh cHBlcigpIGF0IGRiX3RyYWNlX3NlbGZfd3JhcHBlcisweDJiL2ZyYW1lIA0KMHhmZmZmZmUw MjJmMmJkN2UwDQp2cGFuaWMoKSBhdCB2cGFuaWMrMHgxMzIvZnJhbWUgMHhmZmZmZmUwMjJm MmJkOTEwDQpzcGxfcGFuaWMoKSBhdCBzcGxfcGFuaWMrMHgzYS9mcmFtZSAweGZmZmZmZTAy MmYyYmQ5NzANCmRtdV9icnRfY2xvbmUoKSBhdCBkbXVfYnJ0X2Nsb25lKzB4NTU1L2ZyYW1l IDB4ZmZmZmZlMDIyZjJiZDllMA0KemZzX2Nsb25lX3JhbmdlKCkgYXQgemZzX2Nsb25lX3Jh bmdlKzB4YTRjL2ZyYW1lIDB4ZmZmZmZlMDIyZjJiZGJiMA0KemZzX2ZyZWVic2RfY29weV9m aWxlX3JhbmdlKCkgYXQgemZzX2ZyZWVic2RfY29weV9maWxlX3JhbmdlKzB4MThhL2ZyYW1l IA0KMHhmZmZmZmUwMjJmMmJkYzMwDQp2bl9jb3B5X2ZpbGVfcmFuZ2UoKSBhdCB2bl9jb3B5 X2ZpbGVfcmFuZ2UrMHgxNjMvZnJhbWUgMHhmZmZmZmUwMjJmMmJkY2UwDQprZXJuX2NvcHlf ZmlsZV9yYW5nZSgpIGF0IGtlcm5fY29weV9maWxlX3JhbmdlKzB4MzgwL2ZyYW1lIA0KMHhm ZmZmZmUwMjJmMmJkZGIwDQpzeXNfY29weV9maWxlX3JhbmdlKCkgYXQgc3lzX2NvcHlfZmls ZV9yYW5nZSsweDc4L2ZyYW1lIDB4ZmZmZmZlMDIyZjJiZGUwMA0KYW1kNjRfc3lzY2FsbCgp IGF0IGFtZDY0X3N5c2NhbGwrMHgxNTMvZnJhbWUgMHhmZmZmZmUwMjJmMmJkZjMwDQpmYXN0 X3N5c2NhbGxfY29tbW9uKCkgYXQgZmFzdF9zeXNjYWxsX2NvbW1vbisweGY4L2ZyYW1lIDB4 ZmZmZmZlMDIyZjJiZGYzMA0KLS0tIHN5c2NhbGwgKDU2OSwgRnJlZUJTRCBFTEY2NCwgY29w eV9maWxlX3JhbmdlKSwgcmlwID0gMHg3ZmJiMmRhNGFkYSwgDQpyc3AgPSAweDdmYmIwMmM1 ZDQ4LCByYnAgPSAweDdmYmIwMmM2MWUwIC0tLQ0KVXB0aW1lOiAyaDMybTI3cw0KRHVtcGlu ZyA3ODAwIG91dCBvZiAzMjY5NiANCk1COi4uMSUuLjExJS4uMjElLi4zMSUuLjQxJS4uNTEl Li42MSUuLjcxJS4uODElLi45MSUNCg0KIzAgIF9fY3VydGhyZWFkICgpIGF0IC91c3Ivc3Jj L3N5cy9hbWQ2NC9pbmNsdWRlL3BjcHVfYXV4Lmg6NTcNCiMxICBkb2FkdW1wICh0ZXh0ZHVt cD10ZXh0ZHVtcEBlbnRyeT0xKSBhdCANCi91c3Ivc3JjL3N5cy9rZXJuL2tlcm5fc2h1dGRv d24uYzo0MDUNCiMyICAweGZmZmZmZmZmODA2OTQ0ODAgaW4ga2Vybl9yZWJvb3QgKGhvd3Rv PTI2MCkgYXQgDQovdXNyL3NyYy9zeXMva2Vybi9rZXJuX3NodXRkb3duLmM6NTI2DQojMyAg MHhmZmZmZmZmZjgwNjk0OTdmIGluIHZwYW5pYyAoZm10PTB4ZmZmZmZmZmY4MjYwMzQxNSAi VkVSSUZZMyhuYnBzIA0KPT0gbnVtYnVmcykgZmFpbGVkICglbGx1ID09ICVsbHUpXG4iLCBh cD1hcEBlbnRyeT0weGZmZmZmZTAyMmYyYmQ5NTApIGF0IA0KL3Vzci9zcmMvc3lzL2tlcm4v a2Vybl9zaHV0ZG93bi5jOjk3MA0KIzQgIDB4ZmZmZmZmZmY4MjMyOTk5YSBpbiBzcGxfcGFu aWMgKGZpbGU9PG9wdGltaXplZCBvdXQ+LCANCmZ1bmM9PG9wdGltaXplZCBvdXQ+LCBsaW5l PTx1bmF2YWlsYWJsZT4sIGZtdD08dW5hdmFpbGFibGU+KSBhdCANCi91c3Ivc3JjL3N5cy9j b250cmliL29wZW56ZnMvbW9kdWxlL29zL2ZyZWVic2Qvc3BsL3NwbF9taXNjLmM6MTAzDQoj NSAgMHhmZmZmZmZmZjgyM2E2NjA1IGluIGRtdV9icnRfY2xvbmUgKG9zPW9zQGVudHJ5PTB4 ZmZmZmY4MDBjNWNlNDAwMCwgDQpvYmplY3Q9PG9wdGltaXplZCBvdXQ+LCBvZmZzZXQ9b2Zm c2V0QGVudHJ5PTAsIA0KbGVuZ3RoPWxlbmd0aEBlbnRyeT0yMDc0NzcsIHR4PXR4QGVudHJ5 PTB4ZmZmZmY4MDcxYTEwOGQwMCwgDQpicHM9YnBzQGVudHJ5PTB4ZmZmZmZlMDFlMjE4YzAw MCwgbmJwcz0yLCByZXBsYXk9MCkNCiAgICAgYXQgL3Vzci9zcmMvc3lzL2NvbnRyaWIvb3Bl bnpmcy9tb2R1bGUvemZzL2RtdS5jOjIzMDMNCiM2ICAweGZmZmZmZmZmODI1MGY2N2MgaW4g emZzX2Nsb25lX3JhbmdlIChpbnpwPTB4ZmZmZmY4MDQ0MTZhYzAwMCwgDQppbm9mZnA9MHhm ZmZmZjgwMGI4MWNiMDQ4LCBvdXR6cD0weGZmZmZmODA2ZjU4ZjAzYTAsIA0Kb3V0b2ZmcD0w eGZmZmZmODAwYjgwNjMwNDgsIGxlbnA9bGVucEBlbnRyeT0weGZmZmZmZTAyMmYyYmRiZjAs IA0KY3I9MHhmZmZmZjgwMDBhNmZlNjAwKQ0KICAgICBhdCAvdXNyL3NyYy9zeXMvY29udHJp Yi9vcGVuemZzL21vZHVsZS96ZnMvemZzX3Zub3BzLmM6MTMyNg0KIzcgIDB4ZmZmZmZmZmY4 MjM0YjNiYSBpbiB6ZnNfZnJlZWJzZF9jb3B5X2ZpbGVfcmFuZ2UgDQooYXA9MHhmZmZmZmUw MjJmMmJkYzQ4KSBhdCANCi91c3Ivc3JjL3N5cy9jb250cmliL29wZW56ZnMvbW9kdWxlL29z L2ZyZWVic2QvemZzL3pmc192bm9wc19vcy5jOjYyOTQNCiM4ICAweGZmZmZmZmZmODA3OWY0 NDMgaW4gVk9QX0NPUFlfRklMRV9SQU5HRSAoaW52cD0weGZmZmZmODA0NDE2Y2IxYzAsIA0K aW5vZmZwPTB4ZmZmZmY4MDBiODFjYjA0OCwgb3V0dnA9MHhmZmZmZjgwNmY1MWQzMzgwLCAN Cm91dG9mZnA9MHhmZmZmZjgwMGI4MDYzMDQ4LCBsZW5wPTB4ZmZmZmZlMDIyZjJiZGQ0OCwg DQppbmNyZWQ9MHhmZmZmZjgwMDBhNmZlNjAwLCBmbGFncz08b3B0aW1pemVkIG91dD4sDQog ICAgIG91dGNyZWQ9PG9wdGltaXplZCBvdXQ+LCBmc2l6ZXRkPTxvcHRpbWl6ZWQgb3V0Pikg YXQgLi92bm9kZV9pZi5oOjIzODUNCiM5ICB2bl9jb3B5X2ZpbGVfcmFuZ2UgKGludnA9aW52 cEBlbnRyeT0weGZmZmZmODA0NDE2Y2IxYzAsIA0KaW5vZmZwPWlub2ZmcEBlbnRyeT0weGZm ZmZmODAwYjgxY2IwNDgsIA0Kb3V0dnA9b3V0dnBAZW50cnk9MHhmZmZmZjgwNmY1MWQzMzgw LCANCm91dG9mZnA9b3V0b2ZmcEBlbnRyeT0weGZmZmZmODAwYjgwNjMwNDgsIA0KbGVucD1s ZW5wQGVudHJ5PTB4ZmZmZmZlMDIyZjJiZGQ0OCwgZmxhZ3M9ZmxhZ3NAZW50cnk9MCwNCiAg ICAgaW5jcmVkPTB4ZmZmZmY4MDAwYTZmZTYwMCwgb3V0Y3JlZD0weGZmZmZmODAwMGE2ZmU2 MDAsIA0KZnNpemVfdGQ9MHhmZmZmZmUwMjI5MjViM2EwKSBhdCAvdXNyL3NyYy9zeXMva2Vy bi92ZnNfdm5vcHMuYzozMDg3DQojMTAgMHhmZmZmZmZmZjgwNzlhMDcwIGluIGtlcm5fY29w eV9maWxlX3JhbmdlIA0KKHRkPXRkQGVudHJ5PTB4ZmZmZmZlMDIyOTI1YjNhMCwgaW5mZD08 b3B0aW1pemVkIG91dD4sIA0KaW5vZmZwPTB4ZmZmZmY4MDBiODFjYjA0OCwgaW5vZmZwQGVu dHJ5PTB4MCwgb3V0ZmQ9PG9wdGltaXplZCBvdXQ+LCANCm91dG9mZnA9MHhmZmZmZjgwMGI4 MDYzMDQ4LCBvdXRvZmZwQGVudHJ5PTB4MCwgbGVuPTkyMjMzNzIwMzY4NTQ3NzU4MDcsDQog ICAgIGZsYWdzPTApIGF0IC91c3Ivc3JjL3N5cy9rZXJuL3Zmc19zeXNjYWxscy5jOjQ5NzMN CiMxMSAweGZmZmZmZmZmODA3OWExNzggaW4gc3lzX2NvcHlfZmlsZV9yYW5nZSAodGQ9MHhm ZmZmZmUwMjI5MjViM2EwLCANCnVhcD0weGZmZmZmZTAyMjkyNWI3YTApIGF0IC91c3Ivc3Jj L3N5cy9rZXJuL3Zmc19zeXNjYWxscy5jOjUwMTENCiMxMiAweGZmZmZmZmZmODBhOTdhYTMg aW4gc3lzY2FsbGVudGVyICh0ZD0weGZmZmZmZTAyMjkyNWIzYTApIGF0IA0KL3Vzci9zcmMv c3lzL2FtZDY0L2FtZDY0Ly4uLy4uL2tlcm4vc3Vicl9zeXNjYWxsLmM6MTg4DQojMTMgYW1k NjRfc3lzY2FsbCAodGQ9MHhmZmZmZmUwMjI5MjViM2EwLCB0cmFjZWQ9MCkgYXQgDQovdXNy L3NyYy9zeXMvYW1kNjQvYW1kNjQvdHJhcC5jOjExOTQNCiMxNCA8c2lnbmFsIGhhbmRsZXIg Y2FsbGVkPg0KIzE1IDB4MDAwMDA3ZmJiMmRhNGFkYSBpbiA/PyAoKQ0KDQoNCmFuZCBkaXNh YmxpbmcgYmNsb25lIGRvZXMgYXBwZWFyIHRvIGFsbG93IG1lIHRvIGZpbmlzaCBidWlsZHdv cmxkIC8gDQpidWlsZGtlcm5lbC4NCg0KVGhlIHBvb2wgZGlkbid0IGhhdmUgcmVkYWN0aW9u X2xpc3Rfc3BpbGwgZW5hYmxlZC4NCg0KVGhlIEFTU0VSVDNVKG5icHMsID09LCBudW1idWZz KTsgaW4gZG11X2JydF9jbG9uZSB3YXMgYWRkZWQgd2hlbiBibG9jayANCmNsb25lIGlzIGZp cnN0IGltcGxlbWVudGVkLg0KDQpJdCBzZWVtcyB0aGF0IEkgYW0gdGhlIG9ubHkgcGVyc29u IHdobyBpcyBzZWVpbmcgdGhpcyBhcyBvZiB0b2RheS4gIEl0IA0Kc2VlbXMgdGhhdCBibG9j ayBjbG9uZSB3YXMgaW5kZWVkIGJlaW5nIHVzZWQgZm9yIHNvbWUgZGF0YToNCg0Kc2F0dXJu ICBiY2xvbmV1c2VkICAgICAgICAgICAgICAgICAgICAgMS4xOE0gICAgICAgICAgICAgICAg ICAgICAgICAgIC0NCnNhdHVybiAgYmNsb25lc2F2ZWQgICAgICAgICAgICAgICAgICAgIDEu MjFNICAgICAgICAgICAgICAgICAgICAgICAgICAtDQpzYXR1cm4gIGJjbG9uZXJhdGlvICAg ICAgICAgICAgICAgICAgICAyLjAyeCAgICAgICAgICAgICAgICAgICAgICAgICAgLQ0KDQpU aGUgcG9vbCBoYXZlIGRlZHVwIGVuYWJsZWQgZm9yIHNvbWUgZGF0YXNldHMuDQoNCkFueSBz dWdnZXN0aW9ucz8gIChJbiBleHRyZW1lIGNhc2VzIEkgY2FuIHJlY3JlYXRlIHRoZSBzdG9y YWdlIHBvb2wgZnJvbSANCmJhY2t1cCBvciBjb3B5IHRoZSBkYXRhIHNvbWV3aGVyZSBlbHNl LCB0aGVuIHJlY3JlYXRlIHRoZSBwb29sLCB0aGVuIA0KY29weSBkYXRhIGJhY2ssIGJ1dCBJ J2QgbGlrZSB0byBhdm9pZCB0aGF0IGlmIHBvc3NpYmxlKQ0KDQpDaGVlcnMsDQoNCg0KPiAN Cj4gbW0NCj4gDQo+IE9uIDA0LzExLzIwMjMgMTM6MzUsIE1hcmsgTWlsbGFyZCB3cm90ZToN Cj4+IE9uIE5vdiA0LCAyMDIzLCBhdCAwNDozOCwgTWlrZSBLYXJlbHMgPG1pa2VAa2FyZWxz Lm5ldD4gd3JvdGU6DQo+Pg0KPj4+IE9uIDQgTm92IDIwMjMsIGF0IDQ6MDEsIFJvbmFsZCBL bG9wIHdyb3RlOg0KPj4+DQo+Pj4+IE9uIDExLzQvMjMgMDI6MzksIE1hcmsgTWlsbGFyZCB3 cm90ZToNCj4+Pj4+IEl0IGxvb2tzIHRvIG1lIGxpa2UgcmVsZW5nLzE0LjAgKGFzIG9mIDE0 LjAtUkM0KSBzdGlsbCBoYXM6DQo+Pj4+Pg0KPj4+Pj4gaW50IHpmc19iY2xvbmVfZW5hYmxl ZDsNCj4+Pj4+IFNZU0NUTF9JTlQoX3Zmc196ZnMsIE9JRF9BVVRPLCBiY2xvbmVfZW5hYmxl ZCwgQ1RMRkxBR19SV1RVTiwNCj4+Pj4+ICZ6ZnNfYmNsb25lX2VuYWJsZWQsIDAsICJFbmFi bGUgYmxvY2sgY2xvbmluZyIpOw0KPj4+Pj4NCj4+Pj4+IGxlYXZpbmcgYmxvY2sgY2xvbmlu ZyBlZmZlY3RpdmVseSBkaXNhYmxlZCBieSBkZWZhdWx0LCBubw0KPj4+Pj4gbWF0dGVyIHdo YXQgdGhlIHBvb2wgaGFzIGVuYWJsZWQuDQo+Pj4+Pg0KPj4+Pj4gaHR0cHM6Ly93d3cuZnJl ZWJzZC5vcmcvcmVsZWFzZXMvMTQuMFIvcmVsbm90ZXMvIGFsc28gcmVwb3J0czoNCj4+Pj4+ DQo+Pj4+PiBRVU9URQ0KPj4+Pj4gT3BlblpGUyBoYXMgYmVlbiB1cGdyYWRlZCB0byB2ZXJz aW9uIDIuMi4gTmV3IGZlYXR1cmVzIGluY2x1ZGU6DQo+Pj4+PiDigKINCj4+Pj4+IGJsb2Nr IGNsb25pbmcsIHdoaWNoIGFsbG93cyBzaGFsbG93IGNvcGllcyBvZiBibG9ja3MgaW4gZmls ZSANCj4+Pj4+IGNvcGllcy4gVGhpcyBpcyBvcHRpb25hbCwgYW5kIGRpc2FibGVkIGJ5IGRl ZmF1bHQ7IGl0IGNhbiBiZSANCj4+Pj4+IGVuYWJsZWQgd2l0aCBzeXNjdGwgdmZzLnpmcy5i Y2xvbmVfZW5hYmxlZD0xLg0KPj4+Pj4gRU5EIFFVT1RFDQo+Pj4+Pg0KPj4+Pg0KPj4+PiBJ IHRoaW5rIHRoaXMgYW5zd2VycyB5b3VyIHF1ZXN0aW9uIGluIHRoZSBzdWJqZWN0Lg0KPj4+ IEkgdGhpbmsgc28gdG9vIChhbmQgSSB3cm90ZSB0aGF0IHRleHQpLg0KPj4gVGhhbmtzIGZv ciB0aGUgY29uZmlybWF0aW9uIG9mIHRoZSBmaW5hbCBpbnRlbnQuDQo+Pg0KPj4gSSBiZWxp ZXZlIHRoaXMgbWFrZXM6DQo+Pg0KPj4gUVVPVEUNCj4+IGF1dGhvciBCcmlhbiBCZWhsZW5k b3JmIDxiZWhsZW5kb3JmMUBsbG5sLmdvdj4gMjAyMy0wNS0yNSAyMDo1MzowOCArMDAwMA0K Pj4gY29tbWl0dGVyIEdpdEh1YiA8bm9yZXBseUBnaXRodWIuY29tPiAyMDIzLTA1LTI1IDIw OjUzOjA4ICswMDAwDQo+PiBjb21taXQgOTFhMjMyNWM0YTBmYmUwMWQwYmYyMTJlNDRmYTlk ODUwMTc4MzdjZSAocGF0Y2gpDQo+PiB0cmVlIGRkMDFkZmNlNmFlZWYzNTdhZGUxNzc1YWNm MThhYWRlNTM1YzYyNzENCj4+IC4gLiAuDQo+PiBVcGRhdGUgY29tcGF0aWJpbGl0eS5kIGZp bGVzDQo+Pg0KPj4gQWRkIGFuIG9wZW56ZnMtMi4yIGNvbXBhdGliaWxpdHkgZmlsZSBmb3Ig dGhlIG5leHQgcmVsZWFzZS4gRWRvbi1SIA0KPj4gc3VwcG9ydCBoYXMgYmVlbiBlbmFibGVk IGZvciBGcmVlQlNEIHJlbW92aW5nIHRoZSBuZWVkIGZvciBkaWZmZXJlbnQgDQo+PiBGcmVl QlNEIGFuZCBMaW51eCBmaWxlcy4gU3ltbGlua3MgZm9yIHRoZSAtbGludXggYW5kIC1mcmVl YnNkIG5hbWVzIA0KPj4gYXJlIGNyZWF0ZWQgZm9yIGFueSBzY3JpcHRzIGV4cGVjdGluZyB0 aGF0IGNvbnZlbnRpb24uIEFkZGl0aW9uYWxseSwgYSANCj4+IHN5bWxpbmsgZm9yIHVidW51 dHUtMjIuMDQgd2FzIGFkZGVkLiBTaWduZWQtb2ZmLWJ5OiBCcmlhbiBCZWhsZW5kb3JmIA0K Pj4gPGJlaGxlbmRvcmYxQGxsbmwuZ292PiBDbG9zZXMgIzE0ODMzDQo+PiBFTkQgUVVPVEUN Cj4+DQo+PiB0ZWNobmljYWxseSBpbmNvcnJlY3QgaW4gdGhhdCBjb21wYXRpYmlsaXR5LmQv b3Blbnpmcy0yLjItZnJlZWJzZA0KPj4gc2hvdWxkIGJlIGRpc3RpbmN0IGluIGNvbnRlbnQg ZnJvbSBjb21wYXRpYmlsaXR5LmQvb3Blbnpmcy0yLjIgc28NCj4+IHRoYXQgYmxvY2sgY2xv bmluZyB3b3VsZCBub3QgYmUgZW5hYmxlZC4NCj4+DQo+Pg0KPj4+Pj4gSnVzdCBjdXJpb3Vz aXR5IG9uIG15IHBhcnQgYWJvdXQgdGhlIGRlZmF1bHQgY29tcGxldGVuZXNzIG9mDQo+Pj4+ PiBvcGVuemZzLTIuMiBzdXBwb3J0LCBub3QgYW4gb2JqZWN0aW9uIGVpdGhlciB3YXkuDQo+ Pj4+Pg0KPj4+Pg0KPj4+PiBJIGhhdmVuJ3Qgc2VlbiBuZXcgaXNzdWVzIHdpdGggYmxvY2sg Y2xvbmluZyBpbiB0aGUgbGFzdCBmZXcgd2Vla3MgDQo+Pj4+IG1lbnRpb25lZCBvbiB0aGUg bWFpbGluZyBsaXN0cy4gQWxsIGtub3duIGlzc3VlcyBhcmUgZml4ZWQgQUZBSUsuDQo+Pj4+ IEJ1dCBJIGNhbiBpbWFnaW5lIHRoYXQgdGhlIHJpc2srZWZmZWN0IHJhdGlvIG9mIGRhdGEg Y29ycnVwdGlvbiBpcyANCj4+Pj4gc2VlbiBhcyBhIGJpdCB0b28gaGlnaCBmb3IgYSAxNC4w IHJlbGVhc2UgZm9yIHRoaXMgcGFydGljdWxhciANCj4+Pj4gZmVhdHVyZS4gVGhhdCBkb2Vz IG5vdCBkaW1pbmlzaCB0aGUgcmVzdCBvZiB0aGUgY29tcGxldGVuZXNzIG9mIA0KPj4+PiBv cGVuemZzLTIuMi4NCj4+Pj4NCj4+Pj4gTkI6IEknbSBub3QgaW52b2x2ZWQgaW4gZGV2ZWxv cGluZyBvcGVuemZzIG9yIHRoZSBkZWNpc2lvbiBtYWtpbmcgaW4gDQo+Pj4+IHRoZSByZWxl YXNlLiBKdXN0IHJlcGVhdGluZyB3aGF0IEkgcmVhZCBvbiB0aGUgbGlzdHMuDQo+Pj4gVGhl cmUgd2FzIGFub3RoZXIgYmxvY2sgY2xvbmluZyBmaXggaW4gMTQuMC1SQzQ7IHNlZSB0aGUg Y29tbWl0IGxvZy4NCj4+PiBNYXliZSB0aGVyZSB3aWxsIGJlIG5vIG1vcmUgaXNzdWVzLCBi dXQgaXQgc2VlbXMgdGhhdCBjb3JuZXIgY2FzZXMgd2VyZQ0KPj4+IHN0aWxsIGJlaW5nIGZv dW5kIHJlY2VudGx5Lg0KPj4gTG9va3MgbGlrZSBJJ2xsIHN0YXkgYXQgb3Blbnpmcy0yLjEg cG9vbCBmZWF0dXJlcyB1bnRpbCB0aGVyZSBpcw0KPj4gYSByZWxlYXNlIHRoYXQgbm8gbG9u Z2VyIGhhcyB0aGUgZGVmYXVsdCBzdGF0dXM6DQo+Pg0KPj4gMCBmb3Igc3lzY3RsIHZmcy56 ZnMuYmNsb25lX2VuYWJsZWQNCj4+DQo+PiBJIHVzZSBtYWluIFtzbzogMTUgbm93XSBidXQg b25seSBlbmFibGUgb3Blbnpmcy0yLiogcG9vbCBmZWF0dXJlcw0KPj4gc3VwcG9ydGVkIGJ5 IGRlZmF1bHQgb24gc29tZSBGcmVlQlNEIHJlbGVhc2UsIHRoYXQgaGFzIGFuIGFjY3VyYXRl DQo+PiBjb21wYXRpYmlsaXR5LmQvb3Blbnpmcy0yLiotZnJlZWJzZCBmaWxlLg0KPj4NCj4+ ID09PQ0KPj4gTWFyayBNaWxsYXJkDQo+PiBtYXJrbG1pIGF0IHlhaG9vLmNvbQ0KPj4NCj4+ DQo+IA0KDQo= --------------jvk50lwEhNzmRy3Of6INPY0V-- --------------tZbYpB30DpTzqxB0ZbCDEv0z Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature.asc" -----BEGIN PGP SIGNATURE----- wnsEABYIACMWIQS7JJNqVzWrbGqXjc8RA+Lrl1nkxQUCZU3V9AUDAAAAAAAKCRARA+Lrl1nkxb4D AQCFInkBMgpmzhuHaLQe4eyuWHWIF+W62ap/iBOMTI+tuwD7BLApj62nKIr5jSSyDt8wUibAUW93 4SyQdbs2MuTOSg4= =CzEl -----END PGP SIGNATURE----- --------------tZbYpB30DpTzqxB0ZbCDEv0z-- From nobody Fri Nov 10 14:31:38 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRh8z3gKsz512DS for ; Fri, 10 Nov 2023 14:31:47 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRh8z1Xtgz4ctB; Fri, 10 Nov 2023 14:31:47 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy02.your-server.de ([78.47.166.47]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r1SXr-0006Fm-3G; Fri, 10 Nov 2023 15:31:39 +0100 Received: from [188.167.171.2] (helo=[10.0.9.225]) by sslproxy02.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r1SXq-0000W4-SD; Fri, 10 Nov 2023 15:31:38 +0100 Message-ID: Date: Fri, 10 Nov 2023 15:31:38 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? Content-Language: en-US To: d@delphij.net, FreeBSD-STABLE Mailing List , pjd@FreeBSD.org References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> From: Martin Matuska In-Reply-To: <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.10/27089/Fri Nov 10 09:39:24 2023) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Queue-Id: 4SRh8z1Xtgz4ctB Hi Xin, since when have you been using block cloning on the system? Is it possible that there is already corrupted block-cloned data from the past? Is everything on one dataset or are you using multiple datasets for /usr/src and /usr/obj? Best regards, mm On 10. 11. 2023 8:04, Xin Li wrote: > On 2023-11-05 16:34, Martin Matuska wrote: >> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can >> work with pools that have feature@block_cloning enabled. >> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of >> zfs_clone_range() which is called by copy_file_range(). When it is >> set to 0, zfs_clone_range() does not do block cloning. >> If it is set to anything else than 0, zfs_clone_range() does block >> cloning (if all conditions are met - same ZFS pool, correct data >> alignment, etc.). >> >> In FreeBSD-main, this tunable is enabled and I plan to enable it in >> stable/14 somewhere around December 11, 2023. >> >> As of today I personally use block cloning on all my systems. > > I'd like to share a different data point.  It still panics on my > storage (running -CURRENT about a week ago) when enabled and can be > triggered by "make buildworld buildkernel".  I wasn't able to capture > earlier coredump until the most recent one, which panicked with: > > > cpuid = 2 > time = 1699593456 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe022f2bd7e0 > vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910 > spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970 > dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0 > zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > zfs_freebsd_copy_file_range() at > zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30 > vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bdce0 > kern_copy_file_range() at kern_copy_file_range+0x380/frame > 0xfffffe022f2bddb0 > sys_copy_file_range() at sys_copy_file_range+0x78/frame > 0xfffffe022f2bde00 > amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30 > fast_syscall_common() at fast_syscall_common+0xf8/frame > 0xfffffe022f2bdf30 > --- syscall (569, FreeBSD ELF64, copy_file_range), rip = > 0x7fbb2da4ada, rsp = 0x7fbb02c5d48, rbp = 0x7fbb02c61e0 --- > Uptime: 2h32m27s > Dumping 7800 out of 32696 > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > #1  doadump (textdump=textdump@entry=1) at > /usr/src/sys/kern/kern_shutdown.c:405 > #2  0xffffffff80694480 in kern_reboot (howto=260) at > /usr/src/sys/kern/kern_shutdown.c:526 > #3  0xffffffff8069497f in vpanic (fmt=0xffffffff82603415 "VERIFY3(nbps > == numbufs) failed (%llu == %llu)\n", ap=ap@entry=0xfffffe022f2bd950) > at /usr/src/sys/kern/kern_shutdown.c:970 > #4  0xffffffff8232999a in spl_panic (file=, > func=, line=, fmt=) at > /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103 > #5  0xffffffff823a6605 in dmu_brt_clone > (os=os@entry=0xfffff800c5ce4000, object=, > offset=offset@entry=0, length=length@entry=207477, > tx=tx@entry=0xfffff8071a108d00, bps=bps@entry=0xfffffe01e218c000, > nbps=2, replay=0) >     at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303 > #6  0xffffffff8250f67c in zfs_clone_range (inzp=0xfffff804416ac000, > inoffp=0xfffff800b81cb048, outzp=0xfffff806f58f03a0, > outoffp=0xfffff800b8063048, lenp=lenp@entry=0xfffffe022f2bdbf0, > cr=0xfffff8000a6fe600) >     at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326 > #7  0xffffffff8234b3ba in zfs_freebsd_copy_file_range > (ap=0xfffffe022f2bdc48) at > /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 > #8  0xffffffff8079f443 in VOP_COPY_FILE_RANGE > (invp=0xfffff804416cb1c0, inoffp=0xfffff800b81cb048, > outvp=0xfffff806f51d3380, outoffp=0xfffff800b8063048, > lenp=0xfffffe022f2bdd48, incred=0xfffff8000a6fe600, flags= out>, >     outcred=, fsizetd=) at > ./vnode_if.h:2385 > #9  vn_copy_file_range (invp=invp@entry=0xfffff804416cb1c0, > inoffp=inoffp@entry=0xfffff800b81cb048, > outvp=outvp@entry=0xfffff806f51d3380, > outoffp=outoffp@entry=0xfffff800b8063048, > lenp=lenp@entry=0xfffffe022f2bdd48, flags=flags@entry=0, >     incred=0xfffff8000a6fe600, outcred=0xfffff8000a6fe600, > fsize_td=0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > #10 0xffffffff8079a070 in kern_copy_file_range > (td=td@entry=0xfffffe022925b3a0, infd=, > inoffp=0xfffff800b81cb048, inoffp@entry=0x0, outfd=, > outoffp=0xfffff800b8063048, outoffp@entry=0x0, len=9223372036854775807, >     flags=0) at /usr/src/sys/kern/vfs_syscalls.c:4973 > #11 0xffffffff8079a178 in sys_copy_file_range (td=0xfffffe022925b3a0, > uap=0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011 > #12 0xffffffff80a97aa3 in syscallenter (td=0xfffffe022925b3a0) at > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188 > #13 amd64_syscall (td=0xfffffe022925b3a0, traced=0) at > /usr/src/sys/amd64/amd64/trap.c:1194 > #14 > #15 0x000007fbb2da4ada in ?? () > > > and disabling bclone does appear to allow me to finish buildworld / > buildkernel. > > The pool didn't have redaction_list_spill enabled. > > The ASSERT3U(nbps, ==, numbufs); in dmu_brt_clone was added when block > clone is first implemented. > > It seems that I am the only person who is seeing this as of today.  It > seems that block clone was indeed being used for some data: > > saturn  bcloneused 1.18M                          - > saturn  bclonesaved 1.21M                          - > saturn  bcloneratio 2.02x                          - > > The pool have dedup enabled for some datasets. > > Any suggestions?  (In extreme cases I can recreate the storage pool > from backup or copy the data somewhere else, then recreate the pool, > then copy data back, but I'd like to avoid that if possible) > > Cheers, > > >> >> mm >> >> On 04/11/2023 13:35, Mark Millard wrote: >>> On Nov 4, 2023, at 04:38, Mike Karels wrote: >>> >>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote: >>>> >>>>> On 11/4/23 02:39, Mark Millard wrote: >>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: >>>>>> >>>>>> int zfs_bclone_enabled; >>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, >>>>>> &zfs_bclone_enabled, 0, "Enable block cloning"); >>>>>> >>>>>> leaving block cloning effectively disabled by default, no >>>>>> matter what the pool has enabled. >>>>>> >>>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: >>>>>> >>>>>> QUOTE >>>>>> OpenZFS has been upgraded to version 2.2. New features include: >>>>>> • >>>>>> block cloning, which allows shallow copies of blocks in file >>>>>> copies. This is optional, and disabled by default; it can be >>>>>> enabled with sysctl vfs.zfs.bclone_enabled=1. >>>>>> END QUOTE >>>>>> >>>>> >>>>> I think this answers your question in the subject. >>>> I think so too (and I wrote that text). >>> Thanks for the confirmation of the final intent. >>> >>> I believe this makes: >>> >>> QUOTE >>> author Brian Behlendorf 2023-05-25 20:53:08 >>> +0000 >>> committer GitHub 2023-05-25 20:53:08 +0000 >>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) >>> tree dd01dfce6aeef357ade1775acf18aade535c6271 >>> . . . >>> Update compatibility.d files >>> >>> Add an openzfs-2.2 compatibility file for the next release. Edon-R >>> support has been enabled for FreeBSD removing the need for different >>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd names >>> are created for any scripts expecting that convention. Additionally, >>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian >>> Behlendorf Closes #14833 >>> END QUOTE >>> >>> technically incorrect in that compatibility.d/openzfs-2.2-freebsd >>> should be distinct in content from compatibility.d/openzfs-2.2 so >>> that block cloning would not be enabled. >>> >>> >>>>>> Just curiousity on my part about the default completeness of >>>>>> openzfs-2.2 support, not an objection either way. >>>>>> >>>>> >>>>> I haven't seen new issues with block cloning in the last few weeks >>>>> mentioned on the mailing lists. All known issues are fixed AFAIK. >>>>> But I can imagine that the risk+effect ratio of data corruption is >>>>> seen as a bit too high for a 14.0 release for this particular >>>>> feature. That does not diminish the rest of the completeness of >>>>> openzfs-2.2. >>>>> >>>>> NB: I'm not involved in developing openzfs or the decision making >>>>> in the release. Just repeating what I read on the lists. >>>> There was another block cloning fix in 14.0-RC4; see the commit log. >>>> Maybe there will be no more issues, but it seems that corner cases >>>> were >>>> still being found recently. >>> Looks like I'll stay at openzfs-2.1 pool features until there is >>> a release that no longer has the default status: >>> >>> 0 for sysctl vfs.zfs.bclone_enabled >>> >>> I use main [so: 15 now] but only enable openzfs-2.* pool features >>> supported by default on some FreeBSD release, that has an accurate >>> compatibility.d/openzfs-2.*-freebsd file. >>> >>> === >>> Mark Millard >>> marklmi at yahoo.com >>> >>> >> > From nobody Fri Nov 10 16:58:57 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRlR5167Nz509S5 for ; Fri, 10 Nov 2023 16:59:13 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRlR45TbXz3Zgx; Fri, 10 Nov 2023 16:59:12 +0000 (UTC) (envelope-from delphij@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-9e2838bcb5eso373508566b.0; Fri, 10 Nov 2023 08:59:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699635549; x=1700240349; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=NE7A7tPmDKUuN515nzcLndWyjcJPIIY1RfzXoKXy1mo=; b=JEijXXcQIjNWg9IOYAD3/ZgIGSIliYmx+XnDzP4shiacEIz4XZ3c8pyu4EkaFBg7dV pCIqG3nVKyw712vtf5Z2UAIqOEN4JeuUtH6byrw5YaQ0ADta8b9Zj8poXKTGrMnW0fhB ubuTMzvQ3PzjYPtKNVWxwVtjK5MJBCZvYpCfC57A5xxWyokP6POE5PY6ttH4jHL8+j3J CnWk10OQWHq8U1Si/tyM0j9MRMPzaBNaEcgQFVUtEhxjypI2oCyWRrtbxP8Ar989fGOi B6Dfn3C+nhIZfSxzH7LlFeLU8WYgde0YPkuCXgCvqNEgFnHof9QOljgSzA0sZLMGPMLR AtMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699635549; x=1700240349; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NE7A7tPmDKUuN515nzcLndWyjcJPIIY1RfzXoKXy1mo=; b=SJ6zmF4Kqm6b9c9NaQJkIzzyrNrh7X3Df4g/8LyU63Bw/FjfQOXucDFb/IStOLx8nt uyJjXQE+gWHvpzyvyDCBr8qJzwjYSzGwFj6yj+qLVFjcXIazJXJTbLgovLdg1vpfDkI8 AQyr/7aHrBL9Kk/sUvLKpyeNjQCFq0YCEJ7ax9ZcyrLup4QzHozvs6whJCqHQgsXBuDG B+bMAwfuH5oo8PjgePUqEGPj63I2vn9jV5zlb/X4lXeKPUrtDcDwBozfA/aao7kLPn0+ DUq7cuz7Wqq38t3NtZNIzXbT3AKOUYKls8+vKoWibVlVGdXFMzXMBNUpxPIghqJKlKNN 0smg== X-Gm-Message-State: AOJu0YxeYjj8vALZPVkIJx2WG0pztX0D9Wv/j7xGhp7ZxytpAkvlC92D RIqezfAdHfv1CZyDKVAi9NQh0gKZi6ofrlAKQ6yOsUfAMOI= X-Google-Smtp-Source: AGHT+IH9sOK0RhBmXASZkFIjpvpzEq15rIep8lAtlSqXIHKMwGu0MG4thCgote90F1Q07FACzOoOC6ETMSHCYq0UC48= X-Received: by 2002:a17:907:d8e:b0:9ae:6a08:6f53 with SMTP id go14-20020a1709070d8e00b009ae6a086f53mr7671054ejc.63.1699635548820; Fri, 10 Nov 2023 08:59:08 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> In-Reply-To: From: Xin LI Date: Fri, 10 Nov 2023 08:58:57 -0800 Message-ID: Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? To: Martin Matuska Cc: d@delphij.net, FreeBSD-STABLE Mailing List , pjd@freebsd.org Content-Type: multipart/alternative; boundary="0000000000002297970609cf3dc8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4SRlR45TbXz3Zgx --0000000000002297970609cf3dc8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 10, 2023 at 7:50=E2=80=AFAM Martin Matuska wro= te: > Hi Xin, > > since when have you been using block cloning on the system? Is it > possible that there is already corrupted block-cloned data from the That's a good question, I can't 100% rule out this possibility. I was following -CURRENT in ~weekly to ~monthly on that system, and the pool was created in March 2014. Do you think I should try rebuilding the pool from scratch? I do have remote backup on a different server but was avoiding it because it's time consuming. > past? Is everything on one dataset or are you using multiple datasets > for /usr/src and /usr/obj? > /usr/src and /usr/obj are separate datasets, and the system runs Poudriere so it have multiple copies of slightly different /usr/src and /usr/obj's. Is there a way to identify datasets with block cloning, by the way? Maybe I should try recreating these datasets first? > > Best regards, > mm > > On 10. 11. 2023 8:04, Xin Li wrote: > > On 2023-11-05 16:34, Martin Matuska wrote: > >> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can > >> work with pools that have feature@block_cloning enabled. > >> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of > >> zfs_clone_range() which is called by copy_file_range(). When it is > >> set to 0, zfs_clone_range() does not do block cloning. > >> If it is set to anything else than 0, zfs_clone_range() does block > >> cloning (if all conditions are met - same ZFS pool, correct data > >> alignment, etc.). > >> > >> In FreeBSD-main, this tunable is enabled and I plan to enable it in > >> stable/14 somewhere around December 11, 2023. > >> > >> As of today I personally use block cloning on all my systems. > > > > I'd like to share a different data point. It still panics on my > > storage (running -CURRENT about a week ago) when enabled and can be > > triggered by "make buildworld buildkernel". I wasn't able to capture > > earlier coredump until the most recent one, which panicked with: > > > > > > cpuid =3D 2 > > time =3D 1699593456 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > 0xfffffe022f2bd7e0 > > vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910 > > spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970 > > dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0 > > zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > > zfs_freebsd_copy_file_range() at > > zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30 > > vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bdc= e0 > > kern_copy_file_range() at kern_copy_file_range+0x380/frame > > 0xfffffe022f2bddb0 > > sys_copy_file_range() at sys_copy_file_range+0x78/frame > > 0xfffffe022f2bde00 > > amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30 > > fast_syscall_common() at fast_syscall_common+0xf8/frame > > 0xfffffe022f2bdf30 > > --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D > > 0x7fbb2da4ada, rsp =3D 0x7fbb02c5d48, rbp =3D 0x7fbb02c61e0 --- > > Uptime: 2h32m27s > > Dumping 7800 out of 32696 > > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > > > #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > > #1 doadump (textdump=3Dtextdump@entry=3D1) at > > /usr/src/sys/kern/kern_shutdown.c:405 > > #2 0xffffffff80694480 in kern_reboot (howto=3D260) at > > /usr/src/sys/kern/kern_shutdown.c:526 > > #3 0xffffffff8069497f in vpanic (fmt=3D0xffffffff82603415 "VERIFY3(nbp= s > > =3D=3D numbufs) failed (%llu =3D=3D %llu)\n", ap=3Dap@entry=3D0xfffffe0= 22f2bd950) > > at /usr/src/sys/kern/kern_shutdown.c:970 > > #4 0xffffffff8232999a in spl_panic (file=3D, > > func=3D, line=3D, fmt=3D) at > > /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103 > > #5 0xffffffff823a6605 in dmu_brt_clone > > (os=3Dos@entry=3D0xfffff800c5ce4000, object=3D, > > offset=3Doffset@entry=3D0, length=3Dlength@entry=3D207477, > > tx=3Dtx@entry=3D0xfffff8071a108d00, bps=3Dbps@entry=3D0xfffffe01e218c00= 0, > > nbps=3D2, replay=3D0) > > at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303 > > #6 0xffffffff8250f67c in zfs_clone_range (inzp=3D0xfffff804416ac000, > > inoffp=3D0xfffff800b81cb048, outzp=3D0xfffff806f58f03a0, > > outoffp=3D0xfffff800b8063048, lenp=3Dlenp@entry=3D0xfffffe022f2bdbf0, > > cr=3D0xfffff8000a6fe600) > > at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326 > > #7 0xffffffff8234b3ba in zfs_freebsd_copy_file_range > > (ap=3D0xfffffe022f2bdc48) at > > /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 > > #8 0xffffffff8079f443 in VOP_COPY_FILE_RANGE > > (invp=3D0xfffff804416cb1c0, inoffp=3D0xfffff800b81cb048, > > outvp=3D0xfffff806f51d3380, outoffp=3D0xfffff800b8063048, > > lenp=3D0xfffffe022f2bdd48, incred=3D0xfffff8000a6fe600, flags=3D > out>, > > outcred=3D, fsizetd=3D) at > > ./vnode_if.h:2385 > > #9 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff804416cb1c0, > > inoffp=3Dinoffp@entry=3D0xfffff800b81cb048, > > outvp=3Doutvp@entry=3D0xfffff806f51d3380, > > outoffp=3Doutoffp@entry=3D0xfffff800b8063048, > > lenp=3Dlenp@entry=3D0xfffffe022f2bdd48, flags=3Dflags@entry=3D0, > > incred=3D0xfffff8000a6fe600, outcred=3D0xfffff8000a6fe600, > > fsize_td=3D0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > > #10 0xffffffff8079a070 in kern_copy_file_range > > (td=3Dtd@entry=3D0xfffffe022925b3a0, infd=3D, > > inoffp=3D0xfffff800b81cb048, inoffp@entry=3D0x0, outfd=3D, > > outoffp=3D0xfffff800b8063048, outoffp@entry=3D0x0, len=3D92233720368547= 75807, > > flags=3D0) at /usr/src/sys/kern/vfs_syscalls.c:4973 > > #11 0xffffffff8079a178 in sys_copy_file_range (td=3D0xfffffe022925b3a0, > > uap=3D0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011 > > #12 0xffffffff80a97aa3 in syscallenter (td=3D0xfffffe022925b3a0) at > > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188 > > #13 amd64_syscall (td=3D0xfffffe022925b3a0, traced=3D0) at > > /usr/src/sys/amd64/amd64/trap.c:1194 > > #14 > > #15 0x000007fbb2da4ada in ?? () > > > > > > and disabling bclone does appear to allow me to finish buildworld / > > buildkernel. > > > > The pool didn't have redaction_list_spill enabled. > > > > The ASSERT3U(nbps, =3D=3D, numbufs); in dmu_brt_clone was added when bl= ock > > clone is first implemented. > > > > It seems that I am the only person who is seeing this as of today. It > > seems that block clone was indeed being used for some data: > > > > saturn bcloneused 1.18M - > > saturn bclonesaved 1.21M - > > saturn bcloneratio 2.02x - > > > > The pool have dedup enabled for some datasets. > > > > Any suggestions? (In extreme cases I can recreate the storage pool > > from backup or copy the data somewhere else, then recreate the pool, > > then copy data back, but I'd like to avoid that if possible) > > > > Cheers, > > > > > >> > >> mm > >> > >> On 04/11/2023 13:35, Mark Millard wrote: > >>> On Nov 4, 2023, at 04:38, Mike Karels wrote: > >>> > >>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote: > >>>> > >>>>> On 11/4/23 02:39, Mark Millard wrote: > >>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: > >>>>>> > >>>>>> int zfs_bclone_enabled; > >>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, > >>>>>> &zfs_bclone_enabled, 0, "Enable block cloning"); > >>>>>> > >>>>>> leaving block cloning effectively disabled by default, no > >>>>>> matter what the pool has enabled. > >>>>>> > >>>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: > >>>>>> > >>>>>> QUOTE > >>>>>> OpenZFS has been upgraded to version 2.2. New features include: > >>>>>> =E2=80=A2 > >>>>>> block cloning, which allows shallow copies of blocks in file > >>>>>> copies. This is optional, and disabled by default; it can be > >>>>>> enabled with sysctl vfs.zfs.bclone_enabled=3D1. > >>>>>> END QUOTE > >>>>>> > >>>>> > >>>>> I think this answers your question in the subject. > >>>> I think so too (and I wrote that text). > >>> Thanks for the confirmation of the final intent. > >>> > >>> I believe this makes: > >>> > >>> QUOTE > >>> author Brian Behlendorf 2023-05-25 20:53:08 > >>> +0000 > >>> committer GitHub 2023-05-25 20:53:08 +0000 > >>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) > >>> tree dd01dfce6aeef357ade1775acf18aade535c6271 > >>> . . . > >>> Update compatibility.d files > >>> > >>> Add an openzfs-2.2 compatibility file for the next release. Edon-R > >>> support has been enabled for FreeBSD removing the need for different > >>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd names > >>> are created for any scripts expecting that convention. Additionally, > >>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian > >>> Behlendorf Closes #14833 > >>> END QUOTE > >>> > >>> technically incorrect in that compatibility.d/openzfs-2.2-freebsd > >>> should be distinct in content from compatibility.d/openzfs-2.2 so > >>> that block cloning would not be enabled. > >>> > >>> > >>>>>> Just curiousity on my part about the default completeness of > >>>>>> openzfs-2.2 support, not an objection either way. > >>>>>> > >>>>> > >>>>> I haven't seen new issues with block cloning in the last few weeks > >>>>> mentioned on the mailing lists. All known issues are fixed AFAIK. > >>>>> But I can imagine that the risk+effect ratio of data corruption is > >>>>> seen as a bit too high for a 14.0 release for this particular > >>>>> feature. That does not diminish the rest of the completeness of > >>>>> openzfs-2.2. > >>>>> > >>>>> NB: I'm not involved in developing openzfs or the decision making > >>>>> in the release. Just repeating what I read on the lists. > >>>> There was another block cloning fix in 14.0-RC4; see the commit log. > >>>> Maybe there will be no more issues, but it seems that corner cases > >>>> were > >>>> still being found recently. > >>> Looks like I'll stay at openzfs-2.1 pool features until there is > >>> a release that no longer has the default status: > >>> > >>> 0 for sysctl vfs.zfs.bclone_enabled > >>> > >>> I use main [so: 15 now] but only enable openzfs-2.* pool features > >>> supported by default on some FreeBSD release, that has an accurate > >>> compatibility.d/openzfs-2.*-freebsd file. > >>> > >>> =3D=3D=3D > >>> Mark Millard > >>> marklmi at yahoo.com > >>> > >>> > >> > > > > --0000000000002297970609cf3dc8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Fri, Nov 10, 2023 at 7:50=E2=80= =AFAM Martin Matuska <mm@freebsd.org> wrote:
Hi= Xin,

since when have you been using block cloning on the system? Is it
possible that there is already corrupted block-cloned data from the

That's a good question, I can't 100% rule out = this possibility.=C2=A0 I was following -CURRENT in ~weekly to ~monthly on = that system, and the pool was created in March 2014.

Do you t= hink I should try rebuilding the pool from scratch?=C2=A0 I do have remote = backup on a different server but was avoiding it because it's time cons= uming.
past? Is everything on one dataset or are you using multiple datasets <= br> for /usr/src and /usr/obj?


Best regards,
mm

On 10. 11. 2023 8:04, Xin Li wrote:
> On 2023-11-05 16:34, Martin Matuska wrote:
>> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can =
>> work with pools that have feature@block_cloning enabled.
>> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of=
>> zfs_clone_range() which is called by copy_file_range(). When it is=
>> set to 0, zfs_clone_range() does not do block cloning.
>> If it is set to anything else than 0, zfs_clone_range() does block=
>> cloning (if all conditions are met - same ZFS pool, correct data <= br> >> alignment, etc.).
>>
>> In FreeBSD-main, this tunable is enabled and I plan to enable it i= n
>> stable/14 somewhere around December 11, 2023.
>>
>> As of today I personally use block cloning on all my systems.
>
> I'd like to share a different data point.=C2=A0 It still panics on= my
> storage (running -CURRENT about a week ago) when enabled and can be > triggered by "make buildworld buildkernel".=C2=A0 I wasn'= ;t able to capture
> earlier coredump until the most recent one, which panicked with:
>
>
> cpuid =3D 2
> time =3D 1699593456
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfffffe022f2bd7e0
> vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910
> spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970
> dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0
> zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > zfs_freebsd_copy_file_range() at
> zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30
> vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bd= ce0
> kern_copy_file_range() at kern_copy_file_range+0x380/frame
> 0xfffffe022f2bddb0
> sys_copy_file_range() at sys_copy_file_range+0x78/frame
> 0xfffffe022f2bde00
> amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30
> fast_syscall_common() at fast_syscall_common+0xf8/frame
> 0xfffffe022f2bdf30
> --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D
> 0x7fbb2da4ada, rsp =3D 0x7fbb02c5d48, rbp =3D 0x7fbb02c61e0 ---
> Uptime: 2h32m27s
> Dumping 7800 out of 32696
> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>
> #0=C2=A0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > #1=C2=A0 doadump (textdump=3Dtextdump@entry=3D1) at
> /usr/src/sys/kern/kern_shutdown.c:405
> #2=C2=A0 0xffffffff80694480 in kern_reboot (howto=3D260) at
> /usr/src/sys/kern/kern_shutdown.c:526
> #3=C2=A0 0xffffffff8069497f in vpanic (fmt=3D0xffffffff82603415 "= VERIFY3(nbps
> =3D=3D numbufs) failed (%llu =3D=3D %llu)\n", ap=3Dap@entry=3D0xf= ffffe022f2bd950)
> at /usr/src/sys/kern/kern_shutdown.c:970
> #4=C2=A0 0xffffffff8232999a in spl_panic (file=3D<optimized out>= ,
> func=3D<optimized out>, line=3D<unavailable>, fmt=3D<un= available>) at
> /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103
> #5=C2=A0 0xffffffff823a6605 in dmu_brt_clone
> (os=3Dos@entry=3D0xfffff800c5ce4000, object=3D<optimized out>, <= br> > offset=3Doffset@entry=3D0, length=3Dlength@entry=3D207477,
> tx=3Dtx@entry=3D0xfffff8071a108d00, bps=3Dbps@entry=3D0xfffffe01e218c0= 00,
> nbps=3D2, replay=3D0)
> =C2=A0=C2=A0=C2=A0 at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:23= 03
> #6=C2=A0 0xffffffff8250f67c in zfs_clone_range (inzp=3D0xfffff804416ac= 000,
> inoffp=3D0xfffff800b81cb048, outzp=3D0xfffff806f58f03a0,
> outoffp=3D0xfffff800b8063048, lenp=3Dlenp@entry=3D0xfffffe022f2bdbf0, =
> cr=3D0xfffff8000a6fe600)
> =C2=A0=C2=A0=C2=A0 at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnop= s.c:1326
> #7=C2=A0 0xffffffff8234b3ba in zfs_freebsd_copy_file_range
> (ap=3D0xfffffe022f2bdc48) at
> /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294=
> #8=C2=A0 0xffffffff8079f443 in VOP_COPY_FILE_RANGE
> (invp=3D0xfffff804416cb1c0, inoffp=3D0xfffff800b81cb048,
> outvp=3D0xfffff806f51d3380, outoffp=3D0xfffff800b8063048,
> lenp=3D0xfffffe022f2bdd48, incred=3D0xfffff8000a6fe600, flags=3D<op= timized
> out>,
> =C2=A0=C2=A0=C2=A0 outcred=3D<optimized out>, fsizetd=3D<opti= mized out>) at
> ./vnode_if.h:2385
> #9=C2=A0 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff804416cb1c0, <= br> > inoffp=3Dinoffp@entry=3D0xfffff800b81cb048,
> outvp=3Doutvp@entry=3D0xfffff806f51d3380,
> outoffp=3Doutoffp@entry=3D0xfffff800b8063048,
> lenp=3Dlenp@entry=3D0xfffffe022f2bdd48, flags=3Dflags@entry=3D0,
> =C2=A0=C2=A0=C2=A0 incred=3D0xfffff8000a6fe600, outcred=3D0xfffff8000a= 6fe600,
> fsize_td=3D0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > #10 0xffffffff8079a070 in kern_copy_file_range
> (td=3Dtd@entry=3D0xfffffe022925b3a0, infd=3D<optimized out>, > inoffp=3D0xfffff800b81cb048, inoffp@entry=3D0x0, outfd=3D<optimized= out>,
> outoffp=3D0xfffff800b8063048, outoffp@entry=3D0x0, len=3D9223372036854= 775807,
> =C2=A0=C2=A0=C2=A0 flags=3D0) at /usr/src/sys/kern/vfs_syscalls.c:4973=
> #11 0xffffffff8079a178 in sys_copy_file_range (td=3D0xfffffe022925b3a0= ,
> uap=3D0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011
> #12 0xffffffff80a97aa3 in syscallenter (td=3D0xfffffe022925b3a0) at > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188
> #13 amd64_syscall (td=3D0xfffffe022925b3a0, traced=3D0) at
> /usr/src/sys/amd64/amd64/trap.c:1194
> #14 <signal handler called>
> #15 0x000007fbb2da4ada in ?? ()
>
>
> and disabling bclone does appear to allow me to finish buildworld / > buildkernel.
>
> The pool didn't have redaction_list_spill enabled.
>
> The ASSERT3U(nbps, =3D=3D, numbufs); in dmu_brt_clone was added when b= lock
> clone is first implemented.
>
> It seems that I am the only person who is seeing this as of today.=C2= =A0 It
> seems that block clone was indeed being used for some data:
>
> saturn=C2=A0 bcloneused 1.18M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -
> saturn=C2=A0 bclonesaved 1.21M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -
> saturn=C2=A0 bcloneratio 2.02x=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -
>
> The pool have dedup enabled for some datasets.
>
> Any suggestions?=C2=A0 (In extreme cases I can recreate the storage po= ol
> from backup or copy the data somewhere else, then recreate the pool, <= br> > then copy data back, but I'd like to avoid that if possible)
>
> Cheers,
>
>
>>
>> mm
>>
>> On 04/11/2023 13:35, Mark Millard wrote:
>>> On Nov 4, 2023, at 04:38, Mike Karels <mike@karels.net> wrote:
>>>
>>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote:
>>>>
>>>>> On 11/4/23 02:39, Mark Millard wrote:
>>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) s= till has:
>>>>>>
>>>>>> int zfs_bclone_enabled;
>>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTL= FLAG_RWTUN,
>>>>>> &zfs_bclone_enabled, 0, "Enable block clo= ning");
>>>>>>
>>>>>> leaving block cloning effectively disabled by defa= ult, no
>>>>>> matter what the pool has enabled.
>>>>>>
>>>>>> https://www.freebsd.org/rel= eases/14.0R/relnotes/ also reports:
>>>>>>
>>>>>> QUOTE
>>>>>> OpenZFS has been upgraded to version 2.2. New feat= ures include:
>>>>>> =E2=80=A2
>>>>>> block cloning, which allows shallow copies of bloc= ks in file
>>>>>> copies. This is optional, and disabled by default;= it can be
>>>>>> enabled with sysctl vfs.zfs.bclone_enabled=3D1. >>>>>> END QUOTE
>>>>>>
>>>>>
>>>>> I think this answers your question in the subject.
>>>> I think so too (and I wrote that text).
>>> Thanks for the confirmation of the final intent.
>>>
>>> I believe this makes:
>>>
>>> QUOTE
>>> author Brian Behlendorf <behlendorf1@llnl.gov> 2023-05-25 20:53:08 >>> +0000
>>> committer GitHub <noreply@github.com> 2023-05-25 20:53:08 +0000
>>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch)
>>> tree dd01dfce6aeef357ade1775acf18aade535c6271
>>> . . .
>>> Update compatibility.d files
>>>
>>> Add an openzfs-2.2 compatibility file for the next release. Ed= on-R
>>> support has been enabled for FreeBSD removing the need for dif= ferent
>>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd = names
>>> are created for any scripts expecting that convention. Additio= nally,
>>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian >>> Behlendorf <behlendorf1@llnl.gov> Closes #14833
>>> END QUOTE
>>>
>>> technically incorrect in that compatibility.d/openzfs-2.2-free= bsd
>>> should be distinct in content from compatibility.d/openzfs-2.2= so
>>> that block cloning would not be enabled.
>>>
>>>
>>>>>> Just curiousity on my part about the default compl= eteness of
>>>>>> openzfs-2.2 support, not an objection either way.<= br> >>>>>>
>>>>>
>>>>> I haven't seen new issues with block cloning in th= e last few weeks
>>>>> mentioned on the mailing lists. All known issues are f= ixed AFAIK.
>>>>> But I can imagine that the risk+effect ratio of data c= orruption is
>>>>> seen as a bit too high for a 14.0 release for this par= ticular
>>>>> feature. That does not diminish the rest of the comple= teness of
>>>>> openzfs-2.2.
>>>>>
>>>>> NB: I'm not involved in developing openzfs or the = decision making
>>>>> in the release. Just repeating what I read on the list= s.
>>>> There was another block cloning fix in 14.0-RC4; see the c= ommit log.
>>>> Maybe there will be no more issues, but it seems that corn= er cases
>>>> were
>>>> still being found recently.
>>> Looks like I'll stay at openzfs-2.1 pool features until th= ere is
>>> a release that no longer has the default status:
>>>
>>> 0 for sysctl vfs.zfs.bclone_enabled
>>>
>>> I use main [so: 15 now] but only enable openzfs-2.* pool featu= res
>>> supported by default on some FreeBSD release, that has an accu= rate
>>> compatibility.d/openzfs-2.*-freebsd file.
>>>
>>> =3D=3D=3D
>>> Mark Millard
>>> marklmi at yahoo.com
>>>
>>>
>>
>

--0000000000002297970609cf3dc8-- From nobody Fri Nov 10 20:05:15 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRqYr0L1fz50BB5 for ; Fri, 10 Nov 2023 20:05:20 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRqYq4rM1z4XD2; Fri, 10 Nov 2023 20:05:19 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4002a.ext.cloudfilter.net ([10.228.9.250]) by cmsmtp with ESMTPS id 1QUfrbDeD8jpT1XkkrRFCA; Fri, 10 Nov 2023 20:05:18 +0000 Received: from spqr.komquats.com ([70.66.152.170]) by cmsmtp with ESMTPSA id 1XkirK8uFnCF01XkjrRijn; Fri, 10 Nov 2023 20:05:18 +0000 X-Authority-Analysis: v=2.4 cv=MPFzJeVl c=1 sm=1 tr=0 ts=654e8cfe a=y8EK/9tc/U6QY+pUhnbtgQ==:117 a=y8EK/9tc/U6QY+pUhnbtgQ==:17 a=xqWC_Br6kY4A:10 a=IkcTkHD0fZMA:10 a=BNY50KLci1gA:10 a=YxBL1-UpAAAA:8 a=6I5d2MoRAAAA:8 a=EkcXrb_YAAAA:8 a=pGLkceISAAAA:8 a=1QTDH3R-AAAA:8 a=NEAV23lmAAAA:8 a=CjxXgO3LAAAA:8 a=qjopBdeJtiYjyn9jAKoA:9 a=QEXdDO2ut3YA:10 a=Ia-lj3WSrqcvXOmTRaiG:22 a=IjZwj45LgO3ly-622nXo:22 a=LK5xJRSDVpKd5WXXoEvA:22 a=A7PbjfUNzwAiWwc5k9lq:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 2D83236A; Fri, 10 Nov 2023 12:05:16 -0800 (PST) Received: from slippy (localhost [IPv6:::1]) by slippy.cwsent.com (Postfix) with ESMTP id 18E0A217; Fri, 10 Nov 2023 12:05:16 -0800 (PST) Date: Fri, 10 Nov 2023 12:05:15 -0800 From: Cy Schubert To: Xin LI Cc: Martin Matuska , d@delphij.net, FreeBSD-STABLE Mailing List , pjd@freebsd.org Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? Message-ID: <20231110120515.71f61f69@slippy> In-Reply-To: References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> Organization: KOMQUATS X-Mailer: Claws Mail 3.19.1 (GTK+ 2.24.33; amd64-portbld-freebsd15.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-CMAE-Envelope: MS4xfG1+DLd9vqYx+sLKXPw3YiFK/weN40g7KKEACzhEoQgnwtRKc3XgTlg6GB3zxI/Ol5em46Z+e4GEZ4jUUWigB6gJR+wPsuRfN6hDut9UTLEW8pMIQEcw VPezt4aCvCqFfOlTTMsG40jNcw4QcM9jAkCUniA520aPEHslz8KoNydrn3owEwr9rDSBJjU+merSAi/myzMwzhm3cPPxeTOQyGasjR65e8GdERCTRyncpkkX 1pjIOW9tQg0rDVgvXMkEOudZD5KY3mBVSfVSfKnyulAKFu3TkKACcN48I04nrZCgMFwDmu9n3JHfOKAphbDqKQ== X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] X-Rspamd-Queue-Id: 4SRqYq4rM1z4XD2 Just a thought. When I was playing around with block_cloning last summer I made sure to create a zpool checkpoint on the pool. When it went horribly sideways restoring the checkpoint brought my pool back into a sane state. Unfortunately whatever was written to it was lost. That didn't matter as it was a play pool. Something to think about. --=20 Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=3D0 On Fri, 10 Nov 2023 08:58:57 -0800 Xin LI wrote: > On Fri, Nov 10, 2023 at 7:50=E2=80=AFAM Martin Matuska w= rote: >=20 > > Hi Xin, > > > > since when have you been using block cloning on the system? Is it > > possible that there is already corrupted block-cloned data from the >=20 >=20 > That's a good question, I can't 100% rule out this possibility. I was > following -CURRENT in ~weekly to ~monthly on that system, and the pool was > created in March 2014. >=20 > Do you think I should try rebuilding the pool from scratch? I do have > remote backup on a different server but was avoiding it because it's time > consuming. >=20 >=20 > > past? Is everything on one dataset or are you using multiple datasets > > for /usr/src and /usr/obj? > > >=20 > /usr/src and /usr/obj are separate datasets, and the system runs Poudriere > so it have multiple copies of slightly different /usr/src and /usr/obj's. >=20 > Is there a way to identify datasets with block cloning, by the way? Maybe > I should try recreating these datasets first? >=20 >=20 >=20 > > > > Best regards, > > mm > > > > On 10. 11. 2023 8:04, Xin Li wrote: > > > On 2023-11-05 16:34, Martin Matuska wrote: > > >> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can > > >> work with pools that have feature@block_cloning enabled. > > >> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of > > >> zfs_clone_range() which is called by copy_file_range(). When it is > > >> set to 0, zfs_clone_range() does not do block cloning. > > >> If it is set to anything else than 0, zfs_clone_range() does block > > >> cloning (if all conditions are met - same ZFS pool, correct data > > >> alignment, etc.). > > >> > > >> In FreeBSD-main, this tunable is enabled and I plan to enable it in > > >> stable/14 somewhere around December 11, 2023. > > >> > > >> As of today I personally use block cloning on all my systems. > > > > > > I'd like to share a different data point. It still panics on my > > > storage (running -CURRENT about a week ago) when enabled and can be > > > triggered by "make buildworld buildkernel". I wasn't able to capture > > > earlier coredump until the most recent one, which panicked with: > > > > > > > > > cpuid =3D 2 > > > time =3D 1699593456 > > > KDB: stack backtrace: > > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > > 0xfffffe022f2bd7e0 > > > vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910 > > > spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970 > > > dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0 > > > zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > > > zfs_freebsd_copy_file_range() at > > > zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30 > > > vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2b= dce0 > > > kern_copy_file_range() at kern_copy_file_range+0x380/frame > > > 0xfffffe022f2bddb0 > > > sys_copy_file_range() at sys_copy_file_range+0x78/frame > > > 0xfffffe022f2bde00 > > > amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30 > > > fast_syscall_common() at fast_syscall_common+0xf8/frame > > > 0xfffffe022f2bdf30 > > > --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D > > > 0x7fbb2da4ada, rsp =3D 0x7fbb02c5d48, rbp =3D 0x7fbb02c61e0 --- > > > Uptime: 2h32m27s > > > Dumping 7800 out of 32696 > > > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > > > > > #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > > > #1 doadump (textdump=3Dtextdump@entry=3D1) at > > > /usr/src/sys/kern/kern_shutdown.c:405 > > > #2 0xffffffff80694480 in kern_reboot (howto=3D260) at > > > /usr/src/sys/kern/kern_shutdown.c:526 > > > #3 0xffffffff8069497f in vpanic (fmt=3D0xffffffff82603415 "VERIFY3(n= bps > > > =3D=3D numbufs) failed (%llu =3D=3D %llu)\n", ap=3Dap@entry=3D0xfffff= e022f2bd950) > > > at /usr/src/sys/kern/kern_shutdown.c:970 > > > #4 0xffffffff8232999a in spl_panic (file=3D, > > > func=3D, line=3D, fmt=3D) at > > > /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103 > > > #5 0xffffffff823a6605 in dmu_brt_clone > > > (os=3Dos@entry=3D0xfffff800c5ce4000, object=3D, > > > offset=3Doffset@entry=3D0, length=3Dlength@entry=3D207477, > > > tx=3Dtx@entry=3D0xfffff8071a108d00, bps=3Dbps@entry=3D0xfffffe01e218c= 000, > > > nbps=3D2, replay=3D0) > > > at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303 > > > #6 0xffffffff8250f67c in zfs_clone_range (inzp=3D0xfffff804416ac000, > > > inoffp=3D0xfffff800b81cb048, outzp=3D0xfffff806f58f03a0, > > > outoffp=3D0xfffff800b8063048, lenp=3Dlenp@entry=3D0xfffffe022f2bdbf0, > > > cr=3D0xfffff8000a6fe600) > > > at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326 > > > #7 0xffffffff8234b3ba in zfs_freebsd_copy_file_range > > > (ap=3D0xfffffe022f2bdc48) at > > > /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 > > > #8 0xffffffff8079f443 in VOP_COPY_FILE_RANGE > > > (invp=3D0xfffff804416cb1c0, inoffp=3D0xfffff800b81cb048, > > > outvp=3D0xfffff806f51d3380, outoffp=3D0xfffff800b8063048, > > > lenp=3D0xfffffe022f2bdd48, incred=3D0xfffff8000a6fe600, flags=3D > > out>, > > > outcred=3D, fsizetd=3D) at > > > ./vnode_if.h:2385 > > > #9 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff804416cb1c0, > > > inoffp=3Dinoffp@entry=3D0xfffff800b81cb048, > > > outvp=3Doutvp@entry=3D0xfffff806f51d3380, > > > outoffp=3Doutoffp@entry=3D0xfffff800b8063048, > > > lenp=3Dlenp@entry=3D0xfffffe022f2bdd48, flags=3Dflags@entry=3D0, > > > incred=3D0xfffff8000a6fe600, outcred=3D0xfffff8000a6fe600, > > > fsize_td=3D0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > > > #10 0xffffffff8079a070 in kern_copy_file_range > > > (td=3Dtd@entry=3D0xfffffe022925b3a0, infd=3D, > > > inoffp=3D0xfffff800b81cb048, inoffp@entry=3D0x0, outfd=3D, > > > outoffp=3D0xfffff800b8063048, outoffp@entry=3D0x0, len=3D922337203685= 4775807, > > > flags=3D0) at /usr/src/sys/kern/vfs_syscalls.c:4973 > > > #11 0xffffffff8079a178 in sys_copy_file_range (td=3D0xfffffe022925b3a= 0, > > > uap=3D0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011 > > > #12 0xffffffff80a97aa3 in syscallenter (td=3D0xfffffe022925b3a0) at > > > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188 > > > #13 amd64_syscall (td=3D0xfffffe022925b3a0, traced=3D0) at > > > /usr/src/sys/amd64/amd64/trap.c:1194 > > > #14 > > > #15 0x000007fbb2da4ada in ?? () > > > > > > > > > and disabling bclone does appear to allow me to finish buildworld / > > > buildkernel. > > > > > > The pool didn't have redaction_list_spill enabled. > > > > > > The ASSERT3U(nbps, =3D=3D, numbufs); in dmu_brt_clone was added when = block > > > clone is first implemented. > > > > > > It seems that I am the only person who is seeing this as of today. It > > > seems that block clone was indeed being used for some data: > > > > > > saturn bcloneused 1.18M - > > > saturn bclonesaved 1.21M - > > > saturn bcloneratio 2.02x - > > > > > > The pool have dedup enabled for some datasets. > > > > > > Any suggestions? (In extreme cases I can recreate the storage pool > > > from backup or copy the data somewhere else, then recreate the pool, > > > then copy data back, but I'd like to avoid that if possible) > > > > > > Cheers, > > > > > > > > >> > > >> mm > > >> > > >> On 04/11/2023 13:35, Mark Millard wrote: > > >>> On Nov 4, 2023, at 04:38, Mike Karels wrote: > > >>> > > >>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote: > > >>>> > > >>>>> On 11/4/23 02:39, Mark Millard wrote: > > >>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: > > >>>>>> > > >>>>>> int zfs_bclone_enabled; > > >>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, > > >>>>>> &zfs_bclone_enabled, 0, "Enable block cloning"); > > >>>>>> > > >>>>>> leaving block cloning effectively disabled by default, no > > >>>>>> matter what the pool has enabled. > > >>>>>> > > >>>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: > > >>>>>> > > >>>>>> QUOTE > > >>>>>> OpenZFS has been upgraded to version 2.2. New features include: > > >>>>>> =E2=80=A2 > > >>>>>> block cloning, which allows shallow copies of blocks in file > > >>>>>> copies. This is optional, and disabled by default; it can be > > >>>>>> enabled with sysctl vfs.zfs.bclone_enabled=3D1. > > >>>>>> END QUOTE > > >>>>>> > > >>>>> > > >>>>> I think this answers your question in the subject. > > >>>> I think so too (and I wrote that text). > > >>> Thanks for the confirmation of the final intent. > > >>> > > >>> I believe this makes: > > >>> > > >>> QUOTE > > >>> author Brian Behlendorf 2023-05-25 20:53:08 > > >>> +0000 > > >>> committer GitHub 2023-05-25 20:53:08 +0000 > > >>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) > > >>> tree dd01dfce6aeef357ade1775acf18aade535c6271 > > >>> . . . > > >>> Update compatibility.d files > > >>> > > >>> Add an openzfs-2.2 compatibility file for the next release. Edon-R > > >>> support has been enabled for FreeBSD removing the need for different > > >>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd names > > >>> are created for any scripts expecting that convention. Additionally, > > >>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian > > >>> Behlendorf Closes #14833 > > >>> END QUOTE > > >>> > > >>> technically incorrect in that compatibility.d/openzfs-2.2-freebsd > > >>> should be distinct in content from compatibility.d/openzfs-2.2 so > > >>> that block cloning would not be enabled. > > >>> > > >>> > > >>>>>> Just curiousity on my part about the default completeness of > > >>>>>> openzfs-2.2 support, not an objection either way. > > >>>>>> > > >>>>> > > >>>>> I haven't seen new issues with block cloning in the last few weeks > > >>>>> mentioned on the mailing lists. All known issues are fixed AFAIK. > > >>>>> But I can imagine that the risk+effect ratio of data corruption is > > >>>>> seen as a bit too high for a 14.0 release for this particular > > >>>>> feature. That does not diminish the rest of the completeness of > > >>>>> openzfs-2.2. > > >>>>> > > >>>>> NB: I'm not involved in developing openzfs or the decision making > > >>>>> in the release. Just repeating what I read on the lists. > > >>>> There was another block cloning fix in 14.0-RC4; see the commit lo= g. > > >>>> Maybe there will be no more issues, but it seems that corner cases > > >>>> were > > >>>> still being found recently. > > >>> Looks like I'll stay at openzfs-2.1 pool features until there is > > >>> a release that no longer has the default status: > > >>> > > >>> 0 for sysctl vfs.zfs.bclone_enabled > > >>> > > >>> I use main [so: 15 now] but only enable openzfs-2.* pool features > > >>> supported by default on some FreeBSD release, that has an accurate > > >>> compatibility.d/openzfs-2.*-freebsd file. > > >>> > > >>> =3D=3D=3D > > >>> Mark Millard > > >>> marklmi at yahoo.com > > >>> > > >>> > > >> > > > > > > > From nobody Fri Nov 10 22:47:23 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRvGb0Whwz50rFB for ; Fri, 10 Nov 2023 22:52:23 +0000 (UTC) (envelope-from mm@FreeBSD.org) Received: from www541.your-server.de (www541.your-server.de [213.133.107.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRvGZ4svrz3WHf; Fri, 10 Nov 2023 22:52:22 +0000 (UTC) (envelope-from mm@FreeBSD.org) Authentication-Results: mx1.freebsd.org; none Received: from sslproxy03.your-server.de ([88.198.220.132]) by www541.your-server.de with esmtpsa (TLS1.3) tls TLS_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1r1aHc-000Afx-D6; Fri, 10 Nov 2023 23:47:24 +0100 Received: from [188.167.171.2] (helo=[10.0.9.225]) by sslproxy03.your-server.de with esmtpsa (TLSv1.3:TLS_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1r1aHc-0001Ld-46; Fri, 10 Nov 2023 23:47:24 +0100 Content-Type: multipart/alternative; boundary="------------hcDCO5UsWNX1hkTPj6uApmeT" Message-ID: <77ad9593-34a8-48dc-8533-aafb852f1d19@FreeBSD.org> Date: Fri, 10 Nov 2023 23:47:23 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? Content-Language: en-US To: Xin LI Cc: d@delphij.net, FreeBSD-STABLE Mailing List , pjd@freebsd.org References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> From: Martin Matuska In-Reply-To: X-Authenticated-Sender: martin@matuska.de X-Virus-Scanned: Clear (ClamAV 0.103.10/27089/Fri Nov 10 09:39:24 2023) X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:24940, ipnet:213.133.96.0/19, country:DE] X-Rspamd-Queue-Id: 4SRvGZ4svrz3WHf This is a multi-part message in MIME format. --------------hcDCO5UsWNX1hkTPj6uApmeT Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit The issues I had to deal with went away by deleting the problematic files (for good, no snapshot copies left). Deleting a dataset should be even better. On 10. 11. 2023 17:58, Xin LI wrote: > > > On Fri, Nov 10, 2023 at 7:50 AM Martin Matuska wrote: > > Hi Xin, > > since when have you been using block cloning on the system? Is it > possible that there is already corrupted block-cloned data from the > > > That's a good question, I can't 100% rule out this possibility.  I was > following -CURRENT in ~weekly to ~monthly on that system, and the pool > was created in March 2014. > > Do you think I should try rebuilding the pool from scratch?  I do have > remote backup on a different server but was avoiding it because it's > time consuming. > > past? Is everything on one dataset or are you using multiple datasets > for /usr/src and /usr/obj? > > > /usr/src and /usr/obj are separate datasets, and the system runs > Poudriere so it have multiple copies of slightly different /usr/src > and /usr/obj's. > > Is there a way to identify datasets with block cloning, by the way?  > Maybe I should try recreating these datasets first? > > > > Best regards, > mm > > On 10. 11. 2023 8:04, Xin Li wrote: > > On 2023-11-05 16:34, Martin Matuska wrote: > >> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can > >> work with pools that have feature@block_cloning enabled. > >> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of > >> zfs_clone_range() which is called by copy_file_range(). When it is > >> set to 0, zfs_clone_range() does not do block cloning. > >> If it is set to anything else than 0, zfs_clone_range() does block > >> cloning (if all conditions are met - same ZFS pool, correct data > >> alignment, etc.). > >> > >> In FreeBSD-main, this tunable is enabled and I plan to enable > it in > >> stable/14 somewhere around December 11, 2023. > >> > >> As of today I personally use block cloning on all my systems. > > > > I'd like to share a different data point.  It still panics on my > > storage (running -CURRENT about a week ago) when enabled and can be > > triggered by "make buildworld buildkernel".  I wasn't able to > capture > > earlier coredump until the most recent one, which panicked with: > > > > > > cpuid = 2 > > time = 1699593456 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > 0xfffffe022f2bd7e0 > > vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910 > > spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970 > > dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0 > > zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > > zfs_freebsd_copy_file_range() at > > zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30 > > vn_copy_file_range() at vn_copy_file_range+0x163/frame > 0xfffffe022f2bdce0 > > kern_copy_file_range() at kern_copy_file_range+0x380/frame > > 0xfffffe022f2bddb0 > > sys_copy_file_range() at sys_copy_file_range+0x78/frame > > 0xfffffe022f2bde00 > > amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30 > > fast_syscall_common() at fast_syscall_common+0xf8/frame > > 0xfffffe022f2bdf30 > > --- syscall (569, FreeBSD ELF64, copy_file_range), rip = > > 0x7fbb2da4ada, rsp = 0x7fbb02c5d48, rbp = 0x7fbb02c61e0 --- > > Uptime: 2h32m27s > > Dumping 7800 out of 32696 > > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > > > #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > > #1  doadump (textdump=textdump@entry=1) at > > /usr/src/sys/kern/kern_shutdown.c:405 > > #2  0xffffffff80694480 in kern_reboot (howto=260) at > > /usr/src/sys/kern/kern_shutdown.c:526 > > #3  0xffffffff8069497f in vpanic (fmt=0xffffffff82603415 > "VERIFY3(nbps > > == numbufs) failed (%llu == %llu)\n", > ap=ap@entry=0xfffffe022f2bd950) > > at /usr/src/sys/kern/kern_shutdown.c:970 > > #4  0xffffffff8232999a in spl_panic (file=, > > func=, line=, fmt=) at > > /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103 > > #5  0xffffffff823a6605 in dmu_brt_clone > > (os=os@entry=0xfffff800c5ce4000, object=, > > offset=offset@entry=0, length=length@entry=207477, > > tx=tx@entry=0xfffff8071a108d00, bps=bps@entry=0xfffffe01e218c000, > > nbps=2, replay=0) > >     at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303 > > #6  0xffffffff8250f67c in zfs_clone_range (inzp=0xfffff804416ac000, > > inoffp=0xfffff800b81cb048, outzp=0xfffff806f58f03a0, > > outoffp=0xfffff800b8063048, lenp=lenp@entry=0xfffffe022f2bdbf0, > > cr=0xfffff8000a6fe600) > >     at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326 > > #7  0xffffffff8234b3ba in zfs_freebsd_copy_file_range > > (ap=0xfffffe022f2bdc48) at > > > /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 > > #8  0xffffffff8079f443 in VOP_COPY_FILE_RANGE > > (invp=0xfffff804416cb1c0, inoffp=0xfffff800b81cb048, > > outvp=0xfffff806f51d3380, outoffp=0xfffff800b8063048, > > lenp=0xfffffe022f2bdd48, incred=0xfffff8000a6fe600, > flags= > out>, > >     outcred=, fsizetd=) at > > ./vnode_if.h:2385 > > #9  vn_copy_file_range (invp=invp@entry=0xfffff804416cb1c0, > > inoffp=inoffp@entry=0xfffff800b81cb048, > > outvp=outvp@entry=0xfffff806f51d3380, > > outoffp=outoffp@entry=0xfffff800b8063048, > > lenp=lenp@entry=0xfffffe022f2bdd48, flags=flags@entry=0, > >     incred=0xfffff8000a6fe600, outcred=0xfffff8000a6fe600, > > fsize_td=0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > > #10 0xffffffff8079a070 in kern_copy_file_range > > (td=td@entry=0xfffffe022925b3a0, infd=, > > inoffp=0xfffff800b81cb048, inoffp@entry=0x0, outfd=, > > outoffp=0xfffff800b8063048, outoffp@entry=0x0, > len=9223372036854775807, > >     flags=0) at /usr/src/sys/kern/vfs_syscalls.c:4973 > > #11 0xffffffff8079a178 in sys_copy_file_range > (td=0xfffffe022925b3a0, > > uap=0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011 > > #12 0xffffffff80a97aa3 in syscallenter (td=0xfffffe022925b3a0) at > > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188 > > #13 amd64_syscall (td=0xfffffe022925b3a0, traced=0) at > > /usr/src/sys/amd64/amd64/trap.c:1194 > > #14 > > #15 0x000007fbb2da4ada in ?? () > > > > > > and disabling bclone does appear to allow me to finish buildworld / > > buildkernel. > > > > The pool didn't have redaction_list_spill enabled. > > > > The ASSERT3U(nbps, ==, numbufs); in dmu_brt_clone was added when > block > > clone is first implemented. > > > > It seems that I am the only person who is seeing this as of > today.  It > > seems that block clone was indeed being used for some data: > > > > saturn  bcloneused 1.18M                          - > > saturn  bclonesaved 1.21M                          - > > saturn  bcloneratio 2.02x                          - > > > > The pool have dedup enabled for some datasets. > > > > Any suggestions?  (In extreme cases I can recreate the storage pool > > from backup or copy the data somewhere else, then recreate the > pool, > > then copy data back, but I'd like to avoid that if possible) > > > > Cheers, > > > > > >> > >> mm > >> > >> On 04/11/2023 13:35, Mark Millard wrote: > >>> On Nov 4, 2023, at 04:38, Mike Karels wrote: > >>> > >>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote: > >>>> > >>>>> On 11/4/23 02:39, Mark Millard wrote: > >>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: > >>>>>> > >>>>>> int zfs_bclone_enabled; > >>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, > >>>>>> &zfs_bclone_enabled, 0, "Enable block cloning"); > >>>>>> > >>>>>> leaving block cloning effectively disabled by default, no > >>>>>> matter what the pool has enabled. > >>>>>> > >>>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: > >>>>>> > >>>>>> QUOTE > >>>>>> OpenZFS has been upgraded to version 2.2. New features include: > >>>>>> • > >>>>>> block cloning, which allows shallow copies of blocks in file > >>>>>> copies. This is optional, and disabled by default; it can be > >>>>>> enabled with sysctl vfs.zfs.bclone_enabled=1. > >>>>>> END QUOTE > >>>>>> > >>>>> > >>>>> I think this answers your question in the subject. > >>>> I think so too (and I wrote that text). > >>> Thanks for the confirmation of the final intent. > >>> > >>> I believe this makes: > >>> > >>> QUOTE > >>> author Brian Behlendorf 2023-05-25 > 20:53:08 > >>> +0000 > >>> committer GitHub 2023-05-25 20:53:08 +0000 > >>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) > >>> tree dd01dfce6aeef357ade1775acf18aade535c6271 > >>> . . . > >>> Update compatibility.d files > >>> > >>> Add an openzfs-2.2 compatibility file for the next release. > Edon-R > >>> support has been enabled for FreeBSD removing the need for > different > >>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd > names > >>> are created for any scripts expecting that convention. > Additionally, > >>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian > >>> Behlendorf Closes #14833 > >>> END QUOTE > >>> > >>> technically incorrect in that compatibility.d/openzfs-2.2-freebsd > >>> should be distinct in content from compatibility.d/openzfs-2.2 so > >>> that block cloning would not be enabled. > >>> > >>> > >>>>>> Just curiousity on my part about the default completeness of > >>>>>> openzfs-2.2 support, not an objection either way. > >>>>>> > >>>>> > >>>>> I haven't seen new issues with block cloning in the last few > weeks > >>>>> mentioned on the mailing lists. All known issues are fixed > AFAIK. > >>>>> But I can imagine that the risk+effect ratio of data > corruption is > >>>>> seen as a bit too high for a 14.0 release for this particular > >>>>> feature. That does not diminish the rest of the completeness of > >>>>> openzfs-2.2. > >>>>> > >>>>> NB: I'm not involved in developing openzfs or the decision > making > >>>>> in the release. Just repeating what I read on the lists. > >>>> There was another block cloning fix in 14.0-RC4; see the > commit log. > >>>> Maybe there will be no more issues, but it seems that corner > cases > >>>> were > >>>> still being found recently. > >>> Looks like I'll stay at openzfs-2.1 pool features until there is > >>> a release that no longer has the default status: > >>> > >>> 0 for sysctl vfs.zfs.bclone_enabled > >>> > >>> I use main [so: 15 now] but only enable openzfs-2.* pool features > >>> supported by default on some FreeBSD release, that has an accurate > >>> compatibility.d/openzfs-2.*-freebsd file. > >>> > >>> === > >>> Mark Millard > >>> marklmi at yahoo.com > >>> > >>> > >> > > > --------------hcDCO5UsWNX1hkTPj6uApmeT Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit

The issues I had to deal with went away by deleting the problematic files (for good, no snapshot copies left). Deleting a dataset should be even better.

On 10. 11. 2023 17:58, Xin LI wrote:


On Fri, Nov 10, 2023 at 7:50 AM Martin Matuska <mm@freebsd.org> wrote:
Hi Xin,

since when have you been using block cloning on the system? Is it
possible that there is already corrupted block-cloned data from the

That's a good question, I can't 100% rule out this possibility.  I was following -CURRENT in ~weekly to ~monthly on that system, and the pool was created in March 2014.

Do you think I should try rebuilding the pool from scratch?  I do have remote backup on a different server but was avoiding it because it's time consuming.
 
past? Is everything on one dataset or are you using multiple datasets
for /usr/src and /usr/obj?

/usr/src and /usr/obj are separate datasets, and the system runs Poudriere so it have multiple copies of slightly different /usr/src and /usr/obj's.

Is there a way to identify datasets with block cloning, by the way?  Maybe I should try recreating these datasets first?

 

Best regards,
mm

On 10. 11. 2023 8:04, Xin Li wrote:
> On 2023-11-05 16:34, Martin Matuska wrote:
>> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can
>> work with pools that have feature@block_cloning enabled.
>> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of
>> zfs_clone_range() which is called by copy_file_range(). When it is
>> set to 0, zfs_clone_range() does not do block cloning.
>> If it is set to anything else than 0, zfs_clone_range() does block
>> cloning (if all conditions are met - same ZFS pool, correct data
>> alignment, etc.).
>>
>> In FreeBSD-main, this tunable is enabled and I plan to enable it in
>> stable/14 somewhere around December 11, 2023.
>>
>> As of today I personally use block cloning on all my systems.
>
> I'd like to share a different data point.  It still panics on my
> storage (running -CURRENT about a week ago) when enabled and can be
> triggered by "make buildworld buildkernel".  I wasn't able to capture
> earlier coredump until the most recent one, which panicked with:
>
>
> cpuid = 2
> time = 1699593456
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfffffe022f2bd7e0
> vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910
> spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970
> dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0
> zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0
> zfs_freebsd_copy_file_range() at
> zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30
> vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bdce0
> kern_copy_file_range() at kern_copy_file_range+0x380/frame
> 0xfffffe022f2bddb0
> sys_copy_file_range() at sys_copy_file_range+0x78/frame
> 0xfffffe022f2bde00
> amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30
> fast_syscall_common() at fast_syscall_common+0xf8/frame
> 0xfffffe022f2bdf30
> --- syscall (569, FreeBSD ELF64, copy_file_range), rip =
> 0x7fbb2da4ada, rsp = 0x7fbb02c5d48, rbp = 0x7fbb02c61e0 ---
> Uptime: 2h32m27s
> Dumping 7800 out of 32696
> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>
> #0  __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57
> #1  doadump (textdump=textdump@entry=1) at
> /usr/src/sys/kern/kern_shutdown.c:405
> #2  0xffffffff80694480 in kern_reboot (howto=260) at
> /usr/src/sys/kern/kern_shutdown.c:526
> #3  0xffffffff8069497f in vpanic (fmt=0xffffffff82603415 "VERIFY3(nbps
> == numbufs) failed (%llu == %llu)\n", ap=ap@entry=0xfffffe022f2bd950)
> at /usr/src/sys/kern/kern_shutdown.c:970
> #4  0xffffffff8232999a in spl_panic (file=<optimized out>,
> func=<optimized out>, line=<unavailable>, fmt=<unavailable>) at
> /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103
> #5  0xffffffff823a6605 in dmu_brt_clone
> (os=os@entry=0xfffff800c5ce4000, object=<optimized out>,
> offset=offset@entry=0, length=length@entry=207477,
> tx=tx@entry=0xfffff8071a108d00, bps=bps@entry=0xfffffe01e218c000,
> nbps=2, replay=0)
>     at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303
> #6  0xffffffff8250f67c in zfs_clone_range (inzp=0xfffff804416ac000,
> inoffp=0xfffff800b81cb048, outzp=0xfffff806f58f03a0,
> outoffp=0xfffff800b8063048, lenp=lenp@entry=0xfffffe022f2bdbf0,
> cr=0xfffff8000a6fe600)
>     at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326
> #7  0xffffffff8234b3ba in zfs_freebsd_copy_file_range
> (ap=0xfffffe022f2bdc48) at
> /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294
> #8  0xffffffff8079f443 in VOP_COPY_FILE_RANGE
> (invp=0xfffff804416cb1c0, inoffp=0xfffff800b81cb048,
> outvp=0xfffff806f51d3380, outoffp=0xfffff800b8063048,
> lenp=0xfffffe022f2bdd48, incred=0xfffff8000a6fe600, flags=<optimized
> out>,
>     outcred=<optimized out>, fsizetd=<optimized out>) at
> ./vnode_if.h:2385
> #9  vn_copy_file_range (invp=invp@entry=0xfffff804416cb1c0,
> inoffp=inoffp@entry=0xfffff800b81cb048,
> outvp=outvp@entry=0xfffff806f51d3380,
> outoffp=outoffp@entry=0xfffff800b8063048,
> lenp=lenp@entry=0xfffffe022f2bdd48, flags=flags@entry=0,
>     incred=0xfffff8000a6fe600, outcred=0xfffff8000a6fe600,
> fsize_td=0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087
> #10 0xffffffff8079a070 in kern_copy_file_range
> (td=td@entry=0xfffffe022925b3a0, infd=<optimized out>,
> inoffp=0xfffff800b81cb048, inoffp@entry=0x0, outfd=<optimized out>,
> outoffp=0xfffff800b8063048, outoffp@entry=0x0, len=9223372036854775807,
>     flags=0) at /usr/src/sys/kern/vfs_syscalls.c:4973
> #11 0xffffffff8079a178 in sys_copy_file_range (td=0xfffffe022925b3a0,
> uap=0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011
> #12 0xffffffff80a97aa3 in syscallenter (td=0xfffffe022925b3a0) at
> /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188
> #13 amd64_syscall (td=0xfffffe022925b3a0, traced=0) at
> /usr/src/sys/amd64/amd64/trap.c:1194
> #14 <signal handler called>
> #15 0x000007fbb2da4ada in ?? ()
>
>
> and disabling bclone does appear to allow me to finish buildworld /
> buildkernel.
>
> The pool didn't have redaction_list_spill enabled.
>
> The ASSERT3U(nbps, ==, numbufs); in dmu_brt_clone was added when block
> clone is first implemented.
>
> It seems that I am the only person who is seeing this as of today.  It
> seems that block clone was indeed being used for some data:
>
> saturn  bcloneused 1.18M                          -
> saturn  bclonesaved 1.21M                          -
> saturn  bcloneratio 2.02x                          -
>
> The pool have dedup enabled for some datasets.
>
> Any suggestions?  (In extreme cases I can recreate the storage pool
> from backup or copy the data somewhere else, then recreate the pool,
> then copy data back, but I'd like to avoid that if possible)
>
> Cheers,
>
>
>>
>> mm
>>
>> On 04/11/2023 13:35, Mark Millard wrote:
>>> On Nov 4, 2023, at 04:38, Mike Karels <mike@karels.net> wrote:
>>>
>>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote:
>>>>
>>>>> On 11/4/23 02:39, Mark Millard wrote:
>>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has:
>>>>>>
>>>>>> int zfs_bclone_enabled;
>>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN,
>>>>>> &zfs_bclone_enabled, 0, "Enable block cloning");
>>>>>>
>>>>>> leaving block cloning effectively disabled by default, no
>>>>>> matter what the pool has enabled.
>>>>>>
>>>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports:
>>>>>>
>>>>>> QUOTE
>>>>>> OpenZFS has been upgraded to version 2.2. New features include:
>>>>>> •
>>>>>> block cloning, which allows shallow copies of blocks in file
>>>>>> copies. This is optional, and disabled by default; it can be
>>>>>> enabled with sysctl vfs.zfs.bclone_enabled=1.
>>>>>> END QUOTE
>>>>>>
>>>>>
>>>>> I think this answers your question in the subject.
>>>> I think so too (and I wrote that text).
>>> Thanks for the confirmation of the final intent.
>>>
>>> I believe this makes:
>>>
>>> QUOTE
>>> author Brian Behlendorf <behlendorf1@llnl.gov> 2023-05-25 20:53:08
>>> +0000
>>> committer GitHub <noreply@github.com> 2023-05-25 20:53:08 +0000
>>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch)
>>> tree dd01dfce6aeef357ade1775acf18aade535c6271
>>> . . .
>>> Update compatibility.d files
>>>
>>> Add an openzfs-2.2 compatibility file for the next release. Edon-R
>>> support has been enabled for FreeBSD removing the need for different
>>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd names
>>> are created for any scripts expecting that convention. Additionally,
>>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian
>>> Behlendorf <behlendorf1@llnl.gov> Closes #14833
>>> END QUOTE
>>>
>>> technically incorrect in that compatibility.d/openzfs-2.2-freebsd
>>> should be distinct in content from compatibility.d/openzfs-2.2 so
>>> that block cloning would not be enabled.
>>>
>>>
>>>>>> Just curiousity on my part about the default completeness of
>>>>>> openzfs-2.2 support, not an objection either way.
>>>>>>
>>>>>
>>>>> I haven't seen new issues with block cloning in the last few weeks
>>>>> mentioned on the mailing lists. All known issues are fixed AFAIK.
>>>>> But I can imagine that the risk+effect ratio of data corruption is
>>>>> seen as a bit too high for a 14.0 release for this particular
>>>>> feature. That does not diminish the rest of the completeness of
>>>>> openzfs-2.2.
>>>>>
>>>>> NB: I'm not involved in developing openzfs or the decision making
>>>>> in the release. Just repeating what I read on the lists.
>>>> There was another block cloning fix in 14.0-RC4; see the commit log.
>>>> Maybe there will be no more issues, but it seems that corner cases
>>>> were
>>>> still being found recently.
>>> Looks like I'll stay at openzfs-2.1 pool features until there is
>>> a release that no longer has the default status:
>>>
>>> 0 for sysctl vfs.zfs.bclone_enabled
>>>
>>> I use main [so: 15 now] but only enable openzfs-2.* pool features
>>> supported by default on some FreeBSD release, that has an accurate
>>> compatibility.d/openzfs-2.*-freebsd file.
>>>
>>> ===
>>> Mark Millard
>>> marklmi at yahoo.com
>>>
>>>
>>
>

--------------hcDCO5UsWNX1hkTPj6uApmeT--