From nobody Fri Nov 10 16:58:57 2023 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4SRlR5167Nz509S5 for ; Fri, 10 Nov 2023 16:59:13 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4SRlR45TbXz3Zgx; Fri, 10 Nov 2023 16:59:12 +0000 (UTC) (envelope-from delphij@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-ej1-x630.google.com with SMTP id a640c23a62f3a-9e2838bcb5eso373508566b.0; Fri, 10 Nov 2023 08:59:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1699635549; x=1700240349; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=NE7A7tPmDKUuN515nzcLndWyjcJPIIY1RfzXoKXy1mo=; b=JEijXXcQIjNWg9IOYAD3/ZgIGSIliYmx+XnDzP4shiacEIz4XZ3c8pyu4EkaFBg7dV pCIqG3nVKyw712vtf5Z2UAIqOEN4JeuUtH6byrw5YaQ0ADta8b9Zj8poXKTGrMnW0fhB ubuTMzvQ3PzjYPtKNVWxwVtjK5MJBCZvYpCfC57A5xxWyokP6POE5PY6ttH4jHL8+j3J CnWk10OQWHq8U1Si/tyM0j9MRMPzaBNaEcgQFVUtEhxjypI2oCyWRrtbxP8Ar989fGOi B6Dfn3C+nhIZfSxzH7LlFeLU8WYgde0YPkuCXgCvqNEgFnHof9QOljgSzA0sZLMGPMLR AtMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1699635549; x=1700240349; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=NE7A7tPmDKUuN515nzcLndWyjcJPIIY1RfzXoKXy1mo=; b=SJ6zmF4Kqm6b9c9NaQJkIzzyrNrh7X3Df4g/8LyU63Bw/FjfQOXucDFb/IStOLx8nt uyJjXQE+gWHvpzyvyDCBr8qJzwjYSzGwFj6yj+qLVFjcXIazJXJTbLgovLdg1vpfDkI8 AQyr/7aHrBL9Kk/sUvLKpyeNjQCFq0YCEJ7ax9ZcyrLup4QzHozvs6whJCqHQgsXBuDG B+bMAwfuH5oo8PjgePUqEGPj63I2vn9jV5zlb/X4lXeKPUrtDcDwBozfA/aao7kLPn0+ DUq7cuz7Wqq38t3NtZNIzXbT3AKOUYKls8+vKoWibVlVGdXFMzXMBNUpxPIghqJKlKNN 0smg== X-Gm-Message-State: AOJu0YxeYjj8vALZPVkIJx2WG0pztX0D9Wv/j7xGhp7ZxytpAkvlC92D RIqezfAdHfv1CZyDKVAi9NQh0gKZi6ofrlAKQ6yOsUfAMOI= X-Google-Smtp-Source: AGHT+IH9sOK0RhBmXASZkFIjpvpzEq15rIep8lAtlSqXIHKMwGu0MG4thCgote90F1Q07FACzOoOC6ETMSHCYq0UC48= X-Received: by 2002:a17:907:d8e:b0:9ae:6a08:6f53 with SMTP id go14-20020a1709070d8e00b009ae6a086f53mr7671054ejc.63.1699635548820; Fri, 10 Nov 2023 08:59:08 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29.ref@yahoo.com> <2F81D978-7DBD-42CE-8ECF-C020B0CB5C29@yahoo.com> <7a906956-6836-421e-b25e-ff701369e3ed@FreeBSD.org> <830CD3A8-DB62-418D-A7F7-8DA6CB46B1F5@yahoo.com> <05b493bc-94a5-4c78-bebf-5581addc5b7b@FreeBSD.org> <47c5b902-eea6-4194-b84a-99a6343f6bd0@delphij.net> In-Reply-To: From: Xin LI Date: Fri, 10 Nov 2023 08:58:57 -0800 Message-ID: Subject: Re: Is 14.0 to released based on 0 for sysctl vfs.zfs.bclone_enabled ? To: Martin Matuska Cc: d@delphij.net, FreeBSD-STABLE Mailing List , pjd@freebsd.org Content-Type: multipart/alternative; boundary="0000000000002297970609cf3dc8" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4SRlR45TbXz3Zgx --0000000000002297970609cf3dc8 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Fri, Nov 10, 2023 at 7:50=E2=80=AFAM Martin Matuska wro= te: > Hi Xin, > > since when have you been using block cloning on the system? Is it > possible that there is already corrupted block-cloned data from the That's a good question, I can't 100% rule out this possibility. I was following -CURRENT in ~weekly to ~monthly on that system, and the pool was created in March 2014. Do you think I should try rebuilding the pool from scratch? I do have remote backup on a different server but was avoiding it because it's time consuming. > past? Is everything on one dataset or are you using multiple datasets > for /usr/src and /usr/obj? > /usr/src and /usr/obj are separate datasets, and the system runs Poudriere so it have multiple copies of slightly different /usr/src and /usr/obj's. Is there a way to identify datasets with block cloning, by the way? Maybe I should try recreating these datasets first? > > Best regards, > mm > > On 10. 11. 2023 8:04, Xin Li wrote: > > On 2023-11-05 16:34, Martin Matuska wrote: > >> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can > >> work with pools that have feature@block_cloning enabled. > >> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of > >> zfs_clone_range() which is called by copy_file_range(). When it is > >> set to 0, zfs_clone_range() does not do block cloning. > >> If it is set to anything else than 0, zfs_clone_range() does block > >> cloning (if all conditions are met - same ZFS pool, correct data > >> alignment, etc.). > >> > >> In FreeBSD-main, this tunable is enabled and I plan to enable it in > >> stable/14 somewhere around December 11, 2023. > >> > >> As of today I personally use block cloning on all my systems. > > > > I'd like to share a different data point. It still panics on my > > storage (running -CURRENT about a week ago) when enabled and can be > > triggered by "make buildworld buildkernel". I wasn't able to capture > > earlier coredump until the most recent one, which panicked with: > > > > > > cpuid =3D 2 > > time =3D 1699593456 > > KDB: stack backtrace: > > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > > 0xfffffe022f2bd7e0 > > vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910 > > spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970 > > dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0 > > zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > > zfs_freebsd_copy_file_range() at > > zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30 > > vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bdc= e0 > > kern_copy_file_range() at kern_copy_file_range+0x380/frame > > 0xfffffe022f2bddb0 > > sys_copy_file_range() at sys_copy_file_range+0x78/frame > > 0xfffffe022f2bde00 > > amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30 > > fast_syscall_common() at fast_syscall_common+0xf8/frame > > 0xfffffe022f2bdf30 > > --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D > > 0x7fbb2da4ada, rsp =3D 0x7fbb02c5d48, rbp =3D 0x7fbb02c61e0 --- > > Uptime: 2h32m27s > > Dumping 7800 out of 32696 > > MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91% > > > > #0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > > #1 doadump (textdump=3Dtextdump@entry=3D1) at > > /usr/src/sys/kern/kern_shutdown.c:405 > > #2 0xffffffff80694480 in kern_reboot (howto=3D260) at > > /usr/src/sys/kern/kern_shutdown.c:526 > > #3 0xffffffff8069497f in vpanic (fmt=3D0xffffffff82603415 "VERIFY3(nbp= s > > =3D=3D numbufs) failed (%llu =3D=3D %llu)\n", ap=3Dap@entry=3D0xfffffe0= 22f2bd950) > > at /usr/src/sys/kern/kern_shutdown.c:970 > > #4 0xffffffff8232999a in spl_panic (file=3D, > > func=3D, line=3D, fmt=3D) at > > /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103 > > #5 0xffffffff823a6605 in dmu_brt_clone > > (os=3Dos@entry=3D0xfffff800c5ce4000, object=3D, > > offset=3Doffset@entry=3D0, length=3Dlength@entry=3D207477, > > tx=3Dtx@entry=3D0xfffff8071a108d00, bps=3Dbps@entry=3D0xfffffe01e218c00= 0, > > nbps=3D2, replay=3D0) > > at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:2303 > > #6 0xffffffff8250f67c in zfs_clone_range (inzp=3D0xfffff804416ac000, > > inoffp=3D0xfffff800b81cb048, outzp=3D0xfffff806f58f03a0, > > outoffp=3D0xfffff800b8063048, lenp=3Dlenp@entry=3D0xfffffe022f2bdbf0, > > cr=3D0xfffff8000a6fe600) > > at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnops.c:1326 > > #7 0xffffffff8234b3ba in zfs_freebsd_copy_file_range > > (ap=3D0xfffffe022f2bdc48) at > > /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294 > > #8 0xffffffff8079f443 in VOP_COPY_FILE_RANGE > > (invp=3D0xfffff804416cb1c0, inoffp=3D0xfffff800b81cb048, > > outvp=3D0xfffff806f51d3380, outoffp=3D0xfffff800b8063048, > > lenp=3D0xfffffe022f2bdd48, incred=3D0xfffff8000a6fe600, flags=3D > out>, > > outcred=3D, fsizetd=3D) at > > ./vnode_if.h:2385 > > #9 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff804416cb1c0, > > inoffp=3Dinoffp@entry=3D0xfffff800b81cb048, > > outvp=3Doutvp@entry=3D0xfffff806f51d3380, > > outoffp=3Doutoffp@entry=3D0xfffff800b8063048, > > lenp=3Dlenp@entry=3D0xfffffe022f2bdd48, flags=3Dflags@entry=3D0, > > incred=3D0xfffff8000a6fe600, outcred=3D0xfffff8000a6fe600, > > fsize_td=3D0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > > #10 0xffffffff8079a070 in kern_copy_file_range > > (td=3Dtd@entry=3D0xfffffe022925b3a0, infd=3D, > > inoffp=3D0xfffff800b81cb048, inoffp@entry=3D0x0, outfd=3D, > > outoffp=3D0xfffff800b8063048, outoffp@entry=3D0x0, len=3D92233720368547= 75807, > > flags=3D0) at /usr/src/sys/kern/vfs_syscalls.c:4973 > > #11 0xffffffff8079a178 in sys_copy_file_range (td=3D0xfffffe022925b3a0, > > uap=3D0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011 > > #12 0xffffffff80a97aa3 in syscallenter (td=3D0xfffffe022925b3a0) at > > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188 > > #13 amd64_syscall (td=3D0xfffffe022925b3a0, traced=3D0) at > > /usr/src/sys/amd64/amd64/trap.c:1194 > > #14 > > #15 0x000007fbb2da4ada in ?? () > > > > > > and disabling bclone does appear to allow me to finish buildworld / > > buildkernel. > > > > The pool didn't have redaction_list_spill enabled. > > > > The ASSERT3U(nbps, =3D=3D, numbufs); in dmu_brt_clone was added when bl= ock > > clone is first implemented. > > > > It seems that I am the only person who is seeing this as of today. It > > seems that block clone was indeed being used for some data: > > > > saturn bcloneused 1.18M - > > saturn bclonesaved 1.21M - > > saturn bcloneratio 2.02x - > > > > The pool have dedup enabled for some datasets. > > > > Any suggestions? (In extreme cases I can recreate the storage pool > > from backup or copy the data somewhere else, then recreate the pool, > > then copy data back, but I'd like to avoid that if possible) > > > > Cheers, > > > > > >> > >> mm > >> > >> On 04/11/2023 13:35, Mark Millard wrote: > >>> On Nov 4, 2023, at 04:38, Mike Karels wrote: > >>> > >>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote: > >>>> > >>>>> On 11/4/23 02:39, Mark Millard wrote: > >>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) still has: > >>>>>> > >>>>>> int zfs_bclone_enabled; > >>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTLFLAG_RWTUN, > >>>>>> &zfs_bclone_enabled, 0, "Enable block cloning"); > >>>>>> > >>>>>> leaving block cloning effectively disabled by default, no > >>>>>> matter what the pool has enabled. > >>>>>> > >>>>>> https://www.freebsd.org/releases/14.0R/relnotes/ also reports: > >>>>>> > >>>>>> QUOTE > >>>>>> OpenZFS has been upgraded to version 2.2. New features include: > >>>>>> =E2=80=A2 > >>>>>> block cloning, which allows shallow copies of blocks in file > >>>>>> copies. This is optional, and disabled by default; it can be > >>>>>> enabled with sysctl vfs.zfs.bclone_enabled=3D1. > >>>>>> END QUOTE > >>>>>> > >>>>> > >>>>> I think this answers your question in the subject. > >>>> I think so too (and I wrote that text). > >>> Thanks for the confirmation of the final intent. > >>> > >>> I believe this makes: > >>> > >>> QUOTE > >>> author Brian Behlendorf 2023-05-25 20:53:08 > >>> +0000 > >>> committer GitHub 2023-05-25 20:53:08 +0000 > >>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch) > >>> tree dd01dfce6aeef357ade1775acf18aade535c6271 > >>> . . . > >>> Update compatibility.d files > >>> > >>> Add an openzfs-2.2 compatibility file for the next release. Edon-R > >>> support has been enabled for FreeBSD removing the need for different > >>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd names > >>> are created for any scripts expecting that convention. Additionally, > >>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian > >>> Behlendorf Closes #14833 > >>> END QUOTE > >>> > >>> technically incorrect in that compatibility.d/openzfs-2.2-freebsd > >>> should be distinct in content from compatibility.d/openzfs-2.2 so > >>> that block cloning would not be enabled. > >>> > >>> > >>>>>> Just curiousity on my part about the default completeness of > >>>>>> openzfs-2.2 support, not an objection either way. > >>>>>> > >>>>> > >>>>> I haven't seen new issues with block cloning in the last few weeks > >>>>> mentioned on the mailing lists. All known issues are fixed AFAIK. > >>>>> But I can imagine that the risk+effect ratio of data corruption is > >>>>> seen as a bit too high for a 14.0 release for this particular > >>>>> feature. That does not diminish the rest of the completeness of > >>>>> openzfs-2.2. > >>>>> > >>>>> NB: I'm not involved in developing openzfs or the decision making > >>>>> in the release. Just repeating what I read on the lists. > >>>> There was another block cloning fix in 14.0-RC4; see the commit log. > >>>> Maybe there will be no more issues, but it seems that corner cases > >>>> were > >>>> still being found recently. > >>> Looks like I'll stay at openzfs-2.1 pool features until there is > >>> a release that no longer has the default status: > >>> > >>> 0 for sysctl vfs.zfs.bclone_enabled > >>> > >>> I use main [so: 15 now] but only enable openzfs-2.* pool features > >>> supported by default on some FreeBSD release, that has an accurate > >>> compatibility.d/openzfs-2.*-freebsd file. > >>> > >>> =3D=3D=3D > >>> Mark Millard > >>> marklmi at yahoo.com > >>> > >>> > >> > > > > --0000000000002297970609cf3dc8 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Fri, Nov 10, 2023 at 7:50=E2=80= =AFAM Martin Matuska <mm@freebsd.org> wrote:
Hi= Xin,

since when have you been using block cloning on the system? Is it
possible that there is already corrupted block-cloned data from the

That's a good question, I can't 100% rule out = this possibility.=C2=A0 I was following -CURRENT in ~weekly to ~monthly on = that system, and the pool was created in March 2014.

Do you t= hink I should try rebuilding the pool from scratch?=C2=A0 I do have remote = backup on a different server but was avoiding it because it's time cons= uming.
past? Is everything on one dataset or are you using multiple datasets <= br> for /usr/src and /usr/obj?


Best regards,
mm

On 10. 11. 2023 8:04, Xin Li wrote:
> On 2023-11-05 16:34, Martin Matuska wrote:
>> OpenZFS 2.2.0 in FreeBSD 14 fully supports block cloning. You can =
>> work with pools that have feature@block_cloning enabled.
>> The sysctl variable vfs.zfs.bclone_enabled affects the behavior of=
>> zfs_clone_range() which is called by copy_file_range(). When it is=
>> set to 0, zfs_clone_range() does not do block cloning.
>> If it is set to anything else than 0, zfs_clone_range() does block=
>> cloning (if all conditions are met - same ZFS pool, correct data <= br> >> alignment, etc.).
>>
>> In FreeBSD-main, this tunable is enabled and I plan to enable it i= n
>> stable/14 somewhere around December 11, 2023.
>>
>> As of today I personally use block cloning on all my systems.
>
> I'd like to share a different data point.=C2=A0 It still panics on= my
> storage (running -CURRENT about a week ago) when enabled and can be > triggered by "make buildworld buildkernel".=C2=A0 I wasn'= ;t able to capture
> earlier coredump until the most recent one, which panicked with:
>
>
> cpuid =3D 2
> time =3D 1699593456
> KDB: stack backtrace:
> db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame
> 0xfffffe022f2bd7e0
> vpanic() at vpanic+0x132/frame 0xfffffe022f2bd910
> spl_panic() at spl_panic+0x3a/frame 0xfffffe022f2bd970
> dmu_brt_clone() at dmu_brt_clone+0x555/frame 0xfffffe022f2bd9e0
> zfs_clone_range() at zfs_clone_range+0xa4c/frame 0xfffffe022f2bdbb0 > zfs_freebsd_copy_file_range() at
> zfs_freebsd_copy_file_range+0x18a/frame 0xfffffe022f2bdc30
> vn_copy_file_range() at vn_copy_file_range+0x163/frame 0xfffffe022f2bd= ce0
> kern_copy_file_range() at kern_copy_file_range+0x380/frame
> 0xfffffe022f2bddb0
> sys_copy_file_range() at sys_copy_file_range+0x78/frame
> 0xfffffe022f2bde00
> amd64_syscall() at amd64_syscall+0x153/frame 0xfffffe022f2bdf30
> fast_syscall_common() at fast_syscall_common+0xf8/frame
> 0xfffffe022f2bdf30
> --- syscall (569, FreeBSD ELF64, copy_file_range), rip =3D
> 0x7fbb2da4ada, rsp =3D 0x7fbb02c5d48, rbp =3D 0x7fbb02c61e0 ---
> Uptime: 2h32m27s
> Dumping 7800 out of 32696
> MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
>
> #0=C2=A0 __curthread () at /usr/src/sys/amd64/include/pcpu_aux.h:57 > #1=C2=A0 doadump (textdump=3Dtextdump@entry=3D1) at
> /usr/src/sys/kern/kern_shutdown.c:405
> #2=C2=A0 0xffffffff80694480 in kern_reboot (howto=3D260) at
> /usr/src/sys/kern/kern_shutdown.c:526
> #3=C2=A0 0xffffffff8069497f in vpanic (fmt=3D0xffffffff82603415 "= VERIFY3(nbps
> =3D=3D numbufs) failed (%llu =3D=3D %llu)\n", ap=3Dap@entry=3D0xf= ffffe022f2bd950)
> at /usr/src/sys/kern/kern_shutdown.c:970
> #4=C2=A0 0xffffffff8232999a in spl_panic (file=3D<optimized out>= ,
> func=3D<optimized out>, line=3D<unavailable>, fmt=3D<un= available>) at
> /usr/src/sys/contrib/openzfs/module/os/freebsd/spl/spl_misc.c:103
> #5=C2=A0 0xffffffff823a6605 in dmu_brt_clone
> (os=3Dos@entry=3D0xfffff800c5ce4000, object=3D<optimized out>, <= br> > offset=3Doffset@entry=3D0, length=3Dlength@entry=3D207477,
> tx=3Dtx@entry=3D0xfffff8071a108d00, bps=3Dbps@entry=3D0xfffffe01e218c0= 00,
> nbps=3D2, replay=3D0)
> =C2=A0=C2=A0=C2=A0 at /usr/src/sys/contrib/openzfs/module/zfs/dmu.c:23= 03
> #6=C2=A0 0xffffffff8250f67c in zfs_clone_range (inzp=3D0xfffff804416ac= 000,
> inoffp=3D0xfffff800b81cb048, outzp=3D0xfffff806f58f03a0,
> outoffp=3D0xfffff800b8063048, lenp=3Dlenp@entry=3D0xfffffe022f2bdbf0, =
> cr=3D0xfffff8000a6fe600)
> =C2=A0=C2=A0=C2=A0 at /usr/src/sys/contrib/openzfs/module/zfs/zfs_vnop= s.c:1326
> #7=C2=A0 0xffffffff8234b3ba in zfs_freebsd_copy_file_range
> (ap=3D0xfffffe022f2bdc48) at
> /usr/src/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c:6294=
> #8=C2=A0 0xffffffff8079f443 in VOP_COPY_FILE_RANGE
> (invp=3D0xfffff804416cb1c0, inoffp=3D0xfffff800b81cb048,
> outvp=3D0xfffff806f51d3380, outoffp=3D0xfffff800b8063048,
> lenp=3D0xfffffe022f2bdd48, incred=3D0xfffff8000a6fe600, flags=3D<op= timized
> out>,
> =C2=A0=C2=A0=C2=A0 outcred=3D<optimized out>, fsizetd=3D<opti= mized out>) at
> ./vnode_if.h:2385
> #9=C2=A0 vn_copy_file_range (invp=3Dinvp@entry=3D0xfffff804416cb1c0, <= br> > inoffp=3Dinoffp@entry=3D0xfffff800b81cb048,
> outvp=3Doutvp@entry=3D0xfffff806f51d3380,
> outoffp=3Doutoffp@entry=3D0xfffff800b8063048,
> lenp=3Dlenp@entry=3D0xfffffe022f2bdd48, flags=3Dflags@entry=3D0,
> =C2=A0=C2=A0=C2=A0 incred=3D0xfffff8000a6fe600, outcred=3D0xfffff8000a= 6fe600,
> fsize_td=3D0xfffffe022925b3a0) at /usr/src/sys/kern/vfs_vnops.c:3087 > #10 0xffffffff8079a070 in kern_copy_file_range
> (td=3Dtd@entry=3D0xfffffe022925b3a0, infd=3D<optimized out>, > inoffp=3D0xfffff800b81cb048, inoffp@entry=3D0x0, outfd=3D<optimized= out>,
> outoffp=3D0xfffff800b8063048, outoffp@entry=3D0x0, len=3D9223372036854= 775807,
> =C2=A0=C2=A0=C2=A0 flags=3D0) at /usr/src/sys/kern/vfs_syscalls.c:4973=
> #11 0xffffffff8079a178 in sys_copy_file_range (td=3D0xfffffe022925b3a0= ,
> uap=3D0xfffffe022925b7a0) at /usr/src/sys/kern/vfs_syscalls.c:5011
> #12 0xffffffff80a97aa3 in syscallenter (td=3D0xfffffe022925b3a0) at > /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:188
> #13 amd64_syscall (td=3D0xfffffe022925b3a0, traced=3D0) at
> /usr/src/sys/amd64/amd64/trap.c:1194
> #14 <signal handler called>
> #15 0x000007fbb2da4ada in ?? ()
>
>
> and disabling bclone does appear to allow me to finish buildworld / > buildkernel.
>
> The pool didn't have redaction_list_spill enabled.
>
> The ASSERT3U(nbps, =3D=3D, numbufs); in dmu_brt_clone was added when b= lock
> clone is first implemented.
>
> It seems that I am the only person who is seeing this as of today.=C2= =A0 It
> seems that block clone was indeed being used for some data:
>
> saturn=C2=A0 bcloneused 1.18M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -
> saturn=C2=A0 bclonesaved 1.21M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -
> saturn=C2=A0 bcloneratio 2.02x=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 -
>
> The pool have dedup enabled for some datasets.
>
> Any suggestions?=C2=A0 (In extreme cases I can recreate the storage po= ol
> from backup or copy the data somewhere else, then recreate the pool, <= br> > then copy data back, but I'd like to avoid that if possible)
>
> Cheers,
>
>
>>
>> mm
>>
>> On 04/11/2023 13:35, Mark Millard wrote:
>>> On Nov 4, 2023, at 04:38, Mike Karels <mike@karels.net> wrote:
>>>
>>>> On 4 Nov 2023, at 4:01, Ronald Klop wrote:
>>>>
>>>>> On 11/4/23 02:39, Mark Millard wrote:
>>>>>> It looks to me like releng/14.0 (as of 14.0-RC4) s= till has:
>>>>>>
>>>>>> int zfs_bclone_enabled;
>>>>>> SYSCTL_INT(_vfs_zfs, OID_AUTO, bclone_enabled, CTL= FLAG_RWTUN,
>>>>>> &zfs_bclone_enabled, 0, "Enable block clo= ning");
>>>>>>
>>>>>> leaving block cloning effectively disabled by defa= ult, no
>>>>>> matter what the pool has enabled.
>>>>>>
>>>>>> https://www.freebsd.org/rel= eases/14.0R/relnotes/ also reports:
>>>>>>
>>>>>> QUOTE
>>>>>> OpenZFS has been upgraded to version 2.2. New feat= ures include:
>>>>>> =E2=80=A2
>>>>>> block cloning, which allows shallow copies of bloc= ks in file
>>>>>> copies. This is optional, and disabled by default;= it can be
>>>>>> enabled with sysctl vfs.zfs.bclone_enabled=3D1. >>>>>> END QUOTE
>>>>>>
>>>>>
>>>>> I think this answers your question in the subject.
>>>> I think so too (and I wrote that text).
>>> Thanks for the confirmation of the final intent.
>>>
>>> I believe this makes:
>>>
>>> QUOTE
>>> author Brian Behlendorf <behlendorf1@llnl.gov> 2023-05-25 20:53:08 >>> +0000
>>> committer GitHub <noreply@github.com> 2023-05-25 20:53:08 +0000
>>> commit 91a2325c4a0fbe01d0bf212e44fa9d85017837ce (patch)
>>> tree dd01dfce6aeef357ade1775acf18aade535c6271
>>> . . .
>>> Update compatibility.d files
>>>
>>> Add an openzfs-2.2 compatibility file for the next release. Ed= on-R
>>> support has been enabled for FreeBSD removing the need for dif= ferent
>>> FreeBSD and Linux files. Symlinks for the -linux and -freebsd = names
>>> are created for any scripts expecting that convention. Additio= nally,
>>> a symlink for ubunutu-22.04 was added. Signed-off-by: Brian >>> Behlendorf <behlendorf1@llnl.gov> Closes #14833
>>> END QUOTE
>>>
>>> technically incorrect in that compatibility.d/openzfs-2.2-free= bsd
>>> should be distinct in content from compatibility.d/openzfs-2.2= so
>>> that block cloning would not be enabled.
>>>
>>>
>>>>>> Just curiousity on my part about the default compl= eteness of
>>>>>> openzfs-2.2 support, not an objection either way.<= br> >>>>>>
>>>>>
>>>>> I haven't seen new issues with block cloning in th= e last few weeks
>>>>> mentioned on the mailing lists. All known issues are f= ixed AFAIK.
>>>>> But I can imagine that the risk+effect ratio of data c= orruption is
>>>>> seen as a bit too high for a 14.0 release for this par= ticular
>>>>> feature. That does not diminish the rest of the comple= teness of
>>>>> openzfs-2.2.
>>>>>
>>>>> NB: I'm not involved in developing openzfs or the = decision making
>>>>> in the release. Just repeating what I read on the list= s.
>>>> There was another block cloning fix in 14.0-RC4; see the c= ommit log.
>>>> Maybe there will be no more issues, but it seems that corn= er cases
>>>> were
>>>> still being found recently.
>>> Looks like I'll stay at openzfs-2.1 pool features until th= ere is
>>> a release that no longer has the default status:
>>>
>>> 0 for sysctl vfs.zfs.bclone_enabled
>>>
>>> I use main [so: 15 now] but only enable openzfs-2.* pool featu= res
>>> supported by default on some FreeBSD release, that has an accu= rate
>>> compatibility.d/openzfs-2.*-freebsd file.
>>>
>>> =3D=3D=3D
>>> Mark Millard
>>> marklmi at yahoo.com
>>>
>>>
>>
>

--0000000000002297970609cf3dc8--