From nobody Tue Apr 4 16:14:53 2023 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PrXsY004hz43nyc; Tue, 4 Apr 2023 16:14:57 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Received: from omta001.cacentral1.a.cloudfilter.net (omta001.cacentral1.a.cloudfilter.net [3.97.99.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "Client", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PrXsX6Brkz3qMM; Tue, 4 Apr 2023 16:14:56 +0000 (UTC) (envelope-from cy.schubert@cschubert.com) Authentication-Results: mx1.freebsd.org; none Received: from shw-obgw-4004a.ext.cloudfilter.net ([10.228.9.227]) by cmsmtp with ESMTP id jd67pq7x5uZMSjjJ9pxJWP; Tue, 04 Apr 2023 16:14:55 +0000 Received: from spqr.komquats.com ([70.66.148.124]) by cmsmtp with ESMTPA id jjJ7pa2a43fOSjjJ8pLFUP; Tue, 04 Apr 2023 16:14:55 +0000 X-Authority-Analysis: v=2.4 cv=J8G5USrS c=1 sm=1 tr=0 ts=642c4cff a=Cwc3rblV8FOMdVN/wOAqyQ==:117 a=Cwc3rblV8FOMdVN/wOAqyQ==:17 a=dKHAf1wccvYA:10 a=XldT38RWNwACPDQzwzUA:9 a=6I5d2MoRAAAA:8 a=pGLkceISAAAA:8 a=XjLCjAmAAAAA:8 a=YxBL1-UpAAAA:8 a=EkcXrb_YAAAA:8 a=w1B21tWqbSiANL8Dq_4A:9 a=wPNLvfGTeEIA:10 a=wQnKCjUA6w2wFo1OGagA:9 a=B2y7HmGcmWMA:10 a=ics_IjAVWSmO8OVX31YA:9 a=BOg4e644cxQA:10 a=IjZwj45LgO3ly-622nXo:22 a=77EgncuT9sH9Oa-VvGpq:22 a=Ia-lj3WSrqcvXOmTRaiG:22 a=LK5xJRSDVpKd5WXXoEvA:22 Received: from slippy.cwsent.com (slippy [10.1.1.91]) by spqr.komquats.com (Postfix) with ESMTP id 3F6C3BBF; Tue, 4 Apr 2023 09:14:53 -0700 (PDT) Received: by slippy.cwsent.com (Postfix, from userid 1000) id 2B5F62B6; Tue, 4 Apr 2023 09:14:53 -0700 (PDT) X-Mailer: exmh version 2.9.0 11/07/2018 with nmh-1.8+dev Reply-to: Cy Schubert From: Cy Schubert X-os: FreeBSD X-Sender: cy@cwsent.com X-URL: http://www.cschubert.com/ To: Martin Matuska cc: Rick Macklem , Mateusz Guzik , Cy Schubert , src-committers@freebsd.org, dev-commits-src-all@freebsd.org, dev-commits-src-main@freebsd.org Subject: Re: git: 8ee579abe09e - main - zfs: fall back if block_cloning feature is disabled In-reply-to: <98c71e6f-5b48-79f3-e7b0-95d674949624@FreeBSD.org> References: <202304041145.334Bjx6l035872@gitrepo.freebsd.org> <20230404141717.B976D31C@slippy.cwsent.com> <98c71e6f-5b48-79f3-e7b0-95d674949624@FreeBSD.org> Comments: In-reply-to Martin Matuska message dated "Tue, 04 Apr 2023 17:30:25 +0200." List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org Mime-Version: 1.0 Content-Type: multipart/mixed ; boundary="==_Exmh_1680624788_29580" Date: Tue, 04 Apr 2023 09:14:53 -0700 Message-Id: <20230404161453.2B5F62B6@slippy.cwsent.com> X-CMAE-Envelope: MS4xfEJXyDIU4PjE+xbGbzxCa6WBJkcOJwWHjQRNvgp1jQhKnt6Ui7Zshy+TSOir37K2cExaZtW8KHUOz+PDqcS//3gCTVlSKPyC/vPffnu2pmxgrhv8WsnI 6WGBCrfcpHbJn7yG6uMG3/WGni4gsecsx4ViiYERKbhwDSw0hVFw6EyasBJccPoaiauUFDxAaW2yDtwwyZFUEP09/wRRsOfzjpuCkjGY3F2FcXomwLYGu10B FhWrbR4iqM4gACsg+t1jXXcq8E+J5PJHoNLKj14DnR1lyPgbG1Ll2KdEycHPmSn6EOZPTVbSrULHUzGhgVZRclIy1Pc5jSCJpF8gVAqAFS+waI1z+Y9nWoGz S69soKhLUq9/Tl/XEjIL+CW+i+vlepMn03Oii6yd6WJjEOT7ngM= X-Rspamd-Queue-Id: 4PrXsX6Brkz3qMM X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; TAGGED_RCPT(0.00)[]; ASN(0.00)[asn:16509, ipnet:3.96.0.0/15, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N This is a multipart MIME message. --==_Exmh_1680624788_29580 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Rick has posted a patch. Your patch should also be incorporated to work around other EXDEV errors, but a few lines earlier so it is protected by the lock. There were a couple of typos in Rick's patch (a missing keystroke; s/ojset/objset/). The patch (Rick's null pointer dereference fix, Rick's copy file range patch plus your copy file range patch) builds fine on amd64 and i386. Installing and testing it now. A combination of all three patches is attached. It's compile tested but is currently being installed and will be tested when install is completed. In message <98c71e6f-5b48-79f3-e7b0-95d674949624@FreeBSD.org>, Martin Matuska w rites: > So I am now a little bit confused - what is the consensus? :-) > > On 4. 4. 2023 17:26, Rick Macklem wrote: > > On Tue, Apr 4, 2023 at 7:38 AM Mateusz Guzik wrote: > >> CAUTION: This email originated from outside of the University of Guelph. D > o not click links or open attachments unless you recognize the sender and kno > w the content is safe. If in doubt, forward suspicious emails to IThelp@uogue > lph.ca > >> > >> > >> On 4/4/23, Cy Schubert wrote: > >>> In message <202304041145.334Bjx6l035872@gitrepo.freebsd.org>, Martin > >>> Matuska wr > >>> ites: > >>>> The branch main has been updated by mm: > >>>> > >>>> URL: > >>>> https://cgit.FreeBSD.org/src/commit/?id=8ee579abe09ec1fe15c588fc9a08370b > >>>> 83b81cd6 > >>>> > >>>> commit 8ee579abe09ec1fe15c588fc9a08370b83b81cd6 > >>>> Author: Martin Matuska > >>>> AuthorDate: 2023-04-04 11:40:41 +0000 > >>>> Commit: Martin Matuska > >>>> CommitDate: 2023-04-04 11:43:34 +0000 > >>>> > >>>> zfs: fall back if block_cloning feature is disabled > >>>> > >>>> If block_cloning is disabled, or other errors from zfs_clone_range( > ) > >>>> return an EXDEV we should fall back to vn_generic_copy_file_range() > . > >>>> > >>>> This fixes issues when copying files on the same dataset with > >>>> block_cloning disabled. > >>>> > >>>> Upstreamed as pull request to OpenZFS. > >>>> > >>>> Reviewed by: Mateusz Guzik > >>>> OpenZFS pull request: 14713 > >>>> --- > >>>> .../openzfs/module/os/freebsd/zfs/zfs_vnops_os.c | 17 > >>>> ++++++++++----- > >>>> -- > >>>> 1 file changed, 10 insertions(+), 7 deletions(-) > >>>> > >>>> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>>> b/sys/c > >>>> ontrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>>> index 97429b360a36..2cd1d27e37bc 100644 > >>>> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>>> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>>> @@ -6243,13 +6243,6 @@ zfs_freebsd_copy_file_range(struct > >>>> vop_copy_file_range > >>>> _args *ap) > >>>> int error; > >>>> uint64_t len = *ap->a_lenp; > >>>> > >>>> - /* > >>>> - * TODO: If offset/length is not aligned to recordsize, use > >>>> - * vn_generic_copy_file_range() on this fragment. > >>>> - * It would be better to do this after we lock the vnodes, but then > we > >>>> - * need something else than vn_generic_copy_file_range(). > >>>> - */ > >>>> - > >>>> /* Lock both vnodes, avoiding risk of deadlock. */ > >>>> do { > >>>> mp = NULL; > >>>> @@ -6300,6 +6293,16 @@ unlock: > >>>> if (mp != NULL) > >>>> vn_finished_write(mp); > >>>> > >>>> + /* > >>>> + * Fall back if block_cloning feature is disabled > >>>> + * or other EXDEV failures from zfs_vnops.c > >>>> + */ > >>>> + if (error == EXDEV) { > >>>> + error = vn_generic_copy_file_range(ap->a_invp, ap->a_inoffp > , > >>>> + ap->a_outvp, ap->a_outoffp, ap->a_lenp, ap->a_f > lags > >>>> , > >>>> + ap->a_incred, ap->a_outcred, ap->a_fsizetd); > >>>> + } > >>>> + > >>>> return (error); > >>>> } > >>>> > >>>> > >>> This is too late to fall back. On Rick's suggestion the following makes t > he > >>> > >>> determination at > >>> zfs_freebsd_copy_file_range() entry much earlier. > >>> > >> It's not too late, but I agree it is faster to bail out early. > >> > >> The proposed patch adds a condition which *differs* from the one in > >> zfs_clone_range: > >> if (dmu_objset_spa(inos) != dmu_objset_spa(outos)) { > >> zfs_exit_two(inzfsvfs, outzfsvfs, FTAG); > >> return (SET_ERROR(EXDEV)); > >> } > >> > >> ... meaning with the proposed patch the routine can still fail with > >> EXDEV, making zfs_freebsd_copy_file_range also do it, which must not > >> happen. > > Since VOP_COPY_FILE_RANGE() is only called when invp and outvp > > are on the same mount point, I don't think this can happen now. > > However, there is a TO DO comment that suggests a call with invp and > > outvp on different mount points may be in the future. > > > > As such, leaving Martin's patch in so that it calls vn_generic_copy_file_ra > nge() > > when zfs_clone_range() returns EXDEV seems like a good idea to me. > > > >> That aside the code looks rather suspicious for the case where target > >> and source vnode are the same. iow more work is needed here. > > Definitely needs to be tested. I'll do that later to-day. > > > > rick > > > >> As the vnode is unlocked, you *can't* safely access zfsvfs_t > >> *outzfsvfs = ZTOZSB(outzp); in that spot in this manner -- a forced > >> unmount at the same time can free it. > >> > >> iow this patch does *NOT* work. > >> > >> With the committed variant the situation is damage controlled enough > >> that there is time to sort it out correctly. > >> > >>> diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> index d41821ff67f1..e18dcca58192 100644 > >>> --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c > >>> @@ -6243,6 +6243,18 @@ zfs_freebsd_copy_file_range(struct > >>> vop_copy_file_range_args *ap) > >>> int error; > >>> uint64_t len = *ap->a_lenp; > >>> > >>> + znode_t *outzp = VTOZ(ap->a_outvp); > >>> + zfsvfs_t *outzfsvfs = ZTOZSB(outzp); > >>> + objset_t *outos = outzfsvfs->z_os; > >>> + > >>> + if (!spa_feature_is_enabled(dmu_objset_spa(outos), > >>> + SPA_FEATURE_BLOCK_CLONING)) { > >>> + error = vn_generic_copy_file_range(ap->a_invp, ap->a_inoffp > , > >>> + ap->a_outvp, ap->a_outoffp, ap->a_lenp, ap->a_flags > , > >>> + ap->a_incred, ap->a_outcred, ap->a_fsizetd); > >>> + return (error); > >>> + } > >>> + > >>> /* > >>> * TODO: If offset/length is not aligned to recordsize, use > >>> * vn_gene> >>> > >>> > >>> Can you revert your commit and commit this, please. > >>> > >>> > >>> -- > >>> Cheers, > >>> Cy Schubert > >>> FreeBSD UNIX: Web: https://FreeBSD.org > >>> NTP: Web: https://nwtime.org > >>> > >>> e^(i*pi)+1=0 > >>> > >>> > >>> > >>> > >> > >> -- > >> Mateusz Guzik --==_Exmh_1680624788_29580 Content-Type: application/x-patch ; name="zfs-jumbo.patch" Content-Description: zfs-jumbo.patch diff --git a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c index 97429b360a36..16e0176be2ff 100644 --- a/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c +++ b/sys/contrib/openzfs/module/os/freebsd/zfs/zfs_vnops_os.c @@ -6242,6 +6242,30 @@ zfs_freebsd_copy_file_range(struct vop_copy_file_range_args *ap) struct uio io; int error; uint64_t len = *ap->a_lenp; + zfsvfs_t *outzfsvfs; + objset_t *outos; + bool done_outvp; + + mp = NULL; + error = vn_start_write(outvp, &mp, V_WAIT); + if (error == 0) + error = vn_lock(outvp, LK_EXCLUSIVE); + done_outvp = true; + if (error == 0) { + outzfsvfs = ZTOZSB(VTOZ(outvp)); + outos = outzfsvfs->z_os; + if (!spa_feature_is_enabled(dmu_objset_spa(outos), + SPA_FEATURE_BLOCK_CLONING)) { + VOP_UNLOCK(outvp); + if (mp != NULL) + vn_finished_write(mp); + error = vn_generic_copy_file_range(ap->a_invp, + ap->a_inoffp, ap->a_outvp, ap->a_outoffp, + ap->a_lenp, ap->a_flags, ap->a_incred, + ap->a_outcred, ap->a_fsizetd); + return (error); + } + } /* * TODO: If offset/length is not aligned to recordsize, use @@ -6252,27 +6276,29 @@ zfs_freebsd_copy_file_range(struct vop_copy_file_range_args *ap) /* Lock both vnodes, avoiding risk of deadlock. */ do { - mp = NULL; - error = vn_start_write(outvp, &mp, V_WAIT); + if (!done_outvp) { + mp = NULL; + error = vn_start_write(outvp, &mp, V_WAIT); + if (error == 0) + error = vn_lock(outvp, LK_EXCLUSIVE); + } if (error == 0) { - error = vn_lock(outvp, LK_EXCLUSIVE); - if (error == 0) { - if (invp == outvp) - break; - error = vn_lock(invp, LK_SHARED | LK_NOWAIT); - if (error == 0) - break; - VOP_UNLOCK(outvp); - if (mp != NULL) - vn_finished_write(mp); - mp = NULL; - error = vn_lock(invp, LK_SHARED); - if (error == 0) - VOP_UNLOCK(invp); - } + if (invp == outvp) + break; + error = vn_lock(invp, LK_SHARED | LK_NOWAIT); + if (error == 0) + break; + VOP_UNLOCK(outvp); + if (mp != NULL) + vn_finished_write(mp); + mp = NULL; + error = vn_lock(invp, LK_SHARED); + if (error == 0) + VOP_UNLOCK(invp); } if (mp != NULL) vn_finished_write(mp); + done_outvp = false; } while (error == 0); if (error != 0) return (error); @@ -6290,7 +6316,12 @@ zfs_freebsd_copy_file_range(struct vop_copy_file_range_args *ap) goto unlock; error = zfs_clone_range(VTOZ(invp), ap->a_inoffp, VTOZ(outvp), - ap->a_outoffp, &len, ap->a_fsizetd->td_ucred); + ap->a_outoffp, &len, ap->a_outcred); + if (error == EXDEV) + error = vn_generic_copy_file_range(ap->a_invp, + ap->a_inoffp, ap->a_outvp, ap->a_outoffp, + ap->a_lenp, ap->a_flags, ap->a_incred, + ap->a_outcred, ap->a_fsizetd); *ap->a_lenp = (size_t)len; unlock: --==_Exmh_1680624788_29580 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit Cheers, Cy Schubert FreeBSD UNIX: Web: https://FreeBSD.org NTP: Web: https://nwtime.org e^(i*pi)+1=0 --==_Exmh_1680624788_29580--