From nobody Tue Apr 18 01:16:01 2023 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Q0mG956fTz45TGf for ; Tue, 18 Apr 2023 01:16:17 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Received: from mail-ej1-x635.google.com (mail-ej1-x635.google.com [IPv6:2a00:1450:4864:20::635]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Q0mG80vLfz3wpZ for ; Tue, 18 Apr 2023 01:16:16 +0000 (UTC) (envelope-from wlosh@bsdimp.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=bsdimp-com.20221208.gappssmtp.com header.s=20221208 header.b=Ty8fOYHb; spf=none (mx1.freebsd.org: domain of wlosh@bsdimp.com has no SPF policy when checking 2a00:1450:4864:20::635) smtp.mailfrom=wlosh@bsdimp.com; dmarc=none Received: by mail-ej1-x635.google.com with SMTP id kt6so30879491ejb.0 for ; Mon, 17 Apr 2023 18:16:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bsdimp-com.20221208.gappssmtp.com; s=20221208; t=1681780573; x=1684372573; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=yUY7Abn9c9PaqNNRkIxr+v6jaT7Z+RgW2tnfRAp36IA=; b=Ty8fOYHbBYSKSvzEsxrA4Uh/NmrVkJuWwBoxHh70wt77i0gOrCBWntnNs0iOsL5iK2 H0EWY7/j9VuczqkljY9/i7VF5Et/axOfhm0bmhk8eS4oMtjNI+95H/rUgmxBRYSNRj5B KaFGOsUKbjdvxJoM1ADcRqTpP+mZOT1aFuMXClSKBHvq6foq05GLuSbDxI/nTIUrR83W +6i8OBIzXnvVSjQWbtV6bRMB2pbPWCZ8u9wF5qrTVF+P8bc3bTIw1HDkrPP4tY2kz68G V545D2rRTWQZc01wjmCEghdjcd5Js+nLpKOzFxPBS0VWip5NOwzhzwVA2cLWFvrcDdDW Y3zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681780573; x=1684372573; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=yUY7Abn9c9PaqNNRkIxr+v6jaT7Z+RgW2tnfRAp36IA=; b=ecyQr6EcfkblVOXHI/1Xv3GZwvDs/7MrVFuOj9d9S+QGZkCjjMxvJzdnI6Icam13x5 c+tW8BiM6N8LuSQ0rWxL2Cj5ErGhioFz1Uj09xpTcH+A03dKrenGT1fCPaiM/ekdt6HE birnj5Bp9LHnT1FlNJSxGUSs7uvNwIJfONDPaiDuGWHmCskHrQ8TX7HP38XPzjEO56Fp uyOVWl8ZCI1c/b6geGxw/4bX6EuZhal+h2fGfMPpH3tdOv0GZ7ICq8IdtATcDMVvqrzR ZQnAqRjG0HpM+vVeySXzs1E3uwHclpwi+Y4W27pRgc7OPa4Sr0lE+R34dwAWaYBUPVXj TdrQ== X-Gm-Message-State: AAQBX9djRqj8OQiSynchxDan8pDrIWTpClw8sQRLHouslOJCxumRdrRc uiVSM+oGp4eDFmZe976EMMq55kIQyAZj8waTVwLWgA== X-Google-Smtp-Source: AKy350Yi0XGLzSsdbcY+qkeNHQsz7l4CM+D4UyR54lFlZi22QJkdQo0Rl8U6zRT0Ovt5brvvIJPc3cKrc4pwSu2K694= X-Received: by 2002:a17:906:4757:b0:94e:7ce:4d1f with SMTP id j23-20020a170906475700b0094e07ce4d1fmr4587243ejs.2.1681780573309; Mon, 17 Apr 2023 18:16:13 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: <48e02888-c49f-ab2b-fc2d-ad6db6f0e10b@dawidek.net> <20230417232859.18262E2@slippy.cwsent.com> In-Reply-To: From: Warner Losh Date: Mon, 17 Apr 2023 19:16:01 -0600 Message-ID: Subject: Re: another crash and going forward with zfs To: Rick Macklem Cc: Cy Schubert , Pawel Jakub Dawidek , Mateusz Guzik , FreeBSD Current , Glen Barber Content-Type: multipart/alternative; boundary="000000000000a9a06505f9920d11" X-Spamd-Result: default: False [-1.50 / 15.00]; SUSPICIOUS_RECIPS(1.50)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FORGED_SENDER(0.30)[imp@bsdimp.com,wlosh@bsdimp.com]; R_DKIM_ALLOW(-0.20)[bsdimp-com.20221208.gappssmtp.com:s=20221208]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; DMARC_NA(0.00)[bsdimp.com]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::635:from]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RCVD_COUNT_TWO(0.00)[2]; FREEMAIL_TO(0.00)[gmail.com]; TAGGED_RCPT(0.00)[]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[imp@bsdimp.com,wlosh@bsdimp.com]; DKIM_TRACE(0.00)[bsdimp-com.20221208.gappssmtp.com:+]; ARC_NA(0.00)[]; FREEMAIL_CC(0.00)[cschubert.com,freebsd.org,gmail.com]; RCPT_COUNT_FIVE(0.00)[6] X-Rspamd-Queue-Id: 4Q0mG80vLfz3wpZ X-Spamd-Bar: - X-ThisMailContainsUnwantedMimeParts: N --000000000000a9a06505f9920d11 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable On Mon, Apr 17, 2023, 5:37 PM Rick Macklem wrote: > On Mon, Apr 17, 2023 at 4:29=E2=80=AFPM Cy Schubert > wrote: > > > > In message , Pawel > Jakub > > Dawi > > dek writes: > > > On 4/18/23 05:14, Mateusz Guzik wrote: > > > > On 4/17/23, Pawel Jakub Dawidek wrote: > > > >> Correct me if I'm wrong, but from my understanding there were zero > > > >> problems with block cloning when it wasn't in use or now disabled. > > > >> > > > >> The reason I've introduced vfs.zfs.bclone_enabled sysctl, was to > exactly > > > >> avoid mess like this and give us more time to sort all the problem= s > out > > > >> while making it easy for people to try it. > > > >> > > > >> If there is no plan to revert the whole import, I don't see what > value > > > >> removing just block cloning will bring if it is now disabled by > default > > > >> and didn't cause any problems when disabled. > > > >> > > > > > > > > The feature definitely was not properly stress tested and what not > and > > > > trying to do it keeps running into panics. Given the complexity of > the > > > > feature I would expect there are many bug lurking, some of which > > > > possibly related to the on disk format. Not having to deal with any > of > > > > this is can be arranged as described above and is imo the most > > > > sensible route given the timeline for 14.0 > > > > > > Block cloning doesn't create, remove or modify any on-disk data until > it > > > is in use. > > > > > > Again, if we are not going to revert the whole merge, I see no point = in > > > reverting block cloning as until it is enabled, its code is not > > > executed. This allow people who upgraded the pools to do nothing > special > > > and it will allow people to test it easily. > > > > In this case zpool upgrade and zpool status should return no feature > > upgrades are available instead of enticing users to zpool upgrade. The > > userland zpool command should test for this sysctl and print nothing > > regarding block_cloning. I can see a scenario when a user zpool upgrade= s > > their pools, notices the sysctl and does the unthinkable. Not only woul= d > > this fill the mailing lists with angry chatter but it would spawn a > number > > of PRs plus give us a lot of bad press for data loss. > > > > Should we keep the new ZFS in 14, we should: > > > > 1. Make sure that zpool(8) does not mention or offer block_cloning in a= ny > > way if the sysctl is disabled. > > > > 2. Print a cautionary note in release notes advising people not to enab= le > > this experimental sysctl. Maybe even have it print "(experimental)" to > warn > > users that it will hurt. > > > > 3. Update the man pages to caution that block_cloning is experimental a= nd > > unstable. > I would suggest going a step further and making the sysctl RO for > FreeBSD14. > (This could be changed for FreeBSD14.n if/when block_cloning is believed = to > be debugged.) > > I would apply all 3 of the above to "main", since some that install "main= " > will not know how "bleeding edge" this is unless the above is done. > (Yes, I know "main" is "bleeding edge", but some still expect a stable > test system will result from installing it.) > > Thanks go to all that tracked this problem down, rick > Related question: what zfs branch is stable/14 going to track? With 13 it was whatever the next stable branch was. Warner > > > It's not enough to have a sysctl without hiding block_cloning completel= y > > from view. Only expose it in zpool(8) when the sysctl is enabled. Let's > > avoid people mistakenly enabling it. > > > > > > -- > > Cheers, > > Cy Schubert > > FreeBSD UNIX: Web: https://FreeBSD.org > > NTP: Web: https://nwtime.org > > > > e^(i*pi)+1=3D0 > > > > > > > > --000000000000a9a06505f9920d11 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable


On Mon, Apr 17, 2023, 5:37 PM Rick Macklem <rick.macklem@gmail.com> wrote:
On Mon, Apr 17, 2023 at 4:29=E2=80=AF= PM Cy Schubert <Cy.Schubert@cschubert.com> wrote:
>
> In message <b57b06bd-7e73-ae2d-2fba-b= d226883ff34@dawidek.net>, Pawel Jakub
> Dawi
> dek writes:
> > On 4/18/23 05:14, Mateusz Guzik wrote:
> > > On 4/17/23, Pawel Jakub Dawidek <pjd@freebsd.org> wro= te:
> > >> Correct me if I'm wrong, but from my understanding t= here were zero
> > >> problems with block cloning when it wasn't in use or= now disabled.
> > >>
> > >> The reason I've introduced vfs.zfs.bclone_enabled sy= sctl, was to exactly
> > >> avoid mess like this and give us more time to sort all t= he problems out
> > >> while making it easy for people to try it.
> > >>
> > >> If there is no plan to revert the whole import, I don= 9;t see what value
> > >> removing just block cloning will bring if it is now disa= bled by default
> > >> and didn't cause any problems when disabled.
> > >>
> > >
> > > The feature definitely was not properly stress tested and wh= at not and
> > > trying to do it keeps running into panics. Given the complex= ity of the
> > > feature I would expect there are many bug lurking, some of w= hich
> > > possibly related to the on disk format. Not having to deal w= ith any of
> > > this is can be arranged as described above and is imo the mo= st
> > > sensible route given the timeline for 14.0
> >
> > Block cloning doesn't create, remove or modify any on-disk da= ta until it
> > is in use.
> >
> > Again, if we are not going to revert the whole merge, I see no po= int in
> > reverting block cloning as until it is enabled, its code is not > > executed. This allow people who upgraded the pools to do nothing = special
> > and it will allow people to test it easily.
>
> In this case zpool upgrade and zpool status should return no feature > upgrades are available instead of enticing users to zpool upgrade. The=
> userland zpool command should test for this sysctl and print nothing > regarding block_cloning. I can see a scenario when a user zpool upgrad= es
> their pools, notices the sysctl and does the unthinkable. Not only wou= ld
> this fill the mailing lists with angry chatter but it would spawn a nu= mber
> of PRs plus give us a lot of bad press for data loss.
>
> Should we keep the new ZFS in 14, we should:
>
> 1. Make sure that zpool(8) does not mention or offer block_cloning in = any
> way if the sysctl is disabled.
>
> 2. Print a cautionary note in release notes advising people not to ena= ble
> this experimental sysctl. Maybe even have it print "(experimental= )" to warn
> users that it will hurt.
>
> 3. Update the man pages to caution that block_cloning is experimental = and
> unstable.
I would suggest going a step further and making the sysctl RO for FreeBSD14= .
(This could be changed for FreeBSD14.n if/when block_cloning is believed to=
=C2=A0be debugged.)

I would apply all 3 of the above to "main", since some that insta= ll "main"
will not know how "bleeding edge" this is unless the above is don= e.
(Yes, I know "main" is "bleeding edge", but some still = expect a stable
=C2=A0test system will result from installing it.)

Thanks go to all that tracked this problem down, rick

Related question: what= zfs branch is stable/14 going to track? With 13 it was whatever the next s= table branch was.

Warner=


>
> It's not enough to have a sysctl without hiding block_cloning comp= letely
> from view. Only expose it in zpool(8) when the sysctl is enabled. Let&= #39;s
> avoid people mistakenly enabling it.
>
>
> --
> Cheers,
> Cy Schubert <Cy.Schubert@cschubert.com>
> FreeBSD UNIX:=C2=A0 <cy@FreeBSD.org>=C2=A0 =C2=A0Web:=C2=A0 https://FreeBSD.org
> NTP:=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0<cy@nwtime.org>=C2= =A0 =C2=A0 Web:=C2=A0 https://nwtime.org
>
>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 = =C2=A0 =C2=A0 =C2=A0e^(i*pi)+1=3D0
>
>
>

--000000000000a9a06505f9920d11--