From nobody Mon Jul 4 14:12:05 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 3A37782BE2F for ; Mon, 4 Jul 2022 14:12:10 +0000 (UTC) (envelope-from thilo@nispuk.com) Received: from metro707.hostmetro.com (metro707.hostmetro.com [50.31.160.160]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Lc76K2hYSz3Hhy; Mon, 4 Jul 2022 14:12:09 +0000 (UTC) (envelope-from thilo@nispuk.com) Received: from rabenau.nispuk.com ([5.10.190.110] helo=localhost.nispuk.com) by metro707.hostmetro.com with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94.2) (envelope-from ) id 1o8MnZ-000604-38; Mon, 04 Jul 2022 09:11:37 -0500 Received: from smtpclient.apple (ThilosMacBookM1.nispuk.com [192.168.0.178]) (authenticated bits=0) by localhost.nispuk.com (8.17.1.9/8.16.1) with ESMTPSA id 264EC5BB097166 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Mon, 4 Jul 2022 16:12:06 +0200 (CEST) (envelope-from thilo@nispuk.com) X-Authentication-Warning: localhost.nispuk.com: Host ThilosMacBookM1.nispuk.com [192.168.0.178] claimed to be smtpclient.apple From: thilo jeremias Message-Id: <83C5ABFD-F786-4074-8D0D-F60E4B5E7A01@nispuk.com> Content-Type: multipart/alternative; boundary="Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9" List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.100.31\)) Subject: Re: Problem ZFS send / receive 13.1-RELEASE Date: Mon, 4 Jul 2022 16:12:05 +0200 In-Reply-To: Cc: freebsd-stable To: Alan Somers References: X-Mailer: Apple Mail (2.3696.100.31) X-Authenticated-Id: thilo@nispuk.com X-Rspamd-Queue-Id: 4Lc76K2hYSz3Hhy X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of thilo@nispuk.com designates 50.31.160.160 as permitted sender) smtp.mailfrom=thilo@nispuk.com X-Spamd-Result: default: False [-1.64 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.99)[-0.991]; FROM_HAS_DN(0.00)[]; MV_CASE(0.50)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; HAS_XAW(0.00)[]; DMARC_NA(0.00)[nispuk.com]; R_SPF_ALLOW(-0.20)[+a]; NEURAL_HAM_LONG(-1.00)[-1.000]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; NEURAL_HAM_SHORT(-0.85)[-0.849]; RCPT_COUNT_TWO(0.00)[2]; HFILTER_HELO_2(1.00)[metro707.hostmetro.com]; MLMMJ_DEST(0.00)[freebsd-stable]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:23352, ipnet:50.31.128.0/18, country:US]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N --Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Understood. Could this be documented in the man-pages (maybe near the zfs-send = =E2=80=9C-t=E2=80=9D option)? Also I think, it would be =E2=80=9Ca nice to have feature=E2=80=9D if = this throws an error either at the receiving or sending side. (May be the sending options should be part of the transmission stream = and the receiver croaks?) Thank you! Thilo > On 4. Jul 2022, at 15:06, Alan Somers wrote: >=20 > It's not a bug. The problem is that you can't combine zfs send -R and = also zfs recv -s . The former creates several independent send streams. = But the latter generates a token that will only resume a single stream. = And there's no way to fix the problem given ZFS's design. If the send = gets interrupted, you need to go check on the destination side to see = which file datasets are incomplete and resume all of them.=20 >=20 > On Mon, Jul 4, 2022, 5:53 AM thilo jeremias > wrote: > Hello everyone, >=20 > I posted a question on ( = https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resumed= -missing-snapshots.85666/ = ) > Which I now think is a bug in zfs send/receive for 13.1-RELEASE >=20 >=20 > Essentially, an interrupted & restarted receive does not create all = snapshots from the stream. >=20 > The following: > #Interrupted transfer=20 > zfs send -R "$D@final" | dd bs=3D10m count=3D200 | zfs recv -s = "$D"_bad >=20 > #Resume transfer > TOKEN=3D"$(zfs get -H receive_resume_token "$D"_bad | cut -f 3)" > zfs send -t "$TOKEN" | zfs recv -s "$D"_bad >=20 > #Good transfer > zfs send -R "$D@final" | zfs recv -s "$D"_good >=20 >=20 > Produces: > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_good@initial 85.2K - 128K - > Backup/testset_good@add100m 85.2K - 100M - > Backup/testset_good@add300m 85.2K - 400M - > Backup/testset_good@del100m 85.2K - 300M - > Backup/testset_good@del300m 0B - 128K - > Backup/testset_good@final 0B - 128K - > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_bad@initial 85.2K - 128K - > Backup/testset_bad@add100m 0B - 100M - >=20 > Which is not correct since it misses the snapshots from the original = stream (=E2=80=9Cgood=E2=80=9D) >=20 > Can this be reproduced by anyone else? >=20 >=20 > Thilo >=20 >=20 > ------------------------------- > Dipl. Ing. Thilo Jeremias > Hauptstra=C3=9Fe 11 > 35466 Rabenau >=20 > T: +49 15782492240 >=20 >=20 --Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 Understood.

Could this be documented in the man-pages (maybe near the = zfs-send =E2=80=9C-t=E2=80=9D option)?
Also I = think, it would be =E2=80=9Ca nice to have feature=E2=80=9D  if = this throws an error either at the receiving or sending side.
(May be the sending options should be part of the = transmission stream and the receiver croaks?)


Thank you!

Thilo


On 4. Jul 2022, at 15:06, Alan Somers <asomers@freebsd.org>= wrote:

It's not a bug. The problem is that you can't = combine zfs send -R and also zfs recv -s .  The former creates = several independent send streams.  But the latter generates a token = that will only resume a single stream. And there's no way to fix the = problem given ZFS's design. If the send gets interrupted,  you need = to go check on the destination side to see which file datasets are = incomplete and resume all of them. 

On Mon, Jul = 4, 2022, 5:53 AM thilo jeremias <thilo@nispuk.com> wrote:
Hello everyone,

I posted a question on ( https://forums.freebsd.org/threads/zfs-replication-interrupted-= and-resumed-missing-snapshots.85666/ )
Which I = now think is a bug in zfs send/receive for 13.1-RELEASE


Essentially, an interrupted & restarted receive does not = create all snapshots from the stream.

The following:
#Interrupted transfer=20
zfs send -R "$D@final" | dd bs=3D10m count=3D200 | zfs recv -s "$D"_bad

#Resume transfer
TOKEN=3D"$(zfs get -H receive_resume_token   "$D"_bad  | cut -f 3)"
zfs send -t "$TOKEN" | zfs recv -s "$D"_bad


#Good transfer
zfs send -R "$D@final" | zfs recv -s "$D"_good


Produces:
NAME                          USED  AVAIL     REFER  =
MOUNTPOINT
Backup/testset_good@initial  85.2K      -      128K  -
Backup/testset_good@add100m  85.2K      -      100M  -
Backup/testset_good@add300m  85.2K      -      400M  -
Backup/testset_good@del100m  85.2K      -      300M  -
Backup/testset_good@del300m     0B      -      128K  -
Backup/testset_good@final       0B      -      128K  -
NAME                         USED  AVAIL     REFER  MOUNTPOINT
Backup/testset_bad@initial  85.2K      -      128K  -
Backup/testset_bad@add100m     0B      -      100M  -

Which is not correct = since it misses the snapshots from the original stream = (=E2=80=9Cgood=E2=80=9D)

Can this be reproduced by anyone else?

Thilo


-------------------------------
Dipl. Ing. = Thilo Jeremias
Hauptstra=C3=9Fe 11
35466 = Rabenau

T: +49 15782492240



= --Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9--