Date: Mon, 4 Jul 2022 16:12:05 +0200 From: thilo jeremias <thilo@nispuk.com> To: Alan Somers <asomers@freebsd.org> Cc: freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: Problem ZFS send / receive 13.1-RELEASE Message-ID: <83C5ABFD-F786-4074-8D0D-F60E4B5E7A01@nispuk.com> In-Reply-To: <CAOtMX2iu4rpKiF5P7vfizbvcqZ7Mw1QtZoyrzVq%2BLRXhRR2_gA@mail.gmail.com> References: <C9A31AB9-19AE-4FD7-BA52-EC01D0453A3C@nispuk.com> <CAOtMX2iu4rpKiF5P7vfizbvcqZ7Mw1QtZoyrzVq%2BLRXhRR2_gA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 Understood. Could this be documented in the man-pages (maybe near the zfs-send = =E2=80=9C-t=E2=80=9D option)? Also I think, it would be =E2=80=9Ca nice to have feature=E2=80=9D if = this throws an error either at the receiving or sending side. (May be the sending options should be part of the transmission stream = and the receiver croaks?) Thank you! Thilo > On 4. Jul 2022, at 15:06, Alan Somers <asomers@freebsd.org> wrote: >=20 > It's not a bug. The problem is that you can't combine zfs send -R and = also zfs recv -s . The former creates several independent send streams. = But the latter generates a token that will only resume a single stream. = And there's no way to fix the problem given ZFS's design. If the send = gets interrupted, you need to go check on the destination side to see = which file datasets are incomplete and resume all of them.=20 >=20 > On Mon, Jul 4, 2022, 5:53 AM thilo jeremias <thilo@nispuk.com = <mailto:thilo@nispuk.com>> wrote: > Hello everyone, >=20 > I posted a question on ( = https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resumed= -missing-snapshots.85666/ = <https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resume= d-missing-snapshots.85666/> ) > Which I now think is a bug in zfs send/receive for 13.1-RELEASE >=20 >=20 > Essentially, an interrupted & restarted receive does not create all = snapshots from the stream. >=20 > The following: > #Interrupted transfer=20 > zfs send -R "$D@final" | dd bs=3D10m count=3D200 | zfs recv -s = "$D"_bad >=20 > #Resume transfer > TOKEN=3D"$(zfs get -H receive_resume_token "$D"_bad | cut -f 3)" > zfs send -t "$TOKEN" | zfs recv -s "$D"_bad >=20 > #Good transfer > zfs send -R "$D@final" | zfs recv -s "$D"_good >=20 >=20 > Produces: > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_good@initial 85.2K - 128K - > Backup/testset_good@add100m 85.2K - 100M - > Backup/testset_good@add300m 85.2K - 400M - > Backup/testset_good@del100m 85.2K - 300M - > Backup/testset_good@del300m 0B - 128K - > Backup/testset_good@final 0B - 128K - > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_bad@initial 85.2K - 128K - > Backup/testset_bad@add100m 0B - 100M - >=20 > Which is not correct since it misses the snapshots from the original = stream (=E2=80=9Cgood=E2=80=9D) >=20 > Can this be reproduced by anyone else? >=20 >=20 > Thilo >=20 >=20 > ------------------------------- > Dipl. Ing. Thilo Jeremias > Hauptstra=C3=9Fe 11 > 35466 Rabenau >=20 > T: +49 15782492240 >=20 >=20 --Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; line-break: after-white-space;" = class=3D"">Understood.<div class=3D""><br class=3D""></div><div = class=3D"">Could this be documented in the man-pages (maybe near the = zfs-send =E2=80=9C-t=E2=80=9D option)?</div><div class=3D"">Also I = think, it would be =E2=80=9Ca nice to have feature=E2=80=9D if = this throws an error either at the receiving or sending side.</div><div = class=3D"">(May be the sending options should be part of the = transmission stream and the receiver croaks?)</div><div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">Thank you!</div><div class=3D""><br class=3D""></div><div = class=3D"">Thilo</div><div class=3D""><br class=3D""></div><div = class=3D""> <div><br class=3D""><blockquote type=3D"cite" class=3D""><div = class=3D"">On 4. Jul 2022, at 15:06, Alan Somers <<a = href=3D"mailto:asomers@freebsd.org" class=3D"">asomers@freebsd.org</a>>= wrote:</div><br class=3D"Apple-interchange-newline"><div class=3D""><div = dir=3D"auto" class=3D"">It's not a bug. The problem is that you can't = combine zfs send -R and also zfs recv -s . The former creates = several independent send streams. But the latter generates a token = that will only resume a single stream. And there's no way to fix the = problem given ZFS's design. If the send gets interrupted, you need = to go check on the destination side to see which file datasets are = incomplete and resume all of them. </div><br class=3D""><div = class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Mon, Jul = 4, 2022, 5:53 AM thilo jeremias <<a href=3D"mailto:thilo@nispuk.com" = class=3D"">thilo@nispuk.com</a>> wrote:<br class=3D""></div><blockquote= class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc = solid;padding-left:1ex"><div = style=3D"word-wrap:break-word;line-break:after-white-space" = class=3D"">Hello everyone,<div class=3D""><br class=3D""></div><div = class=3D"">I posted a question on ( <a = href=3D"https://forums.freebsd.org/threads/zfs-replication-interrupted-and= -resumed-missing-snapshots.85666/" target=3D"_blank" rel=3D"noreferrer" = class=3D"">https://forums.freebsd.org/threads/zfs-replication-interrupted-= and-resumed-missing-snapshots.85666/</a> )</div><div class=3D"">Which I = now think is a bug in zfs send/receive for 13.1-RELEASE</div><div = class=3D""><br class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">Essentially, an interrupted & restarted receive does not = create all snapshots from the stream.</div><div class=3D""><br = class=3D""></div><div class=3D"">The following:</div><div class=3D""><pre = dir=3D"ltr" = style=3D"box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Rob= oto Mono","Andale Mono","Ubuntu = Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin= -bottom:0px;padding:10px 10px = 0px;line-height:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)" = class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D"">#Interrupted transfer=20 zfs send -R "$D@final" | dd bs=3D10m count=3D200 | zfs recv -s "$D"_bad #Resume transfer TOKEN=3D"$(zfs get -H receive_resume_token "$D"_bad | cut -f 3)" zfs send -t "$TOKEN" | zfs recv -s "$D"_bad </code></pre></div><div class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D""><br class=3D""></code></div><div class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D""><pre dir=3D"ltr" = style=3D"box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Rob= oto Mono","Andale Mono","Ubuntu = Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin= -bottom:0px;padding:10px 10px = 0px;line-height:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)" = class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D"">#Good transfer zfs send -R "$D@final" | zfs recv -s "$D"_good </code></pre><div class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D""><br class=3D""></code></div></code></div><div class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D""><br class=3D""></code></div><div class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D"">Produces:</code></div><div class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D""><pre dir=3D"ltr" = style=3D"box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Rob= oto Mono","Andale Mono","Ubuntu = Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin= -bottom:0px;padding:10px 10px = 0px;line-height:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)" = class=3D""><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em" = class=3D"">NAME USED AVAIL REFER = MOUNTPOINT Backup/testset_good@initial 85.2K - 128K - Backup/testset_good@add100m 85.2K - 100M - Backup/testset_good@add300m 85.2K - 400M - Backup/testset_good@del100m 85.2K - 300M - Backup/testset_good@del300m 0B - 128K - Backup/testset_good@final 0B - 128K - NAME USED AVAIL REFER MOUNTPOINT Backup/testset_bad@initial 85.2K - 128K - Backup/testset_bad@add100m 0B - 100M -</code></pre><div = class=3D""><br class=3D""></div><div class=3D"">Which is not correct = since it misses the snapshots from the original stream = (=E2=80=9Cgood=E2=80=9D)</div><div class=3D""><br class=3D""></div><div = class=3D"">Can this be reproduced by anyone else?</div><div class=3D""><br= class=3D""></div><div class=3D""><br class=3D""></div><div = class=3D"">Thilo</div></code></div><div class=3D""><br = class=3D""></div><div class=3D""><br class=3D""><div class=3D""> <div style=3D"letter-spacing: normal; text-align: start; text-indent: = 0px; text-transform: none; white-space: normal; word-spacing: 0px; = word-wrap: break-word; line-break: after-white-space;" class=3D""><div = style=3D"letter-spacing: normal; text-align: start; text-indent: 0px; = text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: = break-word; line-break: after-white-space;" = class=3D"">-------------------------------<br class=3D"">Dipl. Ing. = Thilo Jeremias<br class=3D"">Hauptstra=C3=9Fe 11<br class=3D"">35466 = Rabenau<br class=3D""><br class=3D"">T: +49 15782492240<br class=3D""><br = class=3D""></div></div> </div> <br class=3D""></div></div></blockquote></div> </div></blockquote></div><br class=3D""></div></body></html>= --Apple-Mail=_A99D67C5-6A2A-4280-B71B-89DB600FE9E9--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?83C5ABFD-F786-4074-8D0D-F60E4B5E7A01>