Date: Mon, 4 Jul 2022 07:06:43 -0600 From: Alan Somers <asomers@freebsd.org> To: thilo jeremias <thilo@nispuk.com> Cc: freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: Problem ZFS send / receive 13.1-RELEASE Message-ID: <CAOtMX2iu4rpKiF5P7vfizbvcqZ7Mw1QtZoyrzVq%2BLRXhRR2_gA@mail.gmail.com> In-Reply-To: <C9A31AB9-19AE-4FD7-BA52-EC01D0453A3C@nispuk.com> References: <C9A31AB9-19AE-4FD7-BA52-EC01D0453A3C@nispuk.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--000000000000ea1ac805e2fa6894 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable It's not a bug. The problem is that you can't combine zfs send -R and also zfs recv -s . The former creates several independent send streams. But the latter generates a token that will only resume a single stream. And there's no way to fix the problem given ZFS's design. If the send gets interrupted, you need to go check on the destination side to see which file datasets are incomplete and resume all of them. On Mon, Jul 4, 2022, 5:53 AM thilo jeremias <thilo@nispuk.com> wrote: > Hello everyone, > > I posted a question on ( > https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resume= d-missing-snapshots.85666/ > ) > Which I now think is a bug in zfs send/receive for 13.1-RELEASE > > > Essentially, an interrupted & restarted receive does not create all > snapshots from the stream. > > The following: > > #Interrupted transfer > zfs send -R "$D@final" | dd bs=3D10m count=3D200 | zfs recv -s "$D"_bad > > #Resume transfer > TOKEN=3D"$(zfs get -H receive_resume_token "$D"_bad | cut -f 3)" > zfs send -t "$TOKEN" | zfs recv -s "$D"_bad > > > #Good transfer > zfs send -R "$D@final" | zfs recv -s "$D"_good > > > > Produces: > > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_good@initial 85.2K - 128K - > Backup/testset_good@add100m 85.2K - 100M - > Backup/testset_good@add300m 85.2K - 400M - > Backup/testset_good@del100m 85.2K - 300M - > Backup/testset_good@del300m 0B - 128K - > Backup/testset_good@final 0B - 128K - > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_bad@initial 85.2K - 128K - > Backup/testset_bad@add100m 0B - 100M - > > > Which is not correct since it misses the snapshots from the original > stream (=E2=80=9Cgood=E2=80=9D) > > Can this be reproduced by anyone else? > > > Thilo > > > ------------------------------- > Dipl. Ing. Thilo Jeremias > Hauptstra=C3=9Fe 11 > 35466 Rabenau > > T: +49 15782492240 > > > --000000000000ea1ac805e2fa6894 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable <div dir=3D"auto">It's not a bug. The problem is that you can't com= bine zfs send -R and also zfs recv -s .=C2=A0 The former creates several in= dependent send streams.=C2=A0 But the latter generates a token that will on= ly resume a single stream. And there's no way to fix the problem given = ZFS's design. If the send gets interrupted,=C2=A0 you need to go check = on the destination side to see which file datasets are incomplete and resum= e all of them.=C2=A0</div><br><div class=3D"gmail_quote"><div dir=3D"ltr" c= lass=3D"gmail_attr">On Mon, Jul 4, 2022, 5:53 AM thilo jeremias <<a href= =3D"mailto:thilo@nispuk.com">thilo@nispuk.com</a>> wrote:<br></div><bloc= kquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #cc= c solid;padding-left:1ex"><div style=3D"word-wrap:break-word;line-break:aft= er-white-space">Hello everyone,<div><br></div><div>I posted a question on (= =C2=A0<a href=3D"https://forums.freebsd.org/threads/zfs-replication-interru= pted-and-resumed-missing-snapshots.85666/" target=3D"_blank" rel=3D"norefer= rer">https://forums.freebsd.org/threads/zfs-replication-interrupted-and-res= umed-missing-snapshots.85666/</a> )</div><div>Which I now think is a bug in= zfs send/receive for 13.1-RELEASE</div><div><br></div><div><br></div><div>= Essentially, an interrupted & restarted receive does not create all sna= pshots from the stream.</div><div><br></div><div>The following:</div><div><= pre dir=3D"ltr" style=3D"box-sizing:border-box;font-family:Monaco,Menlo,Con= solas,"Roboto Mono","Andale Mono","Ubuntu Mono&quo= t;,monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin-bottom:0= px;padding:10px 10px 0px;line-height:1.4;overflow:auto;direction:ltr;color:= rgb(20,20,20)"><code style=3D"box-sizing:border-box;font-family:inherit;fon= t-size:1em">#Interrupted transfer=20 zfs send -R "$D@final" | dd bs=3D10m count=3D200 | zfs recv -s &q= uot;$D"_bad #Resume transfer TOKEN=3D"$(zfs get -H receive_resume_token "$D"_bad | cut= -f 3)" zfs send -t "$TOKEN" | zfs recv -s "$D"_bad </code></pre></div><div><code style=3D"box-sizing:border-box;font-family:in= herit;font-size:1em"><br></code></div><div><code style=3D"box-sizing:border= -box;font-family:inherit;font-size:1em"><pre dir=3D"ltr" style=3D"box-sizin= g:border-box;font-family:Monaco,Menlo,Consolas,"Roboto Mono",&quo= t;Andale Mono","Ubuntu Mono",monospace;font-size:13px;word-w= rap:normal;margin-top:0px;margin-bottom:0px;padding:10px 10px 0px;line-heig= ht:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)"><code style=3D"box-= sizing:border-box;font-family:inherit;font-size:1em">#Good transfer zfs send -R "$D@final" | zfs recv -s "$D"_good </code></pre><div><code style=3D"box-sizing:border-box;font-family:inherit;= font-size:1em"><br></code></div></code></div><div><code style=3D"box-sizing= :border-box;font-family:inherit;font-size:1em"><br></code></div><div><code = style=3D"box-sizing:border-box;font-family:inherit;font-size:1em">Produces:= </code></div><div><code style=3D"box-sizing:border-box;font-family:inherit;= font-size:1em"><pre dir=3D"ltr" style=3D"box-sizing:border-box;font-family:= Monaco,Menlo,Consolas,"Roboto Mono","Andale Mono","= ;Ubuntu Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px= ;margin-bottom:0px;padding:10px 10px 0px;line-height:1.4;overflow:auto;dire= ction:ltr;color:rgb(20,20,20)"><code style=3D"box-sizing:border-box;font-fa= mily:inherit;font-size:1em">NAME USED AVAIL R= EFER MOUNTPOINT Backup/testset_good@initial 85.2K - 128K - Backup/testset_good@add100m 85.2K - 100M - Backup/testset_good@add300m 85.2K - 400M - Backup/testset_good@del100m 85.2K - 300M - Backup/testset_good@del300m 0B - 128K - Backup/testset_good@final 0B - 128K - NAME USED AVAIL REFER MOUNTPOINT Backup/testset_bad@initial 85.2K - 128K - Backup/testset_bad@add100m 0B - 100M -</code></pre><div><br>= </div><div>Which is not correct since it misses the snapshots from the orig= inal stream (=E2=80=9Cgood=E2=80=9D)</div><div><br></div><div>Can this be r= eproduced by anyone else?</div><div><br></div><div><br></div><div>Thilo</di= v></code></div><div><br></div><div><br><div> <div style=3D"color:rgb(0,0,0);letter-spacing:normal;text-align:start;text-= indent:0px;text-transform:none;white-space:normal;word-spacing:0px;word-wra= p:break-word;line-break:after-white-space"><div style=3D"color:rgb(0,0,0);l= etter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;w= hite-space:normal;word-spacing:0px;word-wrap:break-word;line-break:after-wh= ite-space">-------------------------------<br>Dipl. Ing. Thilo Jeremias<br>= Hauptstra=C3=9Fe 11<br>35466 Rabenau<br><br>T: +49 15782492240<br><br></div= ></div> </div> <br></div></div></blockquote></div> --000000000000ea1ac805e2fa6894--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAOtMX2iu4rpKiF5P7vfizbvcqZ7Mw1QtZoyrzVq%2BLRXhRR2_gA>