Date: Mon, 4 Jul 2022 16:12:05 +0200 From: thilo jeremias <thilo@nispuk.com> To: Alan Somers <asomers@freebsd.org> Cc: freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: Problem ZFS send / receive 13.1-RELEASE Message-ID: <83C5ABFD-F786-4074-8D0D-F60E4B5E7A01@nispuk.com> In-Reply-To: <CAOtMX2iu4rpKiF5P7vfizbvcqZ7Mw1QtZoyrzVq%2BLRXhRR2_gA@mail.gmail.com> References: <C9A31AB9-19AE-4FD7-BA52-EC01D0453A3C@nispuk.com> <CAOtMX2iu4rpKiF5P7vfizbvcqZ7Mw1QtZoyrzVq%2BLRXhRR2_gA@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] Understood. Could this be documented in the man-pages (maybe near the zfs-send “-t” option)? Also I think, it would be “a nice to have feature” if this throws an error either at the receiving or sending side. (May be the sending options should be part of the transmission stream and the receiver croaks?) Thank you! Thilo > On 4. Jul 2022, at 15:06, Alan Somers <asomers@freebsd.org> wrote: > > It's not a bug. The problem is that you can't combine zfs send -R and also zfs recv -s . The former creates several independent send streams. But the latter generates a token that will only resume a single stream. And there's no way to fix the problem given ZFS's design. If the send gets interrupted, you need to go check on the destination side to see which file datasets are incomplete and resume all of them. > > On Mon, Jul 4, 2022, 5:53 AM thilo jeremias <thilo@nispuk.com <mailto:thilo@nispuk.com>> wrote: > Hello everyone, > > I posted a question on ( https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resumed-missing-snapshots.85666/ <https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resumed-missing-snapshots.85666/>; ) > Which I now think is a bug in zfs send/receive for 13.1-RELEASE > > > Essentially, an interrupted & restarted receive does not create all snapshots from the stream. > > The following: > #Interrupted transfer > zfs send -R "$D@final" | dd bs=10m count=200 | zfs recv -s "$D"_bad > > #Resume transfer > TOKEN="$(zfs get -H receive_resume_token "$D"_bad | cut -f 3)" > zfs send -t "$TOKEN" | zfs recv -s "$D"_bad > > #Good transfer > zfs send -R "$D@final" | zfs recv -s "$D"_good > > > Produces: > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_good@initial 85.2K - 128K - > Backup/testset_good@add100m 85.2K - 100M - > Backup/testset_good@add300m 85.2K - 400M - > Backup/testset_good@del100m 85.2K - 300M - > Backup/testset_good@del300m 0B - 128K - > Backup/testset_good@final 0B - 128K - > NAME USED AVAIL REFER MOUNTPOINT > Backup/testset_bad@initial 85.2K - 128K - > Backup/testset_bad@add100m 0B - 100M - > > Which is not correct since it misses the snapshots from the original stream (“good”) > > Can this be reproduced by anyone else? > > > Thilo > > > ------------------------------- > Dipl. Ing. Thilo Jeremias > Hauptstraße 11 > 35466 Rabenau > > T: +49 15782492240 > > [-- Attachment #2 --] <html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Understood.<div class=""><br class=""></div><div class="">Could this be documented in the man-pages (maybe near the zfs-send “-t” option)?</div><div class="">Also I think, it would be “a nice to have feature” if this throws an error either at the receiving or sending side.</div><div class="">(May be the sending options should be part of the transmission stream and the receiver croaks?)</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Thank you!</div><div class=""><br class=""></div><div class="">Thilo</div><div class=""><br class=""></div><div class=""> <div><br class=""><blockquote type="cite" class=""><div class="">On 4. Jul 2022, at 15:06, Alan Somers <<a href="mailto:asomers@freebsd.org" class="">asomers@freebsd.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="auto" class="">It's not a bug. The problem is that you can't combine zfs send -R and also zfs recv -s . The former creates several independent send streams. But the latter generates a token that will only resume a single stream. And there's no way to fix the problem given ZFS's design. If the send gets interrupted, you need to go check on the destination side to see which file datasets are incomplete and resume all of them. </div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jul 4, 2022, 5:53 AM thilo jeremias <<a href="mailto:thilo@nispuk.com" class="">thilo@nispuk.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word;line-break:after-white-space" class="">Hello everyone,<div class=""><br class=""></div><div class="">I posted a question on ( <a href="https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resumed-missing-snapshots.85666/" target="_blank" rel="noreferrer" class="">https://forums.freebsd.org/threads/zfs-replication-interrupted-and-resumed-missing-snapshots.85666/</a>; )</div><div class="">Which I now think is a bug in zfs send/receive for 13.1-RELEASE</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Essentially, an interrupted & restarted receive does not create all snapshots from the stream.</div><div class=""><br class=""></div><div class="">The following:</div><div class=""><pre dir="ltr" style="box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Roboto Mono","Andale Mono","Ubuntu Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin-bottom:0px;padding:10px 10px 0px;line-height:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)" class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class="">#Interrupted transfer zfs send -R "$D@final" | dd bs=10m count=200 | zfs recv -s "$D"_bad #Resume transfer TOKEN="$(zfs get -H receive_resume_token "$D"_bad | cut -f 3)" zfs send -t "$TOKEN" | zfs recv -s "$D"_bad </code></pre></div><div class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class=""><br class=""></code></div><div class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class=""><pre dir="ltr" style="box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Roboto Mono","Andale Mono","Ubuntu Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin-bottom:0px;padding:10px 10px 0px;line-height:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)" class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class="">#Good transfer zfs send -R "$D@final" | zfs recv -s "$D"_good </code></pre><div class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class=""><br class=""></code></div></code></div><div class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class=""><br class=""></code></div><div class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class="">Produces:</code></div><div class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class=""><pre dir="ltr" style="box-sizing:border-box;font-family:Monaco,Menlo,Consolas,"Roboto Mono","Andale Mono","Ubuntu Mono",monospace;font-size:13px;word-wrap:normal;margin-top:0px;margin-bottom:0px;padding:10px 10px 0px;line-height:1.4;overflow:auto;direction:ltr;color:rgb(20,20,20)" class=""><code style="box-sizing:border-box;font-family:inherit;font-size:1em" class="">NAME USED AVAIL REFER MOUNTPOINT Backup/testset_good@initial 85.2K - 128K - Backup/testset_good@add100m 85.2K - 100M - Backup/testset_good@add300m 85.2K - 400M - Backup/testset_good@del100m 85.2K - 300M - Backup/testset_good@del300m 0B - 128K - Backup/testset_good@final 0B - 128K - NAME USED AVAIL REFER MOUNTPOINT Backup/testset_bad@initial 85.2K - 128K - Backup/testset_bad@add100m 0B - 100M -</code></pre><div class=""><br class=""></div><div class="">Which is not correct since it misses the snapshots from the original stream (“good”)</div><div class=""><br class=""></div><div class="">Can this be reproduced by anyone else?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Thilo</div></code></div><div class=""><br class=""></div><div class=""><br class=""><div class=""> <div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word; line-break: after-white-space;" class=""><div style="letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; word-wrap: break-word; line-break: after-white-space;" class="">-------------------------------<br class="">Dipl. Ing. Thilo Jeremias<br class="">Hauptstraße 11<br class="">35466 Rabenau<br class=""><br class="">T: +49 15782492240<br class=""><br class=""></div></div> </div> <br class=""></div></div></blockquote></div> </div></blockquote></div><br class=""></div></body></html>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?83C5ABFD-F786-4074-8D0D-F60E4B5E7A01>