Date: Thu, 28 May 2020 05:28:40 +0700 From: Eugene Grosbein <eugen@grosbein.net> To: freebsd-stable@freebsd.org Subject: Re: zfs receive -s: transfer got interrupted, but no token on the receiving side. Message-ID: <b633f858-5c04-ccf5-a652-ada2cc08cfc6@grosbein.net> In-Reply-To: <db413d1a-69db-bfab-2ed6-a0ea334f95ad@norma.perm.ru> References: <db413d1a-69db-bfab-2ed6-a0ea334f95ad@norma.perm.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
27.05.2020 22:11, Eugene M. Zheganin wrote:
> Hello,
>
>
> I have a ZFS dataset about 10T of actual size (may be more) that I need to send over a very laggy connection. So I'm sending it from the shell-script that reattempts to send it after a short timeout, retrieving the send token first. Like that:
>
> ===Cut===
>
> #!/bin/sh
>
> exitstatus=1
> token=`ssh user@server zfs get receive_resume_token data/reference | grep -v SOURCE | awk '{print $3}'`
> while ([ $exitstatus -ne 0 ])
> do
> zfs send -t $token | ssh -C user@server sudo zfs receive -Fus data/reference
> exitstatus=$?
> echo "Send interrupted/ended, sleeping for 5 secs."
> sleep 5
> done
>
> ===Cut===
>
> Usually this goes just flawlessly. But this time, due to a low transfer speed and a very laggy connectivity, thus resulting in a send time of several weeks, I got a problem:
>
> about one reattempt out of 200 fails with a situation when there's no snapshot on the receiving side (only an incomplete dataset which is definitely smaller than original one), thus meaning that the dataset is incomplete, but there's no token on this dataset. This happened already twice, each time on a different size.
>
> How can this even be possible ?
>
>
> Recieving side side:
>
> [root@playkey-nas:~]# zfs list -t all
> NAME USED AVAIL REFER MOUNTPOINT
> data 4,67T 20,8T 128K /data
> data/reference 4,67T 20,8T 128K /data/reference
>
> (as you can see there's no snapshot)
>
> Sending side and the snapshot:
>
> [root@san1:/usr/src]# zfs list -t all | grep data/reference@ver2_5917
> data/reference@ver2_5917 44,3G - 7,73T -
zfs(8) manual page tells:
receive_resume_token
For filesystems or volumes which have saved partially-completed state
from zfs receive -s, this opaque token can be provided to zfs send -t
to resume and complete the zfs receive.
Note that /bin/sh does NOT support storing arbitrary opaque data in its variables in unencoded form.
Maybe sometimes you get token with characters that break such code.
Here is how I encode completely opaque and even binary data when I need to store in a shell variable:
key=$(command_printing_data | b64encode -r -)
# use binary key later:
echo "$key" | b64decode -r | geli attach -pk - $p
This technique works for any kind of data including special symbols, zero bytes etc.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?b633f858-5c04-ccf5-a652-ada2cc08cfc6>