Date: Wed, 14 Jun 2017 07:45:54 +0200 From: Dave Cottlehuber <dch@skunkwerks.at> To: freebsd-fs@freebsd.org Subject: Re: FreeBSD restartable send/receive over WAN Message-ID: <1497419154.879644.1008780888.162DA368@webmail.messagingengine.com> In-Reply-To: <1497418488.877884.1008774696.75A50952@webmail.messagingengine.com> References: <0719669324a44fe0bfba3e8e08b0ae99@exch2-4.slu.se> <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB12619@QLEXC01.Quorum.local> <op.ywdyg82mkndu52@53556c9c.cm-6-6b.dynamic.ziggo.nl> <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB12ABA@QLEXC01.Quorum.local> <op.ywmjkqczkndu52@53556c9c.cm-6-6b.dynamic.ziggo.nl> <3A5A10BE32AC9E45B4A22F89FC90EC0701C35D8A68@QLEXC01.Quorum.local> <op.y1dp5ai1kndu52@53556c9c.cm-6-6b.dynamic.ziggo.nl> <3A5A10BE32AC9E45B4A22F89FC90EC0701C35DA475@QLEXC01.Quorum.local> <1497418488.877884.1008774696.75A50952@webmail.messagingengine.com>
next in thread | previous in thread | raw e-mail | index | archive | help
> On Wed, 14 Jun 2017, at 01:34, Shiva Bhanujan wrote:
> Hi Ronald,
>
> thanks for the info. I'm going to work w/ FreeBSD 11.0, as it has
> restartable send/receive. the compressed send is desirable though.
>
> Regards,
> Shivas
A while back I hacked up a self-restarting zfs send, using mbuffer,
which in my setup was faster than ssh. I can't recall if it can
terminate itself or not, but it did the job, transferring 350GB over the
Pacific Ocean migrating off servers.
- use shells/fish
- on source side
while true
sudo zfs send -Levt \
(ssh -p 2200 root@10.144.157.90 zfs get -Hpo value
receive_resume_token zroot/shared/got) \
| mbuffer -4 -s 128k -m 1G -O 10.144.157.90:9000
end
- on sink side
while true
mbuffer -4 -s 128k -m 1G -I 9000 \
| zfs recv -Fuvs zroot/shared/got
end
NB there's no encryption (security/hpenc is an excellent choice), or
replace mbuffer with spiped.
NNB https://gist.github.com/dch/26f2c660fee145c24ffc79126e9cbb21 is the
pretty version.
I'd love to hear back wrt perfomance/throughput.
A+
Dave
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1497419154.879644.1008780888.162DA368>