Date: Tue, 28 Feb 2017 20:44:58 +0100 From: "Ronald Klop" <ronald-lists@klop.ws> To: =?utf-8?Q?Karli_Sj=C3=B6berg?= <karli.sjoberg@slu.se>, "Gary Palmer" <gpalmer@freebsd.org>, "Shiva Bhanujan" <Shiva.Bhanujan@quorum.net> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>, "Jeremy Faulkner" <gldisater@gmail.com> Subject: Re: FreeBSD restartable send/receive over WAN Message-ID: <op.ywdyg82mkndu52@53556c9c.cm-6-6b.dynamic.ziggo.nl> In-Reply-To: <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB12619@QLEXC01.Quorum.local> References: <0719669324a44fe0bfba3e8e08b0ae99@exch2-4.slu.se> <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB12619@QLEXC01.Quorum.local>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 28 Feb 2017 16:04:16 +0100, Shiva Bhanujan = <Shiva.Bhanujan@quorum.net> wrote: > thanks for all the pointers for the compression algorithms. I ran a f= ew = > tests to compare compression overhead. These are local zfs = > send/receive, and no network is involved. > > time zfs send -v <src-snapshot> | <compress> | <uncompress> | zfs = > receive -s <dst-snapshot> > > Here are the performance results that I got. > > no compression: > real 0m20.892s > user 0m0.000s > sys 0m5.587s > > xz -0: > real 8m38.569s > user 10m28.551s > sys 0m6.866s > > pxz -0: > real 4m38.448s > user 10m55.863s > sys 0m13.324s > > gzip: > real 3m51.297s > user 4m12.035s > sys 0m4.696s > > lz4: > real 0m29.912s > user 0m16.543s > sys 0m10.810s > > > lz4 has the least overhead in terms of time. pxz/xz seem to be = > prohibitive give the above results. Unless, there is something basic = = > I'm missing? > > I was really hoping that compressed sends would be available, as that = = > would actively eliminate this overhead, given that we use lz4 as the = > compression algorithm when writing to disks. Why don't you test this with a network in between? That would give much = = more valuable numbers to compare for your use case. The number above say nothing about the gain vs the bottleneck you are = trying to remove. Ronald. > > > ________________________________ > From: Karli Sj=C3=B6berg [karli.sjoberg@slu.se] > Sent: Sunday, February 26, 2017 8:41 AM > To: Gary Palmer > Cc: Shiva Bhanujan; Jeremy Faulkner; freebsd-fs@freebsd.org > Subject: Re: FreeBSD restartable send/receive over WAN > > > Den 26 feb. 2017 4:16 em skrev Gary Palmer <gpalmer@freebsd.org>: >> >> On Sun, Feb 26, 2017 at 02:08:59PM +0000, Shiva Bhanujan wrote: >> > The compression that we use on our ZFS filesystems is lz4. So, if = I = >> have to pipe it through a compression algorithm, that'd be = >> uncompressing and compressing it 4 times. >> > >> > disk (lz4) -> zfs send (uncompress) -> compress (gzip) -> (network)= = >> -> uncompress (gzip) -> zfs recv (compress) -> disk (lz4) >> > >> > isn't this quite expensive? We have to transfer multi terabyte fil= es = >> on a WAN link. I'm also of the understanding that gzip by itself is = = >> single-threaded, so that'd peg one of the CPUs to 100%. there might = be = >> other compression algorithms that can be used, but sending the ZFS as= = >> it is compressed on the filesystem is something that would be optimal= , = >> and would reduce the overhead of the additional [de]compressions that= = >> are taking place? >> >> Without going into the efficiency part of your message: >> >> archivers/pigz: Parallel GZIP >> archivers/pbzip2: Parallel BZIP2 >> archivers/pixz: Parallel, indexing version of XZ >> archivers/pxz: Parallel LZMA compressor using liblzma > > Also worth mentioning is, obviously: > archivers/lz4 > > :) > > /K > >> >> Regards, >> >> Gary >> >> > >> > ________________________________________ >> > From: owner-freebsd-fs@freebsd.org [owner-freebsd-fs@freebsd.org] o= n = >> behalf of Jeremy Faulkner [gldisater@gmail.com] >> > Sent: Saturday, February 25, 2017 4:03 PM >> > To: freebsd-fs@freebsd.org >> > Subject: Re: FreeBSD restartable send/receive over WAN >> > >> > Pipe it through a compressor >> > >> > On 2017-02-25 2:09 PM, Shiva Bhanujan wrote: >> > > Hi, >> > > >> > > I just tried restartable send/receive in 10.3 and it works like a= = >> charm. I was wondering if compressed send has made its way into = >> FreeBSD? I checked 10.3 and 11.0-RELEASE, and I don't see the = >> -c/--compressed option. Any pointers? >> > > >> > > Regards, >> > > Shiva >> > > >> > > >> > > ________________________________________ >> > > From: owner-freebsd-fs@freebsd.org [owner-freebsd-fs@freebsd.org]= = >> on behalf of Adam Nowacki [nowakpl@platinum.linux.pl] >> > > Sent: Thursday, February 16, 2017 10:41 AM >> > > To: freebsd-fs@freebsd.org >> > > Subject: Re: FreeBSD restartable send/receive over WAN >> > > >> > > On 2017-02-16 19:22, Shiva Bhanujan wrote: >> > >> Hello, >> > >> >> > >> I was wondering if restartable send/receive is available in = >> FreeBSD? We're running 10.2 and have a requirement of sending and = >> receiving ZFS snapshots over a WAN link. The snapshots could be more= = >> than a few terabytes. >> > >> >> > >> Can somebody please give me pointers, and if this feature is or = = >> isn't available in FreeBSD? >> > > >> > > FreeBSD 10.3 and later. >> > > >> > > _______________________________________________ >> > > freebsd-fs@freebsd.org mailing list >> > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> > > To unsubscribe, send any mail to = >> "freebsd-fs-unsubscribe@freebsd.org" >> > > _______________________________________________ >> > > freebsd-fs@freebsd.org mailing list >> > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> > > To unsubscribe, send any mail to = >> "freebsd-fs-unsubscribe@freebsd.org" >> > > >> > _______________________________________________ >> > freebsd-fs@freebsd.org mailing list >> > https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.or= g" >> > _______________________________________________ >> > freebsd-fs@freebsd.org mailing list >> > https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.or= g" >> > >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"= > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?op.ywdyg82mkndu52>