From owner-freebsd-fs@freebsd.org Tue Feb 28 15:04:24 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E27C4CF0536 for ; Tue, 28 Feb 2017 15:04:24 +0000 (UTC) (envelope-from Shiva.Bhanujan@Quorum.net) Received: from mail.quorumlabs.com (mail.quorum.net [64.74.133.216]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (Client CN "mail.quorumlabs.com", Issuer "mail.quorumlabs.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CF3B6FB0; Tue, 28 Feb 2017 15:04:23 +0000 (UTC) (envelope-from Shiva.Bhanujan@Quorum.net) Received: from QLEXC01.Quorum.local ([fe80::79eb:fde8:ed8e:5648]) by QLEXC01.Quorum.local ([fe80::79eb:fde8:ed8e:5648%14]) with mapi id 14.02.0318.001; Tue, 28 Feb 2017 07:04:17 -0800 From: Shiva Bhanujan To: =?iso-8859-1?Q?Karli_Sj=F6berg?= , Gary Palmer CC: Jeremy Faulkner , "freebsd-fs@freebsd.org" Subject: RE: FreeBSD restartable send/receive over WAN Thread-Topic: FreeBSD restartable send/receive over WAN Thread-Index: AQHSkE8fXkuYo0TvTyGgnDeYS4b6VKF+hfcf Date: Tue, 28 Feb 2017 15:04:16 +0000 Message-ID: <3A5A10BE32AC9E45B4A22F89FC90EC0701BDB12619@QLEXC01.Quorum.local> References: <0719669324a44fe0bfba3e8e08b0ae99@exch2-4.slu.se> In-Reply-To: <0719669324a44fe0bfba3e8e08b0ae99@exch2-4.slu.se> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [24.6.174.236] MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Feb 2017 15:04:25 -0000 thanks for all the pointers for the compression algorithms. I ran a few te= sts to compare compression overhead. These are local zfs send/receive, and= no network is involved. time zfs send -v | | | zfs receive -= s Here are the performance results that I got. no compression: real 0m20.892s user 0m0.000s sys 0m5.587s xz -0: real 8m38.569s user 10m28.551s sys 0m6.866s pxz -0: real 4m38.448s user 10m55.863s sys 0m13.324s gzip: real 3m51.297s user 4m12.035s sys 0m4.696s lz4: real 0m29.912s user 0m16.543s sys 0m10.810s lz4 has the least overhead in terms of time. pxz/xz seem to be prohibitiv= e give the above results. Unless, there is something basic I'm missing? I was really hoping that compressed sends would be available, as that would= actively eliminate this overhead, given that we use lz4 as the compression= algorithm when writing to disks. ________________________________ From: Karli Sj=F6berg [karli.sjoberg@slu.se] Sent: Sunday, February 26, 2017 8:41 AM To: Gary Palmer Cc: Shiva Bhanujan; Jeremy Faulkner; freebsd-fs@freebsd.org Subject: Re: FreeBSD restartable send/receive over WAN Den 26 feb. 2017 4:16 em skrev Gary Palmer : > > On Sun, Feb 26, 2017 at 02:08:59PM +0000, Shiva Bhanujan wrote: > > The compression that we use on our ZFS filesystems is lz4. So, if I ha= ve to pipe it through a compression algorithm, that'd be uncompressing and = compressing it 4 times. > > > > disk (lz4) -> zfs send (uncompress) -> compress (gzip) -> (network) -> = uncompress (gzip) -> zfs recv (compress) -> disk (lz4) > > > > isn't this quite expensive? We have to transfer multi terabyte files o= n a WAN link. I'm also of the understanding that gzip by itself is single-= threaded, so that'd peg one of the CPUs to 100%. there might be other comp= ression algorithms that can be used, but sending the ZFS as it is compresse= d on the filesystem is something that would be optimal, and would reduce th= e overhead of the additional [de]compressions that are taking place? > > Without going into the efficiency part of your message: > > archivers/pigz: Parallel GZIP > archivers/pbzip2: Parallel BZIP2 > archivers/pixz: Parallel, indexing version of XZ > archivers/pxz: Parallel LZMA compressor using liblzma Also worth mentioning is, obviously: archivers/lz4 :) /K > > Regards, > > Gary > > > > > ________________________________________ > > From: owner-freebsd-fs@freebsd.org [owner-freebsd-fs@freebsd.org] on be= half of Jeremy Faulkner [gldisater@gmail.com] > > Sent: Saturday, February 25, 2017 4:03 PM > > To: freebsd-fs@freebsd.org > > Subject: Re: FreeBSD restartable send/receive over WAN > > > > Pipe it through a compressor > > > > On 2017-02-25 2:09 PM, Shiva Bhanujan wrote: > > > Hi, > > > > > > I just tried restartable send/receive in 10.3 and it works like a cha= rm. I was wondering if compressed send has made its way into FreeBSD? I c= hecked 10.3 and 11.0-RELEASE, and I don't see the -c/--compressed option. = Any pointers? > > > > > > Regards, > > > Shiva > > > > > > > > > ________________________________________ > > > From: owner-freebsd-fs@freebsd.org [owner-freebsd-fs@freebsd.org] on = behalf of Adam Nowacki [nowakpl@platinum.linux.pl] > > > Sent: Thursday, February 16, 2017 10:41 AM > > > To: freebsd-fs@freebsd.org > > > Subject: Re: FreeBSD restartable send/receive over WAN > > > > > > On 2017-02-16 19:22, Shiva Bhanujan wrote: > > >> Hello, > > >> > > >> I was wondering if restartable send/receive is available in FreeBSD?= We're running 10.2 and have a requirement of sending and receiving ZFS sn= apshots over a WAN link. The snapshots could be more than a few terabytes. > > >> > > >> Can somebody please give me pointers, and if this feature is or isn'= t available in FreeBSD? > > > > > > FreeBSD 10.3 and later. > > > > > > _______________________________________________ > > > freebsd-fs@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > _______________________________________________ > > > freebsd-fs@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > _______________________________________________ > > freebsd-fs@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org"