From owner-freebsd-fs@FreeBSD.ORG Thu Jul 4 08:02:55 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 07D4A9BD for ; Thu, 4 Jul 2013 08:02:55 +0000 (UTC) (envelope-from daniel@digsys.bg) Received: from smtp-sofia.digsys.bg (smtp-sofia.digsys.bg [193.68.21.123]) by mx1.freebsd.org (Postfix) with ESMTP id 67B5A15DB for ; Thu, 4 Jul 2013 08:02:54 +0000 (UTC) Received: from dcave.digsys.bg (dcave.digsys.bg [193.68.6.1]) (authenticated bits=0) by smtp-sofia.digsys.bg (8.14.6/8.14.6) with ESMTP id r6482p5M004700 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Thu, 4 Jul 2013 11:02:51 +0300 (EEST) (envelope-from daniel@digsys.bg) Message-ID: <51D52C2B.5050906@digsys.bg> Date: Thu, 04 Jul 2013 11:02:51 +0300 From: Daniel Kalchev User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130627 Thunderbird/17.0.7 MIME-Version: 1.0 To: freebsd-fs@freebsd.org Subject: Re: EBS snapshot backups from a FreeBSD zfs file system: zpool freeze? References: <87mwq34emp.wl%berend@pobox.com> <20130703200241.GB60515@in-addr.com> <87k3l748gb.wl%berend@pobox.com> <20130703233631.GA74698@icarus.home.lan> <87d2qz42q4.wl%berend@pobox.com> <20130704010815.GB75529@icarus.home.lan> <8761wr3xxk.wl%berend@pobox.com> <20130704021535.GA77546@icarus.home.lan> <20130704025100.GB78374@icarus.home.lan> In-Reply-To: <20130704025100.GB78374@icarus.home.lan> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 04 Jul 2013 08:02:55 -0000 On 04.07.13 05:51, Jeremy Chadwick wrote: > On Wed, Jul 03, 2013 at 07:39:58PM -0700, Freddie Cash wrote: >> On 2013-07-03 7:16 PM, "Jeremy Chadwick" wrote: >>> On Thu, Jul 04, 2013 at 01:40:07PM +1200, Berend de Boer wrote: >>>>>>>>> "Jeremy" == Jeremy Chadwick writes: >>>> >>>> Jeremy> Also, because nobody seems to warn others of this: if >>>> Jeremy> you go the ZFS route on FreeBSD, please do not use >>>> Jeremy> features like dedup or compression. >>>> >>>> Exactly the two reasons why I'm experimenting with FreeBSD on AWs. >>>> >>>> Please tell me more. >>> dedup has immense and crazy memory requirements; the commonly referenced >>> model (which is in no way precise, it's just a general recommendation) >>> is that for every 1TB of data you need 1GB of RAM just for the DDT >>> (deduplication table)) -- understand that ZFS's ARC also eats lots of >>> memory, so when I say 1GB of RAM, I'm talking about that being *purely >>> dedicated* to DDT. >> Correction: 1 GB of *ARC* space per TB of *unique* data in the pool. Each >> unique block in the pool gets an entry in the DDT. >> >> You can use L2ARC to store the DDT, although it takes ARC space to track >> data in L2ARC, so you can't go crazy (512 GB L2 with only 16 GB ARC is a >> no-no). >> >> However, you do need a lot of RAM to make dedupe work, and your I/O does >> drop through the floor. > Thanks Freddie -- I didn't know this (re: ARC space per TB of unique > data); wasn't aware that's where the DDT got placed. (Actually makes > sense now that I think about it...) > The really bad thing about this is that the DDT actually competes with everything else in ARC. You don't want to arrive at the point where you trash the ARC with DDT... ZFS with dedup is really "handy" for an non-interactive storage box, such as an archive server. Mine get over 10x dedup ratio and that means I fit the data in 24 disks instead of 240 disks... Extra RAM and L2ARC is well worth the cost and the drop in performance. If you need higher performance form the storage subsystem though, ignore both dedup and compression. Even if they are bug free, some day. Which brings us back to AWS. I believe AWS will charge for CPU time too, which you will happily waste with both dedup and compression. Yet another reason to avoid it, unless block storage (unlikely) is more expensive. Daniel