Date: Wed, 3 Jul 2013 19:51:00 -0700 From: Jeremy Chadwick <jdc@koitsu.org> To: Freddie Cash <fjwcash@gmail.com> Cc: FreeBSD Filesystems <freebsd-fs@freebsd.org> Subject: Re: EBS snapshot backups from a FreeBSD zfs file system: zpool freeze? Message-ID: <20130704025100.GB78374@icarus.home.lan> In-Reply-To: <CAOjFWZ7XYxqitX8c%2BHpD%2B4bMd2F%2BZFdhzXZWPsF2V99cW2LvEQ@mail.gmail.com> References: <op.wznad7th34t2sn@tech304.office.supranet.net> <87mwq34emp.wl%berend@pobox.com> <20130703200241.GB60515@in-addr.com> <87k3l748gb.wl%berend@pobox.com> <20130703233631.GA74698@icarus.home.lan> <87d2qz42q4.wl%berend@pobox.com> <20130704010815.GB75529@icarus.home.lan> <8761wr3xxk.wl%berend@pobox.com> <20130704021535.GA77546@icarus.home.lan> <CAOjFWZ7XYxqitX8c%2BHpD%2B4bMd2F%2BZFdhzXZWPsF2V99cW2LvEQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Jul 03, 2013 at 07:39:58PM -0700, Freddie Cash wrote: > On 2013-07-03 7:16 PM, "Jeremy Chadwick" <jdc@koitsu.org> wrote: > > > > On Thu, Jul 04, 2013 at 01:40:07PM +1200, Berend de Boer wrote: > > > >>>>> "Jeremy" == Jeremy Chadwick <jdc@koitsu.org> writes: > > > > > > > > > Jeremy> Also, because nobody seems to warn others of this: if > > > Jeremy> you go the ZFS route on FreeBSD, please do not use > > > Jeremy> features like dedup or compression. > > > > > > Exactly the two reasons why I'm experimenting with FreeBSD on AWs. > > > > > > Please tell me more. > > > > dedup has immense and crazy memory requirements; the commonly referenced > > model (which is in no way precise, it's just a general recommendation) > > is that for every 1TB of data you need 1GB of RAM just for the DDT > > (deduplication table)) -- understand that ZFS's ARC also eats lots of > > memory, so when I say 1GB of RAM, I'm talking about that being *purely > > dedicated* to DDT. > > Correction: 1 GB of *ARC* space per TB of *unique* data in the pool. Each > unique block in the pool gets an entry in the DDT. > > You can use L2ARC to store the DDT, although it takes ARC space to track > data in L2ARC, so you can't go crazy (512 GB L2 with only 16 GB ARC is a > no-no). > > However, you do need a lot of RAM to make dedupe work, and your I/O does > drop through the floor. Thanks Freddie -- I didn't know this (re: ARC space per TB of unique data); wasn't aware that's where the DDT got placed. (Actually makes sense now that I think about it...) -- | Jeremy Chadwick jdc@koitsu.org | | UNIX Systems Administrator http://jdc.koitsu.org/ | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20130704025100.GB78374>