Date: Fri, 01 Mar 2013 09:45:32 -0600 From: Karl Denninger <karl@denninger.net> To: freebsd-stable@freebsd.org Subject: Re: Musings on ZFS Backup strategies Message-ID: <5130CD1C.90709@denninger.net> In-Reply-To: <bafb9e19b43b91127be25924ab139529@dweimer.net> References: <5130BA35.5060809@denninger.net> <bafb9e19b43b91127be25924ab139529@dweimer.net>
next in thread | previous in thread | raw e-mail | index | archive | help
On 3/1/2013 9:36 AM, dweimer wrote: > On 03/01/2013 8:24 am, Karl Denninger wrote: >> Dabbling with ZFS now, and giving some thought to how to handle backup >> strategies. >> >> ZFS' snapshot capabilities have forced me to re-think the way that I've >> handled this. Previously near-line (and offline) backup was focused on >> being able to handle both disasters (e.g. RAID adapter goes nuts and >> scribbles on the entire contents of the array), a double-disk (or worse) >> failure, or the obvious (e.g. fire, etc) along with the "aw crap, I just >> rm -rf'd something I'd rather not!" >> >> ZFS makes snapshots very cheap, which means you can resolve the "aw >> crap" situation without resorting to backups at all. This turns the >> backup situation into a disaster recovery one. >> >> And that in turn seems to say that the ideal strategy looks more like: >> >> Take a base snapshot immediately and zfs send it to offline storage. >> Take an incremental at some interval (appropriate for disaster recovery) >> and zfs send THAT to stable storage. >> >> If I then restore the base and snapshot, I get back to where I was when >> the latest snapshot was taken. I don't need to keep the incremental >> snapshot for longer than it takes to zfs send it, so I can do: >> >> zfs snapshot pool/some-filesystem@unique-label >> zfs send -i pool/some-filesystem@base pool/some-filesystem@unique-label >> zfs destroy pool/some-filesystem@unique-label >> >> and that seems to work (and restore) just fine. >> >> Am I looking at this the right way here? Provided that the base backup >> and incremental are both readable, it appears that I have the disaster >> case covered, and the online snapshot increments and retention are >> easily adjusted and cover the "oops" situations without having to resort >> to the backups at all. >> >> This in turn means that keeping more than two incremental dumps offline >> has little or no value; the second merely being taken to insure that >> there is always at least one that has been written to completion without >> error to apply on top of the base. That in turn makes the backup >> storage requirement based only on entropy in the filesystem and not time >> (where the "tower of Hanoi" style dump hierarchy imposed both a time AND >> entropy cost on backup media.) >> >> Am I missing something here? >> >> (Yes, I know, I've been a ZFS resister.... ;-)) > > I briefly did something like this between two FreeNAS boxes, it seemed > to work well, but my secondary Box wasn't quite up to par hardware. > Combine that with the lack of necessary internet bandwidth with a > second physical location in case of something really disastrous, like > a tornado or fire destroying my house. I ended up just using an eSATA > drive dock and Bacula, with a few external drives rotated regularly > into my office at work, rather than upgrading the secondary box. > > If you have the secondary box that is adequate, and either offsite > backups aren't a concern or you have a big enough pipe to a secondary > location that houses the backup this should work. > > I would recommend testing your incremental snapshot rotation, I never > did test a restore from anything but the most recent set of data when > I was running my setup, I did however save a weeks worth of hourly > snapshots on a couple of the more rapidly changing data sets. > I rotate the disaster disks out to a safe-deposit box at the bank, and they're geli-encrypted, so if stolen they're worthless to the thief (other than their cash value as a drive) and if the building goes "poof" I have the ones in the vault to recover from. There's the potential for loss up to the rotation time of course but that is the same risk I had with all UFS filesystems. I've tested the restores onto a spare box and it appears to work as expected... Thanks for the comments! -- -- Karl Denninger /The Market Ticker ®/ <http://market-ticker.org>; Cuda Systems LLC
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5130CD1C.90709>