From owner-freebsd-stable@FreeBSD.ORG Fri Mar 1 14:25:08 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4D3955A8 for ; Fri, 1 Mar 2013 14:25:08 +0000 (UTC) (envelope-from karl@denninger.net) Received: from fs.denninger.net (wsip-70-169-168-7.pn.at.cox.net [70.169.168.7]) by mx1.freebsd.org (Postfix) with ESMTP id 1D38DFA1 for ; Fri, 1 Mar 2013 14:25:07 +0000 (UTC) Received: from [127.0.0.1] (localhost [127.0.0.1]) by fs.denninger.net (8.14.6/8.13.1) with ESMTP id r21EOwmp048051 for ; Fri, 1 Mar 2013 08:24:59 -0600 (CST) (envelope-from karl@denninger.net) Received: from [127.0.0.1] [192.168.1.40] by Spamblock-sys (LOCAL); Fri Mar 1 08:24:59 2013 Message-ID: <5130BA35.5060809@denninger.net> Date: Fri, 01 Mar 2013 08:24:53 -0600 From: Karl Denninger User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130215 Thunderbird/17.0.3 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Musings on ZFS Backup strategies X-Enigmail-Version: 1.5 X-Antivirus: avast! (VPS 130301-0, 03/01/2013), Outbound message X-Antivirus-Status: Clean Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 14:25:08 -0000 Dabbling with ZFS now, and giving some thought to how to handle backup strategies. ZFS' snapshot capabilities have forced me to re-think the way that I've handled this. Previously near-line (and offline) backup was focused on being able to handle both disasters (e.g. RAID adapter goes nuts and scribbles on the entire contents of the array), a double-disk (or worse) failure, or the obvious (e.g. fire, etc) along with the "aw crap, I just rm -rf'd something I'd rather not!" ZFS makes snapshots very cheap, which means you can resolve the "aw crap" situation without resorting to backups at all. This turns the backup situation into a disaster recovery one. And that in turn seems to say that the ideal strategy looks more like: Take a base snapshot immediately and zfs send it to offline storage. Take an incremental at some interval (appropriate for disaster recovery) and zfs send THAT to stable storage. If I then restore the base and snapshot, I get back to where I was when the latest snapshot was taken. I don't need to keep the incremental snapshot for longer than it takes to zfs send it, so I can do: zfs snapshot pool/some-filesystem@unique-label zfs send -i pool/some-filesystem@base pool/some-filesystem@unique-label zfs destroy pool/some-filesystem@unique-label and that seems to work (and restore) just fine. Am I looking at this the right way here? Provided that the base backup and incremental are both readable, it appears that I have the disaster case covered, and the online snapshot increments and retention are easily adjusted and cover the "oops" situations without having to resort to the backups at all. This in turn means that keeping more than two incremental dumps offline has little or no value; the second merely being taken to insure that there is always at least one that has been written to completion without error to apply on top of the base. That in turn makes the backup storage requirement based only on entropy in the filesystem and not time (where the "tower of Hanoi" style dump hierarchy imposed both a time AND entropy cost on backup media.) Am I missing something here? (Yes, I know, I've been a ZFS resister.... ;-)) -- -- Karl Denninger /The Market Ticker ®/ Cuda Systems LLC