From owner-freebsd-stable@FreeBSD.ORG Fri Mar 1 15:29:19 2013 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4C65AB18 for ; Fri, 1 Mar 2013 15:29:19 +0000 (UTC) (envelope-from royce.williams@gmail.com) Received: from mail-la0-x22f.google.com (mail-la0-x22f.google.com [IPv6:2a00:1450:4010:c03::22f]) by mx1.freebsd.org (Postfix) with ESMTP id BF920336 for ; Fri, 1 Mar 2013 15:29:18 +0000 (UTC) Received: by mail-la0-f47.google.com with SMTP id fj20so3012942lab.6 for ; Fri, 01 Mar 2013 07:29:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:mime-version:sender:in-reply-to:references:from:date :x-google-sender-auth:message-id:subject:to:cc:content-type; bh=dJZ6Bdv4+FAMZmjkjUE5O6N9D+q8dAyo9GZT4bL6q5Q=; b=ycBZapWz9dt0Ynk20q26woHIfQF6S121Nx8HXlUTOBfFeTqPY/8Ttyh0P2DakI1F3S ak6zmrRA4t8zj61LKBpW60fHKSJVYzV0k5zGieUTULk8/sqlzxFYrxXXqnUDxmFfaOPY Wp/DaFYAcrcN056eiM0A6a/QchGuz2EmLGrBm2cVPbpgopWi86OzoGCiMkiIGGT7EXw+ lHLvTvxvYXQfVPvBLPG9G6Ioz5Mu+xNBiORsNpwge6qmxV2U1zgfTJ74dmd2oZopuIg5 yvlSfLy7qO6payC2JlIZIMh8RBsIGuqj07LLcMXGmsAeaR/Zgvi+RiNtBlvIfpwNxqfb CZ6g== X-Received: by 10.152.104.18 with SMTP id ga18mr9494456lab.33.1362151757658; Fri, 01 Mar 2013 07:29:17 -0800 (PST) MIME-Version: 1.0 Sender: royce.williams@gmail.com Received: by 10.112.59.34 with HTTP; Fri, 1 Mar 2013 07:28:57 -0800 (PST) In-Reply-To: References: <5130BA35.5060809@denninger.net> From: Royce Williams Date: Fri, 1 Mar 2013 06:28:57 -0900 X-Google-Sender-Auth: r2-5D4F6oyU2cQ7NdNkgdQRGn9E Message-ID: Subject: Re: Musings on ZFS Backup strategies To: Ronald Klop Content-Type: text/plain; charset=ISO-8859-1 Cc: freebsd-stable , Karl Denninger X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Mar 2013 15:29:19 -0000 On Fri, Mar 1, 2013 at 6:06 AM, Ronald Klop wrote: > On Fri, 01 Mar 2013 15:24:53 +0100, Karl Denninger > wrote: > >> Dabbling with ZFS now, and giving some thought to how to handle backup >> strategies. >> >> ZFS' snapshot capabilities have forced me to re-think the way that I've >> handled this. Previously near-line (and offline) backup was focused on >> being able to handle both disasters (e.g. RAID adapter goes nuts and >> scribbles on the entire contents of the array), a double-disk (or worse) >> failure, or the obvious (e.g. fire, etc) along with the "aw crap, I just >> rm -rf'd something I'd rather not!" >> >> ZFS makes snapshots very cheap, which means you can resolve the "aw >> crap" situation without resorting to backups at all. This turns the >> backup situation into a disaster recovery one. >> >> And that in turn seems to say that the ideal strategy looks more like: >> >> Take a base snapshot immediately and zfs send it to offline storage. >> Take an incremental at some interval (appropriate for disaster recovery) >> and zfs send THAT to stable storage. >> >> If I then restore the base and snapshot, I get back to where I was when >> the latest snapshot was taken. I don't need to keep the incremental >> snapshot for longer than it takes to zfs send it, so I can do: >> >> zfs snapshot pool/some-filesystem@unique-label >> zfs send -i pool/some-filesystem@base pool/some-filesystem@unique-label >> zfs destroy pool/some-filesystem@unique-label >> >> and that seems to work (and restore) just fine. >> >> Am I looking at this the right way here? Provided that the base backup >> and incremental are both readable, it appears that I have the disaster >> case covered, and the online snapshot increments and retention are >> easily adjusted and cover the "oops" situations without having to resort >> to the backups at all. >> >> This in turn means that keeping more than two incremental dumps offline >> has little or no value; the second merely being taken to insure that >> there is always at least one that has been written to completion without >> error to apply on top of the base. That in turn makes the backup >> storage requirement based only on entropy in the filesystem and not time >> (where the "tower of Hanoi" style dump hierarchy imposed both a time AND >> entropy cost on backup media.) >> >> Am I missing something here? >> >> (Yes, I know, I've been a ZFS resister.... ;-)) >> > > I do the same. I only use zfs send -I (capital i) so I have all the > snapshots on the backup also. > That way the data survives an oops (rm -r) and a fire at the same time. :-) Concur. There are "disasters" that are not obvious until some time has passed -- such as security breaches, application problems that cause quiet data corruption, etc. I do not know how a live ZFS filesystem could be manipulated by an intruder, but the possibility is there. -- Royce Williams