From owner-freebsd-stable@FreeBSD.ORG Sun May 30 05:27:26 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D65CA1065677 for ; Sun, 30 May 2010 05:27:26 +0000 (UTC) (envelope-from delphij@gmail.com) Received: from mail-pv0-f182.google.com (mail-pv0-f182.google.com [74.125.83.182]) by mx1.freebsd.org (Postfix) with ESMTP id AC5368FC13 for ; Sun, 30 May 2010 05:27:26 +0000 (UTC) Received: by pvg16 with SMTP id 16so1293784pvg.13 for ; Sat, 29 May 2010 22:27:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=v8UcPD1gmgOBhql/BMI3c8dJDgqiLOW9JvsrPK9HVo8=; b=CyyQY7Xobq7SAIAa7uaO7VJGKplNN8No1XS+6p7oq5iXqwwNdpEfUVKKj+w9LYRLoM x6/fgNZPleKZcve1thrXjSRDBvSMuVO/0h416IYz8fggJnOa0GODDT3jSit43BojAZy3 riMaPYbzY23uAaxk0o+GxqjrpssnYIUStvpMk= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=kF8BF3t02TcwzDj4Xf7/jbbgj6odOvXnObrge1R2PM5FH7z3zH9QAhb/3VrhFJ+DuW zLHWehomgCc4YIUgsreIJFRJ0wNCYkZods0rvT/cqBciN2da0TlFTLmk5csy5prTt3gL 3rrvM06t/jW4CW1TgDdIWBhGfJ+hkjybknXnU= MIME-Version: 1.0 Received: by 10.141.101.14 with SMTP id d14mr1935414rvm.232.1275197245157; Sat, 29 May 2010 22:27:25 -0700 (PDT) Received: by 10.140.204.3 with HTTP; Sat, 29 May 2010 22:27:25 -0700 (PDT) In-Reply-To: <4C017419.9010909@strauser.com> References: <4C017419.9010909@strauser.com> Date: Sat, 29 May 2010 22:27:25 -0700 Message-ID: From: Xin LI To: Kirk Strauser Content-Type: text/plain; charset=UTF-8 Cc: FreeBSD-STABLE Mailing List Subject: Re: Make ZFS auto-destroy snapshots when the out of space? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 May 2010 05:27:27 -0000 Hi, Kirk, On Sat, May 29, 2010 at 1:07 PM, Kirk Strauser wrote: > I found some nice scripts to regularly snapshot all the filesystems in my > ZFS pool at > http://www.neces.com/blog/technology/integrating-freebsd-zfs-and-periodic-snapshots-and-scrubs > . One thing bothers me, though: I have to intentionally set how many months' > worth of snapshots I want to keep. Too many and I run out of room. Too few > and I lose some of the benefits of easy recovery of deleted data. My > computer is better at bookkeeping than I am, so why not let it? > > I'd propose standardizing on an attribute like org.freebsd:allowautodestroy. > Modify ZFS's disk full behavior to scan for snapshots with that attribute > set and destroy the oldest one, and continue until there's enough free space > to complete a write requests or until out of "expendable" snapshots to > destroy (at which time the normal disk full handler would run). Also run a > daily periodic script to ensure that the free space stays below a > configurable threshold each day so that ZFS isn't constantly butting up > against completely full drives. > > This would take all configuration guesswork out of the equation and would > let me keep as many snapshots as I have space to maintain. If I want to > extend my reach back in time, I can add another drive to the pool and the > rest is handled automatically. At the same time, should I suddenly *want* to > store massive amounts of new data, the snapshots can be easily and > automatically cleared out to make room for the stuff I want to hold. > > What do you think? It seems like this should be pretty easy to implement > without requiring any upstream changes or new FreeBSD-only data structures. > The whole thing could possibly be implemented in userspace, but I don't know > that ZFS has any exception handling callbacks that would make it easy. > > An unused resource is a wasted resource, right? I think this sounds like a good idea but I think we may probably want to explore a more general mechanism, e.g. a daemon that "listen"s system events like file system full, etc. and execute some user defined actions. One thing that we want to avoid is that by making the "automatic recycle" we would open a new race between system and user backup programs, i.e. if you remove an intermediate snapshot, 'zfs send' may fail at receiving side, if incremental send is being used. We would need a way to "notify" that a 'zfs send' is underway. Cheers, -- Xin LI http://www.delphij.net