From owner-freebsd-stable@FreeBSD.ORG Thu Oct 30 08:52:46 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E48B2106568C for ; Thu, 30 Oct 2008 08:52:45 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.173]) by mx1.freebsd.org (Postfix) with ESMTP id 6E5638FC18 for ; Thu, 30 Oct 2008 08:52:45 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: by ug-out-1314.google.com with SMTP id 30so804260ugs.39 for ; Thu, 30 Oct 2008 01:52:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:cc:message-id:from:to :in-reply-to:content-type:content-transfer-encoding:mime-version :subject:date:references:x-mailer; bh=rtix3TCZH3KNeC/yp/gmd9WkAmzw0C0v4aLQ4dScihg=; b=oUksSH0E9jTLvhW6cy40wA2Q7p5xEnVAXgCfZuKa49fTeWq1EGhc51lwXjJk5K1LVa ZQaqOFcBQHWINhHQQPKJwrOAhKcrdVxQVlGXMShYEzGAP5r+Df4oqgI6qAkTucpefmm/ pNYTjZD/hgAFiHUGXmlgQ67hoJza2MtfOVbd4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=cc:message-id:from:to:in-reply-to:content-type :content-transfer-encoding:mime-version:subject:date:references :x-mailer; b=IPdXyL1xQJL3FVAXHpgcMYBgG5aRCtlQhMevu0USttJ8vpU7SaPZ1EH6+6T2YGSVji CivZhCalOdhnQWJW91i/XhKZ7JKxKJ/5Vq1AQhehxMC49lqP1YgxQEdm82DWuMtC4WFy mIcMFtNvfktOEWaFk8/GosNKJBbwRyXWNPwBA= Received: by 10.103.93.18 with SMTP id v18mr2810262mul.89.1225355162458; Thu, 30 Oct 2008 01:26:02 -0700 (PDT) Received: from ndenev.cmotd.com (blah.sun-fish.com [217.18.249.150]) by mx.google.com with ESMTPS id u9sm4067573muf.9.2008.10.30.01.25.59 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 30 Oct 2008 01:26:00 -0700 (PDT) Message-Id: <95550BEC-DB92-4C68-8409-3DFF7C0B86C0@gmail.com> From: Nikolay Denev To: Freddie Cash In-Reply-To: Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v929.2) Date: Thu, 30 Oct 2008 10:25:56 +0200 References: <20081029231926.GA35188@0lsen.net> <490907AC.5070303@freebsd.org> <490912CC.6000406@modulus.org> X-Mailer: Apple Mail (2.929.2) Cc: freebsd-stable@freebsd.org Subject: Re: Anyone used rsync scriptology for incremental backup? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Oct 2008 08:52:46 -0000 On 30 Oct, 2008, at 07:00 , Freddie Cash wrote: > On Thu, Oct 30, 2008 at 1:50 AM, Andrew Snow > wrote: >> In this way, each day we generate a batch file that lets us step >> back one >> day. The diffs themselves, compressed with gzip, and extremely space >> efficient. We can step back potentially hundreds of days, though >> it seems >> to throw errors sometimes when backing up Windows boxes, which I >> haven't >> tracked down yet. >> >> But to be honest, soon you can save yourself a lot of hassle by >> simply using >> ZFS and taking snapshots. It'll be faster, and with compression >> very space >> efficient. > > That's exactly what we do, use ZFS and RSync. We have a ZFS > /storage/backup filesystem, with directories for each remote site, and > sub-directories for each server to be backed up. > > Each night we snapshot the directory, then run rsync to backup each > server. Snapshots are named with the current date. For 80 FreeBSD > and Linux servers, we average 10 GB of changed data a night. > > No muss, no fuss. We've used it to restore entire servers (boot off > Knoppix/Frenzy CD, format partitions, rsync back), individual files > (no mounting required, just cd into the .zfs/snapshot/snapshotname > directory and scp the file), and even once to restore the permissions > on a pair of servers where a clueless admin "chmod -R user /home" and > "chmod -R 777 /home". > > Our backup script is pretty much just a double-for loop that scans a > set of site-name directories for server config files, and runs rsync > in parallel (1 per remote site). > > We we looking into using variations on rsnapshot, custom > squashfs/hardlink stuff, and other solutions, but once we started > using ZFS, we stopped looking down those roads. We were able to do in > 3 days of testing and scripting what we hadn't been able to do in > almost a month of research and testing. > > -- > Freddie Cash > fjwcash@gmail.com > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " Hi, Do you experience problems with the snapshots? Last time I tried something similiar for backups the bachine began to spit errors after a few days of snapshots. http://lists.freebsd.org/pipermail/freebsd-fs/2008-February/004413.html -- Regards, Nikolay Denev