From owner-freebsd-hackers@FreeBSD.ORG Tue Nov 8 21:21:33 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A8D5F16A41F for ; Tue, 8 Nov 2005 21:21:33 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from gate.bitblocks.com (bitblocks.com [209.204.185.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id 443FB43D48 for ; Tue, 8 Nov 2005 21:21:33 +0000 (GMT) (envelope-from bakul@bitblocks.com) Received: from bitblocks.com (localhost [127.0.0.1]) by gate.bitblocks.com (8.13.4/8.13.1) with ESMTP id jA8LLNZu051263; Tue, 8 Nov 2005 13:21:24 -0800 (PST) (envelope-from bakul@bitblocks.com) Message-Id: <200511082121.jA8LLNZu051263@gate.bitblocks.com> To: Thomas Hurst In-reply-to: Your message of "Tue, 08 Nov 2005 16:10:53 GMT." <20051108161053.GA38553@voi.aagh.net> Date: Tue, 08 Nov 2005 13:21:23 -0800 From: Bakul Shah Cc: freebsd-hackers@freebsd.org, Carlos Silva aka |Danger_Man| Subject: Re: Backup methodes X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Nov 2005 21:21:33 -0000 Thomas Hurst writes: > * Carlos Silva aka |Danger_Man| (full-disclosure@csilva.org) wrote: > > > what is the best method to backup network information and local disk > > information with another disk? > > dump/restore performs snapshotted incremental backups of complete > filesystems. I have been using venti from plan9ports (a set of plan9 programs ported to unix like OSes) for the past few months now. See http://swtch.com/plan9ports Features: - backup ufs1 and ufs2 over the net to a venti server - Initial full backup seems faster than dump's level 0 backup (I get about 3 to 7 MBps to a USB2 disk). - Saves only one copy of every distinct block no matter which file it belongs to or how many times you give it to venti => less filling, more nutricious! - Every backup is a full backup but because of the above feature venti stores only changed or new blocks. This incremental backup works at close to max disk speed. (I can backup a 30GB filesystem in under 25 minutes to a USB2 disk). Speed of the disk being backed up is the bottleneck so you can simulteneously backup multiple disks to utilize a venti server's full disk/net bandwidth. This is fast enough that backing up everything every night actually works! - each backup returns a single `score'. This serves as a handle to grab the same backup later on. - you can nfs mount the backups. *every* snapshot is available. For example, /dump/my-host/2005/1105/usr. - you can ftp browse a specific backup by giving its score. - You can recreate the image of a partition as per a specific backup. For instance 'vcat > /dev/da0s1e' will recreate a specific disk image. You can then mount it just like a normal disk partition. Though I'd much prefer it if md(4) did this as then it can fetch data on demand. mdconfig -a -t venti -v -s It still has some warts (for example its security model doesn't quite work well for a full restore and you have to resort to vcat) but overall it has been a vast improvement over dump/restore for me. venti can also be used back up file trees like tar does. Venti is close to a Ginsu knife of archiving:-) -- bakul