From owner-freebsd-hackers@FreeBSD.ORG  Sun Oct 31 20:14:31 2010
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 0EAB2106566B
	for <freebsd-hackers@freebsd.org>; Sun, 31 Oct 2010 20:14:31 +0000 (UTC)
	(envelope-from perryh@pluto.rain.com)
Received: from agora.rdrop.com (agora.rdrop.com [IPv6:2607:f678:1010::34])
	by mx1.freebsd.org (Postfix) with ESMTP id C4EEE8FC16
	for <freebsd-hackers@freebsd.org>; Sun, 31 Oct 2010 20:14:30 +0000 (UTC)
Received: from agora.rdrop.com (66@localhost [127.0.0.1])
	by agora.rdrop.com (8.13.1/8.12.7) with ESMTP id o9VKERS1008392
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT);
	Sun, 31 Oct 2010 13:14:27 -0700 (PDT)
	(envelope-from perryh@pluto.rain.com)
Received: (from uucp@localhost)
	by agora.rdrop.com (8.13.1/8.12.9/Submit) with UUCP id o9VKERkx008391; 
	Sun, 31 Oct 2010 13:14:27 -0700 (PDT)
Received: from fbsd61 by pluto.rain.com (4.1/SMI-4.1-pluto-M2060407)
	id AA27545; Sun, 31 Oct 10 12:12:52 PST
Date: Sun, 31 Oct 2010 13:12:26 -0700
From: perryh@pluto.rain.com
To: dillon@apollo.backplane.com
Message-Id: <4ccdcdaa.XSDkZZUUYXDXpkXV%perryh@pluto.rain.com>
References: <AANLkTikzZvZn=vNNRtcSViWq8ty7b8qOooQ4NbHiJH5q@mail.gmail.com>
	<AANLkTikpk4O-q_Omh9eAGZB474J1BVu2YJ7OKWvhZm7v@mail.gmail.com>
	<4ccceb10.4n2iAQ/sY/YrDSI2%perryh@pluto.rain.com>
	<201010311635.o9VGZG1O046164@apollo.backplane.com>
In-Reply-To: <201010311635.o9VGZG1O046164@apollo.backplane.com>
User-Agent: nail 11.25 7/29/05
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Cc: cronfy@gmail.com, freebsd-hackers@freebsd.org
Subject: Re: Slow disk access while rsync - what should I tune?
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
	<freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>, 
	<mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
	<mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Sun, 31 Oct 2010 20:14:31 -0000

[missing attribution restored]
Matthew Dillon <dillon@apollo.backplane.com> wrote:
> perryh@pluto.rain.com wrote:
> :cronfy <cronfy@gmail.com> wrote:
> :
> :> And also, maybe there are other ways to create incremental backups
> :> instead of using rsync/hardlinks?
> :
> :Yes.  Use dump(8) -- that's what it's for.  It reads the inodes,
> :directories, and files directly from the disk device, thereby
> :eliminating stat() overhead entirely.
> :
> :Any replication mechanism -- rsync, tar, even dd -- can be used
> :as a backup mechanism, but dump was specifically designed for the
> :purpose.

> Well, dump is 25+ years old ...

Why are you running BSD if you prefer newer (=> less mature) stuff?
Switch to Linux!

> ... On a modern (large) filesystem you are virtually guaranteed
> to get corruption due to the asynchronous nature of the dump.
>
> This can be partially mitigated by using a block level snapshot on
> the UFS source filesystem and then dumping the snapshot instead of
> the live filesystem ...

IOW by using "dump -L"

> Plus dump uses mtime to detect changes, which is unreliable, ...

Are you sure about that?  Last I knew it used ctime.

> and the output produced by dump is not live-accessible whereas a
> snapshot / live filesystem copy is.  That makes the dump fairly
> worthless for anything other than catastrophic recovery.

Ever heard of "restore -i"?