From owner-freebsd-fs@FreeBSD.ORG  Thu Dec  1 21:35:27 2011
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CD2F3106566C
	for <freebsd-fs@freebsd.org>; Thu,  1 Dec 2011 21:35:27 +0000 (UTC)
	(envelope-from zbeeble@gmail.com)
Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com
	[209.85.161.54])
	by mx1.freebsd.org (Postfix) with ESMTP id 602AB8FC0A
	for <freebsd-fs@freebsd.org>; Thu,  1 Dec 2011 21:35:27 +0000 (UTC)
Received: by faak28 with SMTP id k28so2510235faa.13
	for <freebsd-fs@freebsd.org>; Thu, 01 Dec 2011 13:35:26 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=ZR4d5xLiScddCXaXDcA7FKPJWJqvoAz1wgxiYMWO0pQ=;
	b=oZTRu0YmMcb9iIDMPRFUqZ6k9pJE26ebCQCXWqkfuYUYp9w/rphRTIJs9RbViVtFAf
	x6R1uP2zNwB8/yKaA6nO5usbfKP6XZ4e0ttlJiSVqFjyfb/r6wV2xWZ9lz8qFeOLXCs5
	YskV0sTiVsTRQ9VRifx7i/GVcKkHBkBSuTHks=
MIME-Version: 1.0
Received: by 10.204.152.83 with SMTP id f19mr9256042bkw.90.1322773915741; Thu,
	01 Dec 2011 13:11:55 -0800 (PST)
Received: by 10.204.39.141 with HTTP; Thu, 1 Dec 2011 13:11:55 -0800 (PST)
In-Reply-To: <CALfReyckuygjfp9mot7_MJaAAb-Sk-fsX_TwNGsXtdDKNQ0F8Q@mail.gmail.com>
References: <CAEUA181wUZC-KjVwcm=tTY0DoBLzrNAuBF3aFimSbLB=xht0jw@mail.gmail.com>
	<CALfReycy29VdegrmDrBJ7U3Mjt7+OxUvN7hxOKHOqSX4jD5_kg@mail.gmail.com>
	<4ED77B09.1090709@brockmann-consult.de>
	<CALfReyckuygjfp9mot7_MJaAAb-Sk-fsX_TwNGsXtdDKNQ0F8Q@mail.gmail.com>
Date: Thu, 1 Dec 2011 16:11:55 -0500
Message-ID: <CACpH0Mc_OeoVeqxOThqbnVyyn-kVwALcutQ7c7uaw0tM25_BBw@mail.gmail.com>
From: Zaphod Beeblebrox <zbeeble@gmail.com>
To: krad <kraduk@gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Cc: freebsd-fs@freebsd.org
Subject: Re: ZFS dedup and replication
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 01 Dec 2011 21:35:27 -0000

On Thu, Dec 1, 2011 at 11:42 AM, krad <kraduk@gmail.com> wrote:
> On 1 December 2011 13:03, Peter Maloney
> <peter.maloney@brockmann-consult.de>wrote:
>
>> On 12/01/2011 11:20 AM, krad wrote:

>> On my system with 12 TiB used up, what I do in a script is basically:

>> -generate a snap name
>> -make a recursive snapshot
>> -ssh to the remote server and compare snapshots (find the latest common
>> snapshot, to find an incremental reference point)
>> -if a usable reference point exists, start the incremental send like
>> this (which wipes all changes on the remote system without confirmation)=
:
>> =A0 =A0 =A0 =A0zfs send -R -I ${destLastSnap} ${srcLastSnap} | ssh ${des=
tHost}
>> zfs recv -d -F -v ${destPool}
>> -and if no usable reference point existed, then do a full send,
>> non-incremental:
>> =A0 =A0 =A0 =A0zfs send -R ${srcLastSnap} | ssh ${destHost} zfs recv -F =
-v
>> ${destDataSet}


>> The part about finding the reference snapshot is the most complicated
>> part of my script, and missing from anything else I found online when I
>> was looking for a good solution. For example this script:
>> http://blogs.sun.com/clive/resource/zfs_repl.ksh
>> found on this page:
>> http://blogs.oracle.com/clive/entry/replication_using_zfs

Not that everyone doesn't enjoy inventing their own wheel, but would
you mind sharing your snapshot parser?