From owner-freebsd-fs@FreeBSD.ORG Wed Nov 23 15:50:40 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 735C8106566B for ; Wed, 23 Nov 2011 15:50:40 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id F33418FC0A for ; Wed, 23 Nov 2011 15:50:39 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RTF5o-00038U-Vu for freebsd-fs@freebsd.org; Wed, 23 Nov 2011 16:50:37 +0100 Received: from ib-jtotz.ib.ic.ac.uk ([155.198.110.220]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 23 Nov 2011 16:50:36 +0100 Received: from johannes by ib-jtotz.ib.ic.ac.uk with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 23 Nov 2011 16:50:36 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Johannes Totz Date: Wed, 23 Nov 2011 15:50:17 +0000 Lines: 62 Message-ID: References: <89025E71-B020-406A-BA58-BAF6529974A1@irbisnet.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: ib-jtotz.ib.ic.ac.uk User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0) Gecko/20111105 Thunderbird/8.0 In-Reply-To: <89025E71-B020-406A-BA58-BAF6529974A1@irbisnet.com> Subject: Re: backing up zfs dataset X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Nov 2011 15:50:40 -0000 On 12/11/2011 15:21, Andriy Bakay wrote: > On 2011-11-11, at 11:36 , Johannes Totz wrote: > >> Hi, >> >> To back up a zfs dataset there are a few possibilities: >> 1) rsync file data to another machine >> 2) zfs-send to another machine, into a zfs dataset >> 3) zfs-send to another machine, dumping the stream to a file >> >> The first one works alright but you loose admin info, properties >> seton the dataset, etc >> The second is prefered but requires another machine which runs >> zfs. The third is bad. >> >> So far I have been doing (3), for daily short-term backups, works, >> tested, everything is peachy. However, I dont like it anymore for >> obvious reasons. Ideally, I would like to go with (2). But I dont have >> another zfs-capable machine, or the machine that I would like to backup >> onto will not ever run zfs. >> >> So I came up with another crazy idea, assuming the remote machine >> exports a block device (somehow): >> 4) zpool-attach the remote block dev as a mirror, let it resilver, >> offline it during the day, at night online it, resilver, and so on >> 5) create a pool on the block dev locally on the >> to-be-backed-up-machine and periodically zfs-send stuff over >> >> I would go for (4), it seems to be the mostly automatic. >> Any thoughts on this? >> Should I expect things to go titsup if there's network issues? As Xin mentioned, (4) is not a good idea. Onlining a device starts a resilver from scratch. This is not incremental, i.e. not suited for a nightly backup thing. > > I am using the following schema: sparse file + GGATE + GELI + ZFS. > The destination (remote) machine export/share sparse file through > GGATE. The source/local machine attach GGATE block device, encrypted > with GELI and on top of it create "local" ZFS pool. You can skip GELI > and/or instead sparse file you can use any block device. In this > scenario destination/remote machine does not even know what inside > that sparse file and does not need to support ZFS. You could > periodically attach GGATE/GELI, import ZFS pool and > zfs-send/zfs-receive snapshot(s). This is quite similar to (5) above, and is what I'm going to do. Thanks to you and Xin for suggestions! > > NOTE: GELI will provide storage security, but I am not sure will it > be secure in terms of transmission. Basically you need to make sure > your GGATE layer go through secure/trusted (VPN, SSH tunnel, trusted > LAN) link. > > You could mirror to remote GGATE devices, but it could lead to some > problems (try to search it in mailing list). In this case I recommend > you to consider HAST.