From owner-freebsd-fs@FreeBSD.ORG  Mon Feb  6 17:38:31 2012
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CF573106564A
	for <freebsd-fs@freebsd.org>; Mon,  6 Feb 2012 17:38:31 +0000 (UTC)
	(envelope-from fjwcash@gmail.com)
Received: from mail-vw0-f54.google.com (mail-vw0-f54.google.com
	[209.85.212.54])
	by mx1.freebsd.org (Postfix) with ESMTP id 834C98FC08
	for <freebsd-fs@freebsd.org>; Mon,  6 Feb 2012 17:38:31 +0000 (UTC)
Received: by vbbfa15 with SMTP id fa15so6235164vbb.13
	for <freebsd-fs@freebsd.org>; Mon, 06 Feb 2012 09:38:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:cc:content-type:content-transfer-encoding;
	bh=mIKF0AB4gGQhsMIQ8ibZah4k1K6hcFfG3DtV/Pvo4Wo=;
	b=jUEa3uUUObVIHWbGj/Z8P/SA2KsyaqnKbEv8Y8+oeLSqTmlj6e/hXOenFfA37T9uNY
	BdjQ5VcVMCaBjW240Fw2pNMZolc2BSQBH0HLwoxDRUxp3HOpQqFPOfPbrWgtYjBOKVin
	B6kO0lr4zRJj6sd4scA5erc52/5zxkx4o8MAY=
MIME-Version: 1.0
Received: by 10.52.29.11 with SMTP id f11mr2203526vdh.66.1328549910781; Mon,
	06 Feb 2012 09:38:30 -0800 (PST)
Received: by 10.220.181.4 with HTTP; Mon, 6 Feb 2012 09:38:30 -0800 (PST)
In-Reply-To: <413B1A6F-B076-4F50-90EA-7E17CF4B6E36@digsys.bg>
References: <4F2FF72B.6000509@pean.org> <20120206162206.GA541@icarus.home.lan>
	<CAOjFWZ44nP5MVPgvux=Y-x+T+By-WWGVyuAegJYrv6mLmmaN-w@mail.gmail.com>
	<4F300CEA.5000901@fuckner.net>
	<413B1A6F-B076-4F50-90EA-7E17CF4B6E36@digsys.bg>
Date: Mon, 6 Feb 2012 09:38:30 -0800
Message-ID: <CAOjFWZ5cVfvaFR+ecPeuj-cByn=7R87BCSZD_rBsLW5-VDC_gA@mail.gmail.com>
From: Freddie Cash <fjwcash@gmail.com>
To: Daniel Kalchev <daniel@digsys.bg>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Cc: freebsd-fs@freebsd.org
Subject: Re: HPC and zfs.
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 06 Feb 2012 17:38:31 -0000

On Mon, Feb 6, 2012 at 9:34 AM, Daniel Kalchev <daniel@digsys.bg> wrote:
> On Feb 6, 2012, at 7:24 PM, Michael Fuckner wrote:
>> Another thing to think about is CPU: you probably need weeks for a rebui=
ld of a single disk in a Petabyte Filesystem- I haven't tried this with ZFS=
 yet, but I'm really interested if anyone already did this.
>
> This is where ZFS will shine. Depending on how you stripe disks, you can =
either get super fast resilver (if you go for stripe of mirrors), to fast (=
if you go for small number of disks raidz) to reasonable (if you of for lar=
ge number of disks raidz). If you need high TPS you will want to go with mi=
rrors anyway.
>
> The thing is doable with commodity hardware, but I wonder how one ever ba=
ckups such setup?

With a second box configured similarily.  :)  Although, trying to find
"downtime" to do the backups ...

--=20
Freddie Cash
fjwcash@gmail.com