From owner-freebsd-stable@FreeBSD.ORG Wed Mar 24 16:20:29 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 87A1F106567E for ; Wed, 24 Mar 2010 16:20:29 +0000 (UTC) (envelope-from fjwcash@gmail.com) Received: from mail-px0-f200.google.com (mail-px0-f200.google.com [209.85.216.200]) by mx1.freebsd.org (Postfix) with ESMTP id 5AB988FC1D for ; Wed, 24 Mar 2010 16:20:29 +0000 (UTC) Received: by pxi38 with SMTP id 38so4501091pxi.27 for ; Wed, 24 Mar 2010 09:20:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:content-type; bh=sLJysUoZZF5sCFHiquj3ibUSmSQ8VZ6JBVuqiLB3Mv4=; b=HnpmLx3eFpxaJNRvWIeEcF2fBGl0XocO32clTg3qHSN+SsFLbhbIZnznBWgaG1JLEm aExDW2bXBn/9ZyME0OmSTZk/QgYUy+Md6XUzo76Q6Uyg4lyH9c7eZvUhEA0MuD6KPd2v z8a9u88cLYAcFq+5YFUCuT9HZ7QoYt4xv0xzs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=Z5Af8Av/+GNPe9lWZM48znURlr7irF17BMYeagPV7zvpL7kHwqUDSvtFVaX+joWiTo a/cIhh5smDsULsPT1ACGb9P5I0LaD2Q8EBrxbzG2+xjHDqFaEOFrUFz3FIG3U6Dj4O+G uNIshTXEg1YnqMMfhps6uwutZ/SvIx2wHG04Q= MIME-Version: 1.0 Received: by 10.141.13.7 with SMTP id q7mr3150189rvi.257.1269447628534; Wed, 24 Mar 2010 09:20:28 -0700 (PDT) In-Reply-To: <4BAA3409.6080406@ionic.co.uk> References: <4BAA3409.6080406@ionic.co.uk> Date: Wed, 24 Mar 2010 09:20:28 -0700 Message-ID: From: Freddie Cash To: freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: Multi node storage, ZFS X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2010 16:20:29 -0000 On Wed, Mar 24, 2010 at 8:47 AM, Michal wrote: > I wrote a really long e-mail but realised I could ask this question far > far easier, if it doesn't make sense, the original e-mail is bellow > > Can I use ZFS to create a multinode storage area. Multiple HDD's in > Multiple servers to create one target of, for example, //officestorage > Allowing me to expand the storage space when needed and clients being > able to retrieve data (like RAID0 but over devices not HDD) > > Here is an example I found which is where I'm getting some ideas from > http://www.howtoforge.com/how-to-build-a-low-cost-san-p3 > > Horribly, horribly, horribly complex. But, then, that's the Linux world. :) Server 1: bunch of disks exported via iSCSI Server 2: bunch of disks exported via iSCSI Server 3: bunch of disks exported via iSCSI "SAN" box: uses all those iSCSI exports to create a ZFS pool Use 1 iSCSI export from each server to create a raidz vdev. Or multiple mirror vdevs. When you need more storage, just add another server full of disks, export them via iSCSI to the "SAN" box, and expand the ZFS pool. And, if you need fail-over, on your "SAN" box, you can use HAST at the lower layers (currently only available in 9-CURRENT) to mirror the storage across two systems, and use CARP to provide a single IP for the two boxes. --------------------------------------------------------------------- > I've been searching around and I am finding myself confused and reading > conflicting information. I would like to build a Storage system where by > I have multiple nodes. At the minute I have a number or NAS's which work > well and RAID6 works well in the situation we have, but unfortunately > it's a short-term solution I inherited and once you crunch the numbers > of 6 devices with 6 HDD's in RAID6 you realise how much space you have > wasted then say, 1 device of RAID6 with 36 HDD's (the saving is a fair > few TB) > > Yes, you save space, but your throughput will be horribly horribly horribly low. RAID arrays should be narrow (1-9 disks), not wide (30+ disks), and then combined into a larger array (multiple small RAID6 arrays joined into a RAID0 stripe). -- Freddie Cash fjwcash@gmail.com