From owner-freebsd-stable@FreeBSD.ORG Wed Mar 24 15:47:29 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F2326106566B for ; Wed, 24 Mar 2010 15:47:28 +0000 (UTC) (envelope-from michal@ionic.co.uk) Received: from mail1.sharescope.co.uk (pm1.ionic.co.uk [85.159.80.19]) by mx1.freebsd.org (Postfix) with ESMTP id 860CC8FC0C for ; Wed, 24 Mar 2010 15:47:27 +0000 (UTC) Received: from localhost (unknown [127.0.0.1]) by mail1.sharescope.co.uk (Postfix) with ESMTP id CBB20FC0AB for ; Wed, 24 Mar 2010 15:47:27 +0000 (UTC) X-Virus-Scanned: amavisd-new at sharescope.co.uk Received: from mail1.sharescope.co.uk ([127.0.0.1]) by localhost (mail1.sharescope.co.uk [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HrnZV2EYPthN for ; Wed, 24 Mar 2010 15:47:23 +0000 (GMT) Received: from [192.168.2.37] (office.ionic.co.uk [85.159.85.2]) (Authenticated sender: chris@sharescope.co.uk) by mail1.sharescope.co.uk (Postfix) with ESMTPSA id 5E03CFC0A2 for ; Wed, 24 Mar 2010 15:47:23 +0000 (GMT) Message-ID: <4BAA3409.6080406@ionic.co.uk> Date: Wed, 24 Mar 2010 15:47:21 +0000 From: Michal User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.8) Gecko/20100227 Lightning/1.0b1 Thunderbird/3.0.3 MIME-Version: 1.0 To: freebsd-stable@freebsd.org X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Subject: Multi node storage, ZFS X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Mar 2010 15:47:29 -0000 I wrote a really long e-mail but realised I could ask this question far far easier, if it doesn't make sense, the original e-mail is bellow Can I use ZFS to create a multinode storage area. Multiple HDD's in Multiple servers to create one target of, for example, //officestorage Allowing me to expand the storage space when needed and clients being able to retrieve data (like RAID0 but over devices not HDD) Here is an example I found which is where I'm getting some ideas from http://www.howtoforge.com/how-to-build-a-low-cost-san-p3 Any pointers would be helpful, Thanks --------------------------------------------------------------------- I've been searching around and I am finding myself confused and reading conflicting information. I would like to build a Storage system where by I have multiple nodes. At the minute I have a number or NAS's which work well and RAID6 works well in the situation we have, but unfortunately it's a short-term solution I inherited and once you crunch the numbers of 6 devices with 6 HDD's in RAID6 you realise how much space you have wasted then say, 1 device of RAID6 with 36 HDD's (the saving is a fair few TB) There are other issues as well, increasing the size, 3rd party NAS device features missing which other storage devices have...etc so I looked around and my grand idea was basically this; "Build a system where I can have multiple nodes which create one target (we will use //officestorage for our example) as opposed to //nas1/ which is of course 1 device. Using multiple nodes will allow us to add a new device, thus increasing the space available but the target will always be the same and to the client nothing has change (other then available space) (Think of this as RAID0). Multiple nodes will also allow for redundancy across devices (think RAID1) and give better IO as it's multiple devices and not just 1 device. I could have devices in different locations so a whole building could burn down and still not lose the data" After looking around I found this https://help.ubuntu.com/community/HighlyAvailableAoETarget Which looks quite good, it's basically RAID1 but instead of HDD's it's across servers, I've used DRBD and it worked well, but this doesn't give me better IO as only 1 device is live. I then found this http://www.howtoforge.com/how-to-build-a-low-cost-san-p3 Which looks fantastic, though I will need a master server to create the RAID0 and RAID1 across the multiple nodes and then share this out, which is ok but I would need a hot swap master server, so I'm looking at two of those. I then started thinking about ZFS, I've heard lots of good things about it in the past and thinking can ZFS do what I want. I have read some things which say it can do what that 2nd link does and others which say it can't. Everything I come across is about using just 1 device and I could build 1 device with DRBD, but that doesn't help, nor will it allow me to expand it (if your server runs out of physical space you can't add more HDD's. Anyone point me in the right direction?? Thanks