From owner-freebsd-questions@FreeBSD.ORG Thu May 31 18:39:09 2012 Return-Path: <owner-freebsd-questions@FreeBSD.ORG> Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC124106566B for <freebsd-questions@freebsd.org>; Thu, 31 May 2012 18:39:09 +0000 (UTC) (envelope-from dg@pki2.com) Received: from btw.pki2.com (btw.pki2.com [IPv6:2001:470:a:6fd::2]) by mx1.freebsd.org (Postfix) with ESMTP id 20EA28FC12 for <freebsd-questions@freebsd.org>; Thu, 31 May 2012 18:39:08 +0000 (UTC) Received: from btw.pki2.com (btw.pki2.com [192.168.23.1]) by btw.pki2.com (8.14.5/8.14.5) with ESMTP id q4VId4c8051572 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 31 May 2012 11:39:04 -0700 (PDT) (envelope-from dg@pki2.com) Date: Thu, 31 May 2012 11:39:04 -0700 (PDT) From: Dennis Glatting <dg@pki2.com> X-X-Sender: dennisg@btw.pki2.com To: Kaya Saman <kayasaman@gmail.com> In-Reply-To: <CAPj0R5LiVorSRxzOUGN9T8Yeu33dy6o_SkO=WR+bqy9qNAfBUg@mail.gmail.com> Message-ID: <alpine.BSF.2.00.1205311130070.48219@btw.pki2.com> References: <CACxnZKM__Lt9LMabyUC_HOCg2zsMT=3bpqwVrGj16py1A=qffg@mail.gmail.com> <CAPj0R5LwBkQbRvV_1MWbkNwEHOL=zCqXQt1o0pKbyipJmBUbig@mail.gmail.com> <CACxnZKOJHeQH3=T4X3Jd-F3hrgOrirrgypG5HbdfhtFhkcaFzA@mail.gmail.com> <CAPj0R5LiVorSRxzOUGN9T8Yeu33dy6o_SkO=WR+bqy9qNAfBUg@mail.gmail.com> User-Agent: Alpine 2.00 (BSF 1167 2008-08-23) MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="3463506553-125945988-1338489544=:48219" X-yoursite-MailScanner-Information: Dennis Glatting X-yoursite-MailScanner-ID: q4VId4c8051572 X-yoursite-MailScanner: Found to be clean X-MailScanner-From: dg@pki2.com Cc: freebsd-questions@freebsd.org, Oscar Hodgson <oscar.hodgson@gmail.com> Subject: Re: Anyone using freebsd ZFS for large storage servers? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions <freebsd-questions.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, <mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions> List-Post: <mailto:freebsd-questions@freebsd.org> List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, <mailto:freebsd-questions-request@freebsd.org?subject=subscribe> X-List-Received-Date: Thu, 31 May 2012 18:39:09 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --3463506553-125945988-1338489544=:48219 Content-Type: TEXT/PLAIN; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 8BIT On Thu, 31 May 2012, Kaya Saman wrote: > On Thu, May 31, 2012 at 5:05 PM, Oscar Hodgson <oscar.hodgson@gmail.com> wrote: >> That helps. Thank you. >> >> This is an academic departmental instructional / research environment. >> We had a great relationship with Sun, they provided great >> opportunities to put Solaris in front of students. Oracle, not so >> much, and the Oracle single-tier support model simply isn't affordable >> for this "business" (there's no ROI at the departmental level <g>). >> Solaris is not a viable option. > > We found Oracle to be the cheapest out of all the solutions we looked > at: Netapp, MSI, et el..... > >> >> FreeBSD looks like the next best available option at the moment, >> particularly considering the use of the storage heads as compute >> machines. OpenIndiana shows promise. Nexenta has a great product, >> but the user community expects more flexibility in software options. > > FreeBSD is better then Linux in my opinion though lacking some > software and multimedia functionality that Linux has and not for the > Desktop as it's not as "bleeding edge" as say Fedora 16, however, if > FreeBSD offered Gnome3 and supported my wireless NIC I'd be all over > it like a "bad rash" :-) > >> >> Is there anything like a list of "supported" (known good) SAS HBA's? > > LSI HBA's are really good! > > For my DIY solution at home I used a SuperMicro system board with > non-RAID LSI HBA....... > Similarly: mc => Tyan S8812WGM3NR iirc => Supermicro H8DGi bd3 => Soon another Supermicro H8DGi Others are consumer boards from Gigabyte (preferred). I also have a small collection of Supermicro AOC-USAS2-L8i boards. Generally, I have had no trouble but ESXi 5.0 hated them. For work I looked at two Supermicro 848A chassis with a H8QGL board and 20 3TB disks for two different projects, but they lie in limbo. > It is a similar solution that we will use for our test NAS at work > though we already have a Dell R700 series server. For this setup > however I will need to use an LSI HBA with both internal and external > Mini-SAS ports. > > Instead of Promise we will use NetStor JBOD solutions as they work > with 6Gbps drives and overall give better performance. > >> >> Oscar > > Regards, > > > Kaya > >> >> On Thu, May 31, 2012 at 11:38 AM, Kaya Saman <kayasaman@gmail.com> wrote: >>> If this is any consellation I run a 36TB cluster using a self built >>> server with a Promise DAS (VessJBOD 1840) using ZFS at home! to >>> support my OpenSource projects and personal files. >>> >>> As for OS take your pick: NexentaStor, FreeBSD, Solaris 11 >>> >>> >>> All capable, of course Solaris has latest version of ZFS but still..... >>> >>> >>> At work we're looking into getting a StorEdge appliance wich will >>> handle up to 140+ TB. >>> >>> >>> I am also in charge of redesigning one of our virtual SAN's to a >>> FreeBSD ZFS storage system which will run.... well how many JBOD's can >>> you fit on the system?? Probably round ~100TB or so..... >>> >>> >>> Regards, >>> >>> >>> Kaya >>> >>> >>> On Thu, May 31, 2012 at 4:32 PM, Oscar Hodgson <oscar.hodgson@gmail.com> wrote: >>>> The subject is pretty much the question. Perhaps there's a better >>>> place to be asking this question ... >>>> >>>> We have (very briefly) discussed the possibility of using FreeBSD >>>> pizza boxes as a storage heads direct attached to external JBOD arrays >>>> with ZFS. In perusing the list, I haven't stumbled across indications >>>> of people actually doing this. External JBODs would be running 24 to >>>> 48TB each, roughly. There would be a couple of units. The pizza >>>> boxes would be used for computational tasks, and nominally would have >>>> 8 cores and 96G+ RAM. >>>> >>>> Obvious questions are hardware compatibility and stability. I've set >>>> up small FreeBSD 9 machines with ZFS roots and simple mirrors for >>>> other tasks here, and those have been successful so far. >>>> >>>> Observations would be appreciated. >>>> >>>> Oscar. >>>> _______________________________________________ >>>> freebsd-questions@freebsd.org mailing list >>>> http://lists.freebsd.org/mailman/listinfo/freebsd-questions >>>> To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-questions@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-questions > To unsubscribe, send any mail to "freebsd-questions-unsubscribe@freebsd.org" > > --3463506553-125945988-1338489544=:48219--