From owner-freebsd-questions@FreeBSD.ORG Tue Sep 30 23:00:50 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 316C21065693 for ; Tue, 30 Sep 2008 23:00:50 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA07.westchester.pa.mail.comcast.net (qmta07.westchester.pa.mail.comcast.net [76.96.62.64]) by mx1.freebsd.org (Postfix) with ESMTP id 907D98FC0C for ; Tue, 30 Sep 2008 23:00:49 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA06.westchester.pa.mail.comcast.net ([76.96.62.51]) by QMTA07.westchester.pa.mail.comcast.net with comcast id M3N81a00216LCl057AkoTV; Tue, 30 Sep 2008 22:44:48 +0000 Received: from koitsu.dyndns.org ([67.180.253.227]) by OMTA06.westchester.pa.mail.comcast.net with comcast id MAkn1a00K4v8bD73SAknHq; Tue, 30 Sep 2008 22:44:48 +0000 X-Authority-Analysis: v=1.0 c=1 a=p9pqlfZpwwoA:10 a=6FVY1_NhIzwA:10 a=6I5d2MoRAAAA:8 a=QycZ5dHgAAAA:8 a=AhB9zNJ2ICMwCcWvEhgA:9 a=sO42JhqZWtrjQ-l_fnAA:7 a=ffx_Ub8bYWeY-j7kQndxBF3pUsAA:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id 2912EC9419; Tue, 30 Sep 2008 15:44:47 -0700 (PDT) Date: Tue, 30 Sep 2008 15:44:47 -0700 From: Jeremy Chadwick To: Danny Do Message-ID: <20080930224447.GA58065@icarus.home.lan> References: <1222681181.48e0a25d094c3@www.inbox.lv> <48E21C66.8080407@FreeBSD.org> <20080930161407.E16761@wojtek.tensor.gdynia.pl> <20080930205435.B20033@wojtek.tensor.gdynia.pl> <00ee01c92346$69508fa0$3bf1aee0$@com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <00ee01c92346$69508fa0$3bf1aee0$@com> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: 'Wojciech Puchar' , freebsd-questions@freebsd.org Subject: Re: Optimal File System config for 2.5TB RAID5 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Sep 2008 23:00:50 -0000 On Wed, Oct 01, 2008 at 04:49:27AM +0700, Danny Do wrote: > I got Perc 4E-DI Embedded Raid Adapter (256MB) from DELL for my current SCSI > system. They said it's the enterprise class. I don't know much about the > performance between software RAID and hardware RAID. I'm not familiar with PERC (LSI) controllers, just for the record. > Could you please tell me if this type of hardware RAID controller could > match the software RAID you were talking about? What you're asking for is "too much" -- and this conversation is starting to delve into freebsd-hardware, not freebsd-questions. Unless someone out there has done full benchmarks comparing FreeBSD ZFS or FreeBSD gvinum to a PERC 4E-DI, with all kinds of test cases (what sort of server it is, what it's doing disk-wise, etc.), I doubt you'll be able to get a conclusive answer here. Such benchmarking would require weeks of effort by someone. Heck, I'm not even sure FreeBSD supports the PERC 4E-DI. That said, if you go with that controller, you should be aware of the following things: there are many problems with hardware RAID. 1) If the controller goes bad after the lifetime of the controller has expired, there is very little chance the vendor will give you a replacement controller that understands the metadata of the previous/bad controller. You are flat out stuck with that model of controller for the rest of your life, unless the vendor can *guarantee* backwards compatibility when providing a newer controller. And I'm willing to bet money that general technical support has no idea what "metadata" is, or any technical details; they just know what they're told ("controller X is no longer available, give them controller Y") 2) Driver support is often "iffy" with such controllers, at least under FreeBSD. FreeBSD SCSI CAM is quite reliable, so that's not the problem. Here's some past evidence of mfi(4) and mpt(4) having problems administrating arrays, or experiencing horrible performance, requiring tuning be done and much troubleshooting: http://wiki.freebsd.org/JeremyChadwick/Commonly_reported_issues 3) You are at the whim of the hardware RAID controller's BIOS. Performance can be affected by bugs in the BIOS, or BIOS bugs can cause you trouble down the road. You have to ask yourself how much you ultimately trust the technical support people at Dell vs. the FreeBSD community. 4) Driver regressions may hurt you. There may be a day when you go to upgrade to FreeBSD 8.0 (when it becomes stable), only to find that your controller isn't recognised, or has odd problems. (I myself just ran into this situation with -CURRENT last week, where my SATA controller isn't detected, while works perfectly in RELENG_7). You're then "stuck" on an older FreeBSD until those problems can be worked out. The only hardware RAID controller I've seen praise for, under FreeBSD, are Areca controllers. I'm told the performance (on a purely general level) is "absolutely incredible/blazing fast". I don't know what those people are comparing against, though. Be aware that many developers, including folks like Matt Dillon (of DragonflyBSD) and Ade Lovett (very familiar with filers and disk storage) recommend you *completely avoid* hardware RAID controllers or on-motherboard RAID (e.g. Intel MatrixRAID), and go with OS-based RAID (ZFS, gvinum, or standalone UFS2+SU filesystems). If you reach a point where disk I/O on that server is becoming so heavy that you feel you need a hardware RAID controller, that would be when you should come back to the list (freebsd-stable, freebsd-hardware, or freebsd-isp) to discuss the problems you're having with performance. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |