From owner-freebsd-questions@FreeBSD.ORG Fri Apr 22 16:46:08 2005 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 86ED216A4CE for ; Fri, 22 Apr 2005 16:46:08 +0000 (GMT) Received: from mxc1.crockettint.com (chameleon1.crockettint.com [206.224.72.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8482443D4C for ; Fri, 22 Apr 2005 16:46:05 +0000 (GMT) (envelope-from emartinez@crockettint.com) Received: from localhost (localhost.crockettint.com [127.0.0.1]) by mxc1.crockettint.com (Postfix) with ESMTP id 2430B34D03; Fri, 22 Apr 2005 11:46:04 -0500 (CDT) Received: from mxc1.crockettint.com ([127.0.0.1]) by localhost (mxc1.crockettint.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 92126-07; Fri, 22 Apr 2005 11:45:51 -0500 (CDT) Received: from memnoch (ausc1.crockettint.com [24.153.230.82]) by mxc1.crockettint.com (Postfix) with ESMTP id 30B8434D02; Fri, 22 Apr 2005 11:45:50 -0500 (CDT) From: "Edgar Martinez" To: "'Nick Pavlica'" Date: Fri, 22 Apr 2005 11:45:55 -0500 Organization: Crockett International MIME-Version: 1.0 X-Mailer: Microsoft Office Outlook, Build 11.0.6353 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2527 Thread-Index: AcVB3rl2vJCWuIbwSGuCo9dAEtZstAFebXJg In-Reply-To: Message-Id: <20050422164550.30B8434D02@mxc1.crockettint.com> X-Virus-Scanned: amavisd-new at crockettint.com Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.1 cc: 'Nick Evans' cc: 'Benson Wong' cc: 'Dan Nelson' cc: freebsd-questions@freebsd.org Subject: RE: 5.8TB RAID5 SATA Array Questions - UPDATE X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: emartinez@crockettint.com List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Apr 2005 16:46:08 -0000 All, So, after a soild chunk of life has fully been drained from me.here are several conclusions.obviously open for discussion if anyone wants to pick my brain.(yes we reduced our array size.you'll see why) As much loved as BSD is to me.it simply just isn't up to the challenge at all.its far too difficult to get in a properly working state.and the limitations imposed are just too difficult to overcome easily. I ended up using Ubuntu which not only had all the driver support to all the devices and controllers.but also had little to no problem getting the system installed properly.It however does not like/want to boot to the array.so I installed additional drives (Seagate sata) and created a mirror (300GB) for the system to live on and bring up the array (/dev/md0) using mdadm.overall it was easy and nice.there are several caveats left to wrestle with. Currently although the 3ware controller can create a huge 4TB raid5 array, nothing exists that I am aware of that can utilize the entire container. Every single OS that exists seems to all share the 2TB limitations..so while the BIOS can "see" it.everything else will only see 2TB..this includes NFS on OSX (which don't get me started on the horrible implementation mistakes from apple and their poor NFS support..i mean NFSv4 comeon! Why is that hard!!) So to get past Ubuntu's 2TB problem, I created 2xRAID5 2TB (1.8TB reporting) containers on the array.and then using software raid.created 1xRAID0 using the 2xRAID5 containers.which create 1xRAID0 @4TB. So.samba allows clients to "see" <2TB and FTP also allow for <2TB..this is probably the most complicated.and yet simple thing I can say I have done. Utterly horrible.probably the WORST half-assed installation imaginable.in my honest opinion.here are my desires. Create 1xRAID5 @ 4TB.install the OS TO the array.boot to the array and then share out 4TB via NFS/SMB.was that too much to ask?? Obviously it was. So in response.I can modified the requirements. Create 1xRAID5@4TB...install an OS TO a 1xRAID1@300GB...BOOT to the RAID1..and SHARE out the 4TB. Any one else think they know of a better method?? _____ From: Nick Pavlica [mailto:linicks@gmail.com] Sent: Friday, April 15, 2005 12:15 PM To: emartinez@crockettint.com Cc: Dan Nelson; Nick Evans; Benson Wong; freebsd-questions@freebsd.org Subject: Re: 5.8TB RAID5 SATA Array Questions On 4/15/05, Edgar Martinez wrote: OK...so now we are going into some new territory...I am curious if you would care to elaborate a bit more...I am intrigued...if anyone wants me to do some experiments or test something, let me know...I for one welcome any attempts at pushing any limits or trying new things... I would help do some testing but I don't have any storage that large at the moment. I curious how 5.4RC2 or > handles very large volumes. Have you already tried fdisk, newfs ? --Nick