Date: Fri, 22 Apr 2005 13:45:29 -0500 From: "Edgar Martinez" <emartinez@crockettint.com> To: "'Benson Wong'" <tummytech@gmail.com> Cc: freebsd-questions@freebsd.org Subject: RE: 5.8TB RAID5 SATA Array Questions - UPDATE Message-ID: <20050422184523.B03F434D23@mxc1.crockettint.com> In-Reply-To: <860807bf0504221140590ac52f@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Are you booting to the array? Is it over 2TB? Or are you mounting the array? -----Original Message----- From: Benson Wong [mailto:tummytech@gmail.com] Sent: Friday, April 22, 2005 1:40 PM To: emartinez@crockettint.com Cc: Nick Pavlica; Dan Nelson; Nick Evans; freebsd-questions@freebsd.org Subject: Re: 5.8TB RAID5 SATA Array Questions - UPDATE Hi Edgar, Good to hear you finally got it running. Sounds like you went through the same challenges I went through. I wound up getting FreeBSD 5.4-STABLE running and it's been stable for weeks. I put it through quite a bit of load lately and it seems to running well. Comments below: > > As much loved as BSD is to me.it simply just isn't up to the challenge at > all.its far too difficult to get in a properly working state.and the > limitations imposed are just too difficult to overcome easily. Sounds like you hit the same 2TB limit on both FBSD and Linux. What was the limitations that were too difficult to overcome? > > I ended up using Ubuntu which not only had all the driver support to all > the > devices and controllers.but also had little to no problem getting the > system > installed properly.It however does not like/want to boot to the array.so I > installed additional drives (Seagate sata) and created a mirror (300GB) for > the system to live on and bring up the array (/dev/md0) using mdadm.overall > it was easy and nice.there are several caveats left to wrestle with. I wonder why it wouldn't want to boot off of your large array. It could be that it is way too big for the old PC bios to recognize. I think you could get around this by creating a small partition at the beginning of your array. I tried this too, but no luck. My arrays were over fiber channel but that should have been taken care of by the FC card. > > Currently although the 3ware controller can create a huge 4TB raid5 array, > nothing exists that I am aware of that can utilize the entire container. > Every single OS that exists seems to all share the 2TB limitations..so > while > the BIOS can "see" it.everything else will only see 2TB..this includes NFS > on OSX (which don't get me started on the horrible implementation mistakes > from apple and their poor NFS support..i mean NFSv4 comeon! Why is that > hard!!) That is strange that OSX can't see larger than 2TB partitions over NFS. I would assume that an OSX client talking to an XServe would be able to see it. I haven't tested this so I wouldn't know for sure. I'm more curious about the 2TB limit on Linux. I figured Linux, with it's great file system support, would be able to handle a larger than 2TB partition. What were the limitations you ran into? > So to get past Ubuntu's 2TB problem, I created 2xRAID5 2TB (1.8TB > reporting) > containers on the array.and then using software raid.created 1xRAID0 using > the 2xRAID5 containers.which create 1xRAID0 @4TB. Why did software raid0 help you get over the 2TB limitation? Wouldn't it still appear as one filesystem that is way too big to use? Something doesn't add up here. Pun not intended. :) > > Utterly horrible.probably the WORST half-assed installation imaginable.in > my > honest opinion.here are my desires. > I chose to break my 4.4TB system into 4 x 1.1TB arrays. This is very well supported by FreeBSD. The downside is that I had to modify my email system configuration and maintenance scripts to work with four smaller arrays rather than a single large one. I purposely avoided using software raid because it makes maintenance of the array a lot more complex. It usually doesn't take a lot of skills or time to fix a hardware array but the learning curve for fixing a software array is a lot higher. Plus I don't think software raid on linux is any good, or on FreeBSD for that matter. > Create 1xRAID5 @ 4TB.install the OS TO the array.boot to the array and then > share out 4TB via NFS/SMB.was that too much to ask?? Obviously it was. > > So in response.I can modified the requirements. > > Create 1xRAID5@4TB...install an OS TO a 1xRAID1@300GB...BOOT to the > RAID1..and SHARE out the 4TB. This is essentially what I did as well. Didn't know about the limitations when I first started. ben
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20050422184523.B03F434D23>