Date: Thu, 22 Oct 1998 13:52:16 -0600 (MDT) From: "Kenneth D. Merry" <ken@plutotech.com> To: mlnn4@oaks.com.au Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: Multi-terabyte disk farm Message-ID: <199810221952.NAA17707@panzer.plutotech.com> In-Reply-To: <199810221320.XAA07482@mail.aussie.org> from Hallam Oaks at "Oct 22, 98 11:19:28 pm"
next in thread | previous in thread | raw e-mail | index | archive | help
Hallam Oaks wrote... > G'Day; > > I've a few things I'd like, If I may, to get opinions on. > > My current client (I'm a contractor) hired me to provide them with some > assistance on a project that involves the digitising and non-real-time > satellite transmission of television commercials. As part of this project, > they want to store about 30,000 ads (they call them 'dubs') online for > retrieval in a digital form. > > We estimate that we will need approximately 3-4 terabytes of storage to > achieve this aim. Sounds interesting. :) [ ... ] One thing you might want to take a look at is a Pluto VideoSpace box. Check out: http://www.plutotech.com The largest VideoSpace boxes can currently hold 4 hours of standard def video. It may or may not be what you're looking for, but it does run FreeBSD. :) (yes, I work for them...) > Needless to say I'm going to put my preferred solution as FreeBSD-based. > Some of the criteria that I can't yet answer and would like feedback on > are these - > > o is FreeBSD able to be made to recognise a new SCSI drive that wasn't > present on boot ? i.e. a new drive is plugged into the hot bays. can > it be recognised, formatted, and mounted by manual intervention ? > > o ditto if a drive fries. can it be taken out without the kernel getting > too upset ? These two are much easier to do under CAM. It's also much more robust under CAM. > o is it feasable to automatically umount and spin down drives that > haven't been accessed for a day or so ? typically, the older data > (> 6 months) will be rarely, if ever, accessed before its two-year > span expires and it's erased. Yes. There were some CAM bugs that prevented this from working, but they were fixed prior to 3.0 release. You can probably just spin the disk down, and when someone wants data from it, they can just access the data and the disk will be automatically spun up. > o would the boot time of a system be dramatically prolonged by it having > 500 or so gigabytes of SCSI drives hanging off its backside ? (I'm > referring to a normal boot, with the drives having been properly > unmounted. I don't even want to THINK about waiting on an fsck of 500 > gigs of unclean disk ;). OTOH the size of the files is quite large, so > it'd be feasable to use huge nodes. Well, if you're talking about a CAM system, it really depends on whether the drives are spun up or not. CAM will spin up 4 at a time (by default, it's tuneable) and the boot probably won't happen until they're all spun up. If the drives are already spinning, the boot will happen quite fast. All busses are probed in parallel on a CAM system, so instead of the boot time being (number of busses) * (bus settle delay) as it was with the old SCSI code, it's now just (bus settle delay). > I've no particular objection to a reboot if need be to add/remove a > drive, but if it took more than, say, 10 minutes, it'd be an issue I'd > have to tackle with management. > > o we're thinking of using Seagate Elite 47gb drives. These are 5400 RPM > units (speed isn't an issue to us). Does anyone have any opinions > about these (good/bad/indifferent) or of previous members of that > drive family ? I don't know much about them, but I'd be more comfortable going with some of their 3.5" drives (barracuda or cheetah). IBM is another good drive vendor. Another thing is that (I think) they're phasing those drives out of production. I think that's the last in their line of 5.25" full height disks. > o does anyone have an opinion as to whether it's safe to assume that > drive prices will continue to fall as they have done over the past > two years ? I think it's a reasonably safe assumption. > I guess my main interest, despite the above questions, is really to hear > if others think this is a realistic goal to be aiming for, and if others- > who-have-gone-before-me have struck major problems doing the same (or a > similar) thing. Satoshi is probably the guy to ask. As someone else pointed out, he's one of the people behind the tertiary disk project at Berkeley. Ken -- Kenneth Merry ken@plutotech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199810221952.NAA17707>