Date: Thu, 22 Oct 1998 12:47:51 -0500 (CDT) From: Joe Greco <jgreco@solaria.sol.net> To: hackers@FreeBSD.ORG Cc: mlnn4@oaks.com.au Subject: Re: Multi-terabyte disk farm Message-ID: <199810221747.MAA26363@aurora.sol.net>
next in thread | raw e-mail | index | archive | help
> We estimate that we will need approximately 3-4 terabytes of storage to > achieve this aim. > Secondly (and this is a major call I'm making), it won't work out cheaper > for our estimated need of three terabytes unless the cost of HDD's keep on > falling. We won't need full capacity until about two years has passed, > meaning that we can start out with only a few hundred gig and scale it as > we go, taking advantage of (hopefully) falling disk prices and increasing > drive sizes. The latest I've heard is that you shouldn't expect much of a slowing until the Big Date... that is, the capacities will keep increasing and the price will keep falling until after 2000. After that point, it starts to get more speculative, due to the fact that you begin to reach some theoretical density barriers, although new technology is expected to help there. With 18GB drives going for less than $1000, and 47GB Elites for about $2500, there are lots of opportunities. This comes out to $56k per terabyte at today's prices. If I was going to do this, here's the general idea of what I would do: 1) Figure out what I really need to buy and support _today_. If the immediate need is only for a quarter of a terabyte of disk space, with another quarter terabyte every three months, then you can make some cost predictions that may show the disk-based system as being less expensive. 2) Build inexpensive, low-end fileservers that talk to a large number of drives (since speed isn't a critical issue). Look at using vinum and the raid5 support, which will mildly increase the number of disks required, but will improve your reliability. It's not clear to me whether or not we support more than 32 drives yet... but even a machine hosting only 30 18GB drives (two 15-drive fast/wide SCSI chains) is 540GB, and 30 47GB drives is 1410GB. If you do the raid5 thing, I would recommend clustering drives in a four- plus-one pattern that gives you three filesystems of 72 or 168GB each per SCSI chain. This effectively limits the amount of data you need to fsck, and the amount of potential loss in case of catastrophic failure. It does drive the price up a bit. You could build a 432GB RAID5-capable array for about $32,000, PC included, but you have to figure out how to house the drives. You may not want to start out using vinum, which is relatively new and untested. 3.0R and the CAM subsystem appear to offer a lot of things that would be handy to you, including the ability to probe new devices on a SCSI bus. My largest filesystems are only in the 108GB and 72GB ranges. They take no time to fsck for a clean boot, but do take a bit for crash recovery. Additional notes: the early Elite 47's had problems. Dunno about the current ones. See the UCB NOW project, they have a terabyte-filesystem project. ... Joe ------------------------------------------------------------------------------- Joe Greco - Systems Administrator jgreco@ns.sol.net Solaria Public Access UNIX - Milwaukee, WI 414/342-4847 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199810221747.MAA26363>