From owner-freebsd-hackers Mon Feb 5 9:51: 4 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id 8410537B67D; Mon, 5 Feb 2001 09:50:42 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f15HoZ021657; Mon, 5 Feb 2001 09:50:35 -0800 (PST) (envelope-from dillon) Date: Mon, 5 Feb 2001 09:50:35 -0800 (PST) From: Matt Dillon Message-Id: <200102051750.f15HoZ021657@earth.backplane.com> To: "Michael C . Wu" Cc: Mitch Collinsworth , hackers@FreeBSD.ORG, fs@FreeBSD.ORG Subject: Re: Extremely large (70TB) File system/server planning References: <20010205100016.C97400@peorth.iteration.net> <20010205112420.A98288@peorth.iteration.net> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :| > The files are accessed approximately 3 or 4 times a day on average. :| > Older files are archived for reference purpose and may never :| > be accessed after a week. :| :| Ok, this is a start. Now is the 70 TB the size of the active files? :| Or does that also include the older archived files that may never be :| accessed again? :70TB is the size of the sum of all files, access or no access. :(They still want to maintain accessibility even though the chances are slim.) :-- :+------------------------------------------------------------------+ :| keichii@peorth.iteration.net | keichii@bsdconspiracy.net | :| http://peorth.iteration.net/~keichii | Yes, BSD is a conspiracy. | :+------------------------------------------------------------------+ This doesn't sound like something you can just throw together with off-the-shelf PCs and still have something reliable to show for it. You need a big honking RAID system - maybe a NetApp, maybe something else. You have to look at the filesystem and file size limitations of the unit and the client(s). FreeBSD can only support 1 TB sized filesystems. Our device layer converts everything to DEV_BSIZE'd (512) blocks, so to be safe: 2^31 x 512 bytes = 1 TB on Intel boxes. Our NFS implementation has the same per-filesystem limitation. Theoretically UFS/FFS are limited to 2^31 x blocksize, where blocksize can be larger (e.g. 16384 bytes, 65536 bytes), but I have grave doubts that that actually works.. I'm fairly certain that we still convert things to 512 byte block numbers at the device level, and we only use a 32 bit int to store the block number. So FreeBSD could be used as an NFS client, but probably not a server for your application. Considering the number of disks you need to manage, something like a NetApp or other completely self contained RAID-5-capable system for handling the disks is mandatory. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message