From owner-freebsd-performance@FreeBSD.ORG Tue Sep 2 12:50:10 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F031B16A4BF for ; Tue, 2 Sep 2003 12:50:10 -0700 (PDT) Received: from otter3.centtech.com (moat3.centtech.com [207.200.51.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 020A743FE0 for ; Tue, 2 Sep 2003 12:50:10 -0700 (PDT) (envelope-from anderson@centtech.com) Received: from centtech.com (neutrino.centtech.com [204.177.173.28]) by otter3.centtech.com (8.12.3/8.12.3) with ESMTP id h82Jo9ob095688; Tue, 2 Sep 2003 14:50:09 -0500 (CDT) (envelope-from anderson@centtech.com) Message-ID: <3F54F46E.2070203@centtech.com> Date: Tue, 02 Sep 2003 14:50:06 -0500 From: Eric Anderson User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Max Clark References: In-Reply-To: Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-performance@freebsd.org Subject: Re: FW: 20TB Storage System X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2003 19:50:11 -0000 Max Clark wrote: > Hi all, > > I need to attach 20TB of storage to a network (as low cost as possible), I > need to sustain 250Mbit/s or 30MByte/s of sustained IO from the storage to > the disk. > > I have found external Fibre Channel -> ATA 133 Raid enclosures. These > enclosures will house 16 drives so with 250GB drives a total of 3.5TB each > after a RAID 5 format. These enclosures have advertised sustained IO of > 90-100MByte/s each. > > One solution we are thinking about is to use a Intel XEON server with 3x FC > HBA controller cards in the server each attached to a separate storage > enclosure. In any event we would be required to use ccd or vinum to stripe > multiple storage enclosures together to form one logical volume. > > I can partition this system into two separate 10TB storage pools. > > Given the above: > 1) What would my expected IO be using vinum to stripe the storage enclosures > detailed above? > 2) What is the maximum size of a filesystem that I can present to the host > OS using vinum/ccd? Am I limited anywhere that I am not aware of? > 3) Could I put all 20TB on one system, or will I need two to sustain the IO > required? > 4) If you were building this system how would you do it? (The installed $/GB > must be below $5.00 dollars). > > My other options are to use Solaris or Windows (which I would rather not > do). I can tell you right now I have Solaris and Windows machines attempting file server traffic, and only Solaris even gets in the right realm of speed, but FreeBSD blows them both flat over. Your bottleneck will most likely be the bus speed of the host, so make sure to use PCI-X adapters if possible. Also, how are you sharing this data? NFS? Samba? FTP? Eric -- ------------------------------------------------------------------ Eric Anderson Systems Administrator Centaur Technology All generalizations are false, including this one. ------------------------------------------------------------------