From owner-freebsd-performance@FreeBSD.ORG Tue Sep 2 13:06:15 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 35BD016A4BF for ; Tue, 2 Sep 2003 13:06:15 -0700 (PDT) Received: from Chow.corp.media.net (rottie.media.net [66.113.65.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1AA0C44014 for ; Tue, 2 Sep 2003 13:06:14 -0700 (PDT) (envelope-from max.clark@media.net) Received: from MCLARK (76.0.6.10.IN-ADDR.ARPA [10.6.0.76]) by Chow.corp.media.net (Netscape Messaging Server 4.15) with SMTP id HKLRMG00.V3N; Tue, 2 Sep 2003 13:01:28 -0700 From: "Max Clark" To: "Eric Anderson" Date: Tue, 2 Sep 2003 13:11:59 -0700 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2911.0) Importance: Normal In-Reply-To: <3F54F46E.2070203@centtech.com> X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 cc: freebsd-performance@freebsd.org Subject: RE: FW: 20TB Storage System X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 02 Sep 2003 20:06:15 -0000 This will be mostly Samba with a little bit of FTP. What about the Raid 0 Stripe to combine the disk shelves, ccd or vinum? What will I get better performance with, what should I expect as I add each shelf? Is there anyone out there with 5+TB of storage configured like this that could share some performance numbers? Thanks, Max -----Original Message----- From: Eric Anderson [mailto:anderson@centtech.com] Sent: Tuesday, September 02, 2003 12:50 PM To: Max Clark Cc: freebsd-performance@freebsd.org Subject: Re: FW: 20TB Storage System Max Clark wrote: > Hi all, > > I need to attach 20TB of storage to a network (as low cost as possible), I > need to sustain 250Mbit/s or 30MByte/s of sustained IO from the storage to > the disk. > > I have found external Fibre Channel -> ATA 133 Raid enclosures. These > enclosures will house 16 drives so with 250GB drives a total of 3.5TB each > after a RAID 5 format. These enclosures have advertised sustained IO of > 90-100MByte/s each. > > One solution we are thinking about is to use a Intel XEON server with 3x FC > HBA controller cards in the server each attached to a separate storage > enclosure. In any event we would be required to use ccd or vinum to stripe > multiple storage enclosures together to form one logical volume. > > I can partition this system into two separate 10TB storage pools. > > Given the above: > 1) What would my expected IO be using vinum to stripe the storage enclosures > detailed above? > 2) What is the maximum size of a filesystem that I can present to the host > OS using vinum/ccd? Am I limited anywhere that I am not aware of? > 3) Could I put all 20TB on one system, or will I need two to sustain the IO > required? > 4) If you were building this system how would you do it? (The installed $/GB > must be below $5.00 dollars). > > My other options are to use Solaris or Windows (which I would rather not > do). I can tell you right now I have Solaris and Windows machines attempting file server traffic, and only Solaris even gets in the right realm of speed, but FreeBSD blows them both flat over. Your bottleneck will most likely be the bus speed of the host, so make sure to use PCI-X adapters if possible. Also, how are you sharing this data? NFS? Samba? FTP? Eric -- ------------------------------------------------------------------ Eric Anderson Systems Administrator Centaur Technology All generalizations are false, including this one. ------------------------------------------------------------------