From owner-freebsd-fs@FreeBSD.ORG Thu Jul 10 15:55:19 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BBB77106566C for ; Thu, 10 Jul 2008 15:55:19 +0000 (UTC) (envelope-from juri_mian@yahoo.com) Received: from n68.bullet.mail.sp1.yahoo.com (n68.bullet.mail.sp1.yahoo.com [98.136.44.44]) by mx1.freebsd.org (Postfix) with SMTP id AB2E98FC1D for ; Thu, 10 Jul 2008 15:55:19 +0000 (UTC) (envelope-from juri_mian@yahoo.com) Received: from [216.252.122.217] by n68.bullet.mail.sp1.yahoo.com with NNFMP; 10 Jul 2008 15:55:18 -0000 Received: from [69.147.65.166] by t2.bullet.sp1.yahoo.com with NNFMP; 10 Jul 2008 15:55:18 -0000 Received: from [127.0.0.1] by omp501.mail.sp1.yahoo.com with NNFMP; 10 Jul 2008 15:55:18 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 867558.34047.bm@omp501.mail.sp1.yahoo.com Received: (qmail 96804 invoked by uid 60001); 10 Jul 2008 15:55:18 -0000 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=s1024; d=yahoo.com; h=Received:X-Mailer:Date:From:Reply-To:Subject:To:Cc:In-Reply-To:MIME-Version:Content-Type:Message-ID; b=EC0rfusi1hdLwcTJugoFYxJozS/AkQYnTeKbG4XHXeNnqIHM0zUQ3IzOaNihLQ49gHZT/qwwPQIuBk37EJGzPGZ3kR0/w4wQJh1INl5KFXoCXq/gog6At8qOkmbeGeLNAcE4huiv/mopn0npow1HMZ+DqLfARb7q1omUlz6czts=; Received: from [71.63.232.32] by web45607.mail.sp1.yahoo.com via HTTP; Thu, 10 Jul 2008 08:55:17 PDT X-Mailer: YahooMailWebService/0.7.199 Date: Thu, 10 Jul 2008 08:55:17 -0700 (PDT) From: Juri Mianovich To: Alexandre Biancalana , Jeff Mohler In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Message-ID: <223496.96060.qm@web45607.mail.sp1.yahoo.com> X-Mailman-Approved-At: Thu, 10 Jul 2008 16:12:44 +0000 Cc: freebsd-fs@freebsd.org Subject: Re: 24 TB UFS2 reality check ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: juri_mian@yahoo.com List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 15:55:19 -0000 Jeff, --- On Wed, 7/9/08, Jeff Mohler wrote: > Lets see..a peak of maybe 25-30 random drive IOPS/sec at > 15ms MINIMAL > latency per IO (likely more like 35-40)..gonna be ugly. > > Complicated by normal load IOPS..you could expect it all to > simply > "dissapear" for a day while it reconstructs. Once again, thank you very much - your comments are very helpful. So we've moved from "dangerous" (24 TB with no raid) to "inconvenient" (24 TB with raid 6). Two final questions: 1. What would _you_ do with 24 1 TB disks and a 24 port 3ware card ? Assume an i386, 4 GB machine, and that fsck is workable because of "newfs -i 131072" 2. What number should I ask my vendor (3ware) to do the rebuild calculations ? You are talking about IOPS/s - I think I should ask them how many IOPS/s the card does when rebuilding a 24 disk raid-6 array, and then combine that with the IOPS/s I see in my normal workload. How do you measure IOPS/s in FreeBSD on a running machine ? And, of course, any other comments appreciated. Thanks.