From owner-freebsd-fs@FreeBSD.ORG Thu Jul 10 17:25:25 2008 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8BF23106567A for ; Thu, 10 Jul 2008 17:25:25 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from noop.in-addr.com (in-addr.broker.freenet6.net [IPv6:2001:5c0:8fff:fffe::214d]) by mx1.freebsd.org (Postfix) with ESMTP id 43AF18FC25 for ; Thu, 10 Jul 2008 17:25:25 +0000 (UTC) (envelope-from gpalmer@freebsd.org) Received: from gjp by noop.in-addr.com with local (Exim 4.54 (FreeBSD)) id 1KGztS-0005Hg-PN; Thu, 10 Jul 2008 13:25:22 -0400 Date: Thu, 10 Jul 2008 13:25:22 -0400 From: Gary Palmer To: Juri Mianovich Message-ID: <20080710172522.GA92945@in-addr.com> References: <223496.96060.qm@web45607.mail.sp1.yahoo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <223496.96060.qm@web45607.mail.sp1.yahoo.com> Cc: freebsd-fs@freebsd.org Subject: Re: 24 TB UFS2 reality check ? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Jul 2008 17:25:25 -0000 On Thu, Jul 10, 2008 at 08:55:17AM -0700, Juri Mianovich wrote: > > Jeff, > > > --- On Wed, 7/9/08, Jeff Mohler wrote: > > > Lets see..a peak of maybe 25-30 random drive IOPS/sec at > > 15ms MINIMAL > > latency per IO (likely more like 35-40)..gonna be ugly. > > > > Complicated by normal load IOPS..you could expect it all to > > simply > > "dissapear" for a day while it reconstructs. > > > Once again, thank you very much - your comments are very helpful. > > So we've moved from "dangerous" (24 TB with no raid) to "inconvenient" (24 TB with raid 6). > > Two final questions: > > 1. What would _you_ do with 24 1 TB disks and a 24 port 3ware card ? > Assume an i386, 4 GB machine, and that fsck is workable because of > "newfs -i 131072" > > 2. What number should I ask my vendor (3ware) to do the rebuild > calculations ? You are talking about IOPS/s - I think I should ask > them how many IOPS/s the card does when rebuilding a 24 disk raid-6 > array, and then combine that with the IOPS/s I see in my normal workload. > At least on older 3Ware cards, and I suspect the one you're talking about will do it also, you can control the rebuild rate. There are 5 different options provided on my card (8506-4LP) for "background task rate" which either increases or decreases the IOPS used for the rebuild. > How do you measure IOPS/s in FreeBSD on a running machine ? iostat and/or systat are the two I use. Regards, Gary