From owner-freebsd-performance@FreeBSD.ORG Wed Apr 21 11:42:48 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F2AF616A4CE for ; Wed, 21 Apr 2004 11:42:47 -0700 (PDT) Received: from gen129.n001.c02.escapebox.net (gen129.n001.c02.escapebox.net [213.73.91.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8411B43D4C for ; Wed, 21 Apr 2004 11:42:47 -0700 (PDT) (envelope-from gemini@geminix.org) Message-ID: <4086C0A3.30407@geminix.org> Date: Wed, 21 Apr 2004 20:42:43 +0200 From: Uwe Doering Organization: Private UNIX Site User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.6) Gecko/20040119 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <20040420195010.GZ87362@nasby.net> <4085869E.7090306@he.iki.fi> <4085AA4B.1020700@geminix.org> <20040421170556.GB41429@nasby.net> In-Reply-To: <20040421170556.GB41429@nasby.net> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Received: from gemini by geminix.org with asmtp (TLSv1:AES256-SHA:256) (Exim 3.36 #1) id 1BGMgH-0009x7-00; Wed, 21 Apr 2004 20:42:46 +0200 Subject: Re: vfs.hirunningspace on a 3ware 8506 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 Apr 2004 18:42:48 -0000 Jim C. Nasby wrote: > On Wed, Apr 21, 2004 at 12:55:07AM +0200, Uwe Doering wrote: >>Petri Helenius wrote: >>>Jim C. Nasby wrote: >>> >>>>Has anyone done any testing to see what value of vfs.hirunningspace is >>>>optimal for a 3ware 8506-8? >>> >>>Do the 3ware controllers actually care about this value due to the >>>onboard processing and cache? I thought all writes are satisfied >>>immediately? >> >>The controller itself doesn't care, but the kernel does. With the >>current implementation, the amount of memory associated with outstanding >>read requests is subtracted from vfs.hirunningspace. With many >>concurrent read requests there is no reserve left for write operations, >>so write performance can suffer substantially. >> >>This balancing effect is actually intended in order to give read >>requests some priority, but in high performance systems with fast, >>caching raid controllers the default value of said variable is too low >>and therefore poses a bottleneck. > > Unfortunately, it seems the 8500 series only has 1.8MB of cache, so it > seems like the out-of-the-box setting of 1M may not be too far off. > > Is it normally advisable to set vfs.hirunningspace = whatever the > controller's cache is? Well, 1.8 MB is not much. The Adaptec controllers we use have 16 MB buffer space, so I set 'vfs.hirunningspace' to half of that (8 MB). Basically by the seat of my pants. My thinking was that in cases where all of this amount was used for write operations the kernel shouldn't be able to completely flush the controller's buffer in one go. Just a conservative approach. In your case, you'll probably get away with picking a value equal to or even larger than the controller's cache. It might result in a slight performance penalty, but in case your database server really suffers from write starvation due to many concurrent read requests, removing this bottleneck is likely to outweigh that penalty by far. In any case, unless the effect of tweaking 'vfs.hirunningspace' isn't outright spectacular you will probably have to run benchmark tests in order to find the best value for your server. Uwe -- Uwe Doering | EscapeBox - Managed On-Demand UNIX Servers gemini@geminix.org | http://www.escapebox.net