From owner-freebsd-performance@FreeBSD.ORG Mon Jan 5 00:48:11 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7183416A4CE for ; Mon, 5 Jan 2004 00:48:11 -0800 (PST) Received: from geminix.org (gen129.n001.c02.escapebox.net [213.73.91.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id EE97343D2D for ; Mon, 5 Jan 2004 00:48:09 -0800 (PST) (envelope-from gemini@geminix.org) Message-ID: <3FF924C6.7040902@geminix.org> Date: Mon, 05 Jan 2004 09:48:06 +0100 From: Uwe Doering Organization: Private UNIX Site User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.5) Gecko/20031206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: <200401041052.i04AqLPk047912@xs1.xs4all.nl> In-Reply-To: <200401041052.i04AqLPk047912@xs1.xs4all.nl> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Received: from gemini by geminix.org with asmtp (TLSv1:AES256-SHA:256) (Exim 3.36 #1) id 1AdQPA-000Pac-00; Mon, 05 Jan 2004 09:48:08 +0100 Subject: Re: slowdown on internal harddisks X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Jan 2004 08:48:11 -0000 Cor Bosman wrote: > Hi all, im running a large usenet setup (30+ servers, 20TB or so) and am > running into some trouble with one of the servers. > > It's a dual Xeon 2.6, 4GB memory, mirrored 2 x 17GB / on ADAPTEC 2015S (asr) > extra 70GB disk on the same asr controller, and 3 x 1.5 TB on > Qlogic ISP 2300 PCI FC-AL Adapter (isp) set up as 6 750GB partitions. > > This server pushes out about 300mbit of data on an Intel Pro 1000 (em). > Im also using a 1GB MFS which i need to support a very high amount of seeks > on a usenet history file (300 MB used). > > As the server starts doing lots of io on the Qlogic controller all disk writes > seem to drop to about 0. Also on the MFS, and on the internal asr controller. > Copying a file from any disk to any other, even to the unused extra 70GB > disk has a max throughput of about 50KB/s. Read performance is still ok, it's > just write()s that are slow. (cp-ing to /dev/null is fast). Try increasing the sysctl(8) variables 'vfs.hirunningspace' and 'vfs.lorunningspace', for instance to '8388608' and '6291456', respectively. Lots of pending read operations at the same time tend to throttle write operations. This is by (kernel) design in order to give priority to read operations. While the (lower) default values appear to work well with conventional "dumb" disk controllers we have found that they are insufficient for more sophisticated caching disk controllers. The effect of too low settings would be exactly what you're seeing: Good read response, but anemic write performance. Maybe that tweak helps in your case, too. Uwe -- Uwe Doering | EscapeBox - Managed On-Demand UNIX Servers gemini@geminix.org | http://www.escapebox.net