From owner-freebsd-hackers Wed May 15 0:24:31 2002 Delivered-To: freebsd-hackers@freebsd.org Received: from clifford.inch.com (ns.biglist.com [216.223.208.40]) by hub.freebsd.org (Postfix) with SMTP id E981237B40A for ; Wed, 15 May 2002 00:24:26 -0700 (PDT) Received: (qmail 30941 invoked by uid 501); 15 May 2002 07:24:25 -0000 Date: Wed, 15 May 2002 03:24:25 -0400 From: Omar Thameen To: Doug White Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: tuning a CPU bound server Message-ID: <20020515032425.A23491@clifford.inch.com> References: <20020514142441.A73151@clifford.inch.com> <20020514211907.W70761-100000@resnet.uoregon.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: <20020514211907.W70761-100000@resnet.uoregon.edu>; from dwhite@resnet.uoregon.edu on Tue, May 14, 2002 at 09:25:21PM -0700 Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Tue, May 14, 2002 at 09:25:21PM -0700, Doug White wrote: > Mmm, mail server tuning, something I have some experience in :-) Just what I was hoping to hear! > First off, what are the specs of the server? Cpu? Disk? Memory? Network? > You mention it's a dual 800MHz. What kind of NIC does it have? What is the > speed and duplex set to on it? Dual PIII/800 2G SDRAM 2x18G IBM 10,000 rpm SCSI drives, running vinum RAID0 because I originally thought disk accesses would be a bottleneck (but they aren't) Tyan Thunder LE S2510 motherboard: sym0,sym1 SCSI controller (on board, one drive on each channel) fxp0 network (on-board), 100M, full-duplex > Secondly, what period was your vmstat run over? The vmstat shows most of > the time spent in system. I'm thinking it might be network-based but > without the specs its hard to tell. Updating every second. It has about the same network I/O as the heavily hit webserver (same # of interrupts/second), but smtp is a lengthier negotiation, right? > While you're getting specs run netstat -m every so often and collect the > mbuf and mbuf cluster utilization numbers. Will do. Since there's no big delivery happening at this moment, I'll follow up later. Last I checked, it wasn't running out of mbufs (kern.ipc.nmbufs: 34816). > qmail is also very inefficient when it comes to large delivery -- the fork > per message and the qmail-remote trigger-hitting will eventually > bottleneck you. It's probable you've run into it. My sympathies. :) You > might try *dropping* concurrencyremote somewhat to reduce the > context-switch thrash (although your context-switch numbers aren't too > high, I've seen worse and the machine wasn't taxed too heavily). I've grown to this concurrencyremote fairly gradually. There was improvement up to about 600 concurrencyremote. I added a 2nd identical qmail queue so the deliveries wouldn't be so serial, but didn't see anywhere near a doubling of the delivery capacity. When only one of the queues is delivering, I see very similar vmstat and throughput values. > It might be interesting to run top and find what the major culprit of cpu > usage is. Somehow I think it'll be qmail-send. It's dnscache (~30%), then the 2 qmail-rspawn processes (~20-25% each). I thought about running dnscache on a separate server, but don't think that will give the performance improvement I was hoping for. Omar > On Tue, 14 May 2002, Omar Thameen wrote: > > > I've got a pretty heavily loaded server which is doing mostly > > mailing list delivery. It happens to be running qmail with a large > > number of concurrencyremotes (about 1000), meaning that there is > > one smtp delivery process spawned for each message. I've hit a > > plateau with regards to the amount of bandwidth that the server > > can push out - I see similar performance with half the concurrencyremotes. > > > > As best as I can tell, the server is CPU bound. My main question > > is whether there are any kernel parameters I can tune to improve > > performance, or whether I just need to get more powerful processors > > (Xeons?). The current system is a dual PIII/800. [...] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message