From owner-freebsd-hackers@FreeBSD.ORG Tue Mar 16 11:07:08 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 732A116A4CE for ; Tue, 16 Mar 2004 11:07:08 -0800 (PST) Received: from mail006.syd.optusnet.com.au (mail006.syd.optusnet.com.au [211.29.132.63]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33CE643D45 for ; Tue, 16 Mar 2004 11:07:07 -0800 (PST) (envelope-from PeterJeremy@optushome.com.au) Received: from cirb503493.alcatel.com.au (c211-30-75-229.belrs2.nsw.optusnet.com.au [211.30.75.229]) i2GJ75w30150 for ; Wed, 17 Mar 2004 06:07:06 +1100 Received: from cirb503493.alcatel.com.au (localhost.alcatel.com.au [127.0.0.1])i2GJ75SU025978 for ; Wed, 17 Mar 2004 06:07:05 +1100 (EST) (envelope-from jeremyp@cirb503493.alcatel.com.au) Received: (from jeremyp@localhost) by cirb503493.alcatel.com.au (8.12.10/8.12.9/Submit) id i2GJ75Oh025977 for freebsd-hackers@freebsd.org; Wed, 17 Mar 2004 06:07:05 +1100 (EST) (envelope-from jeremyp) Date: Wed, 17 Mar 2004 06:07:05 +1100 From: Peter Jeremy To: freebsd-hackers@freebsd.org Message-ID: <20040316190705.GM56509@cirb503493.alcatel.com.au> References: <20040315150438.GA48241@icomag.de> <20040316070601.GK56509@cirb503493.alcatel.com.au> <20040316084225.GA55231@icomag.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20040316084225.GA55231@icomag.de> User-Agent: Mutt/1.4.2i Subject: Re: kernel activity X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Mar 2004 19:07:08 -0000 Please don't top-post. On Tue, Mar 16, 2004 at 09:42:25AM +0100, Bogdan TARU wrote: > Thanks for the mails & advices... The box is a dual xeon @3GHz, with > 4GB of ram and raid 5 on board (scsi HDDs), with a 4.9 on it. Not short of horsepower then. 250K syscalls/sec may not be overly excessive for such a beast. Note that the 4.9 kernel is not re-entrant: only one CPU can be in the kernel at once. This will magnify the impact of high kernel load. > The box has two > NICs, one of them is a fxp with link0 activated (cannot use polling > because I don't want to give up SMP -- the userland activity is > already 40%, so giving up one CPU as to reduce sys load it's just > gonna leave the bottleneck where it is -- CPU, that is), You might be able to use polling with SMP - some people say it works but I think Luigi has some concerns - try rummaging through the archives. > other one is an em, but cannot use it since i don't have a gb > switch. Some em chips can run at 100Mbps - I presume you don't have a suitable interconnect. > Before activating link0 on fxp, the level of interrups/sec on > this interface peaked 6k, but after activating link0 it was reduced > to 2k. Still, a lot of sys activity... Have a look at "systat -v 1" or "top" and see if the system time is really interrupt time. If so, your only real option is a more efficient NIC and/or polling. > As a webserver, I run apache, stripped down from the modules that I > don't need, and compiled in php and some other modules > (statically). Most of the content that I serve is static, there are > only a few php scripts and they don't get much hits. I believe Apache2 can run in a multi-threaded mode rather than the multi-process mode that Apache1 uses. I presume you're not using multi-threading - if you are, you might like to compile it out and see what happens (threading adds quite a number of additional syscalls) > And no, I haven't tried turning HTT on and off, should I do that? The 2nd (virtual) CPU only provides a subset of the main CPU functionality and doesn't help kernel performance at all. It's possible that you will get better performance with it disabled. All I can suggest is trying it to see. > I am also considering trussing one of the apaches, to see what system > calls it's doing... I prefer ktrace(1) but this may be a reasonable idea. If all else fails, a second box with load balancing has the added bonus of providing you with some redundancy if either box fails. Your other possible option is 5.2.1 - 5.x can make much better use of multiple CPUs. If you go down this path, test it well and make sure that 5.x works for you. Peter