From owner-freebsd-performance@FreeBSD.ORG Tue Feb 3 09:29:33 2004 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 44F2C16A4CE for ; Tue, 3 Feb 2004 09:29:33 -0800 (PST) Received: from mail.ubergeeks.com (lorax.ubergeeks.com [209.145.65.55]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6519243D2D for ; Tue, 3 Feb 2004 09:29:31 -0800 (PST) (envelope-from adrian+freebsd-perf@ubergeeks.com) Received: from mail.ubergeeks.com (localhost [127.0.0.1]) by mail.ubergeeks.com (8.12.9p2/8.12.9) with ESMTP id i13HQJqZ025727 for ; Tue, 3 Feb 2004 12:26:19 -0500 (EST) (envelope-from adrian+freebsd-perf@ubergeeks.com) Received: from localhost (adrian@localhost)i13HQJxC025724 for ; Tue, 3 Feb 2004 12:26:19 -0500 (EST) (envelope-from adrian+freebsd-perf@ubergeeks.com) X-Authentication-Warning: lorax.ubergeeks.com: adrian owned process doing -bs Date: Tue, 3 Feb 2004 12:26:19 -0500 (EST) From: Adrian Filipi Sender: adrian@ubergeeks.com To: freebsd-performance@freebsd.org Message-ID: <20040201184719.S3066@lorax.ubergeeks.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-MailScanner-Information: Please contact the ISP for more information X-MailScanner: Found to be clean Subject: frustratingly slow box at 4GB, but not 1GB of memory X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Feb 2004 17:29:33 -0000 Hi folks, I've got a box I'm helping tune and it's not really responding as I'd expect. It's a dual 2GHz Xeon box running on a Supermicro P4DLR motherboard with 4GB of physical RAM. It has 4GB of swap configured on a hardware RAID5 configuration. It's running 4.9-RELEASE. It's also running Apache with PHP and Perl CGI's. The CGI's read/write to a MySQL DB. Briefly, it responds pretty quickly when I only let the kernel use 1GB of the RAM, but gets bogged down when I let it use the full 4GB. I'm limiting the RAM by setting hw.physmem. I've tried some of the things that I could dig up in the archives, but the knowledge is pretty widely scattered. :-( I have MAXUSERS set to 96 as that gets me plenty of file descriptors and pid's. While tight, I believe I could get away with only 32, as there should not be more than 300-400 processes running at any one time. I set vm.pmap.shpgperproc="300" because of warnings from the kernel. I'm not sure if this is the best value, but it made the warnings go away. First, upping the KVA_PAGES to 768 to achieve a 3:1 ration of kernel to user memory, didn't change things. Disabling swap seems to make a huge difference. It changes the time results for "time top -bu" by two orders of magnitude from 0.006s to 0.6s! In fact, I can run the box reasonably well with 2GB of RAM with swap disabled. If I let the box boot using all of the RAM, it is very slow. e.g. "time top -bu >/dev/null" takes about 6 seconds. I am seeing large runs of time (15-30 seconds) where the system time usage is in the high range (75-99). Does anyone have a suggestion on how to determine which kernel data structures are the problems? thanks, Adrian -- [ adrian@ubergeeks.com ]