From owner-freebsd-isp@FreeBSD.ORG Sun Apr 6 10:47:30 2003 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A180837B401; Sun, 6 Apr 2003 10:47:30 -0700 (PDT) Received: from out001.verizon.net (out001pub.verizon.net [206.46.170.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id A2BF943FBD; Sun, 6 Apr 2003 10:47:29 -0700 (PDT) (envelope-from cswiger@mac.com) Received: from mac.com ([129.44.43.88]) by out001.verizon.net (InterMail vM.5.01.05.27 201-253-122-126-127-20021220) with ESMTP id <20030406174726.LQJG19613.out001.verizon.net@mac.com>; Sun, 6 Apr 2003 12:47:26 -0500 Message-ID: <3E906828.8060001@mac.com> Date: Sun, 06 Apr 2003 13:47:20 -0400 From: Chuck Swiger Organization: The Courts of Chaos User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.4a) Gecko/20030401 X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-isp@freebsd.org References: <20030406120112.I11095-100000@netmint.com> In-Reply-To: <20030406120112.I11095-100000@netmint.com> X-Enigmail-Version: 0.74.1.0 X-Enigmail-Supports: pgp-inline, pgp-mime Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Authentication-Info: Submitted using SMTP AUTH at out001.verizon.net from [129.44.43.88] at Sun, 6 Apr 2003 12:47:25 -0500 cc: Support cc: freebsd-stable@freebsd.org Subject: Re: load testing and tuning a 4GB RAM server X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Apr 2003 17:47:31 -0000 Support wrote: [ ... ] > Now my question. I am trying to stress test a new Dell PowerEdge server > and find the limits of its hardware and my tuning. Here are the server > stats: > > * 2x Xeon 2.8 with SMP compiled, hyperthreading NOT compiled in kernel > * 4 GB of RAM, 8 GB of swap on Raid 1 (15K RPM drives) for speed > * User Data is on Raid 5 in a detached PowerVault Given your planned usage, and the type of hardware you've got, I'd like to redirect this thread to freebsd-isp as being more appropriate. Also, your configuration is suited for where the users will be doing mostly reads and not a lot of writes. If the users might be doing a lot of writes, you should use RAID-1,0 instead of RAID-5. > Contents of /boot/loader.conf: > > kern.ipc.maxsockets=32768 > kern.ipc.nmbclusters=32768 > kern.ipc.nmbufs=131072 > kern.ipc.nsfbufs=6656 > kern.ipc.shm_use_phys=1 > kern.maxfiles=32768 > kern.maxproc=8192 > kern.maxswzone=33554432 > kern.nbuf=16384 > kern.ncallout=32768 > kern.vm.kmem.size=268435456 > kern.vm.pmap.shpgperproc=2048 > net.inet.tcp.tcbhashsize=16384 > > Contents of /etc/sysctl.conf: > > kern.ip.somaxconn=4096 > vm.v_free_min=131072 > vm.v_free_target=262144 > vm.f_free_reserved=32768 > vm.v_free_severe=65536 > kern.ps_showallprocs=0 Given the amount of memory you have, things like ipc.shm_use_phys and vfs.vmiodirenable are a good idea. If this machine is primarily using the network for web traffic, see the section in "man tuning" about net.inet.tcp.sendspace and net.inet.tcp.recvspace in particular. However, I wouldn't change the vm sysctl settings blindly-- the VM system in FreeBSD does a really good job of self-tuning. > Some info from kernel config: > > maxusers 0 > options KVA_PAGES=512 > options NSWAPDEV=1 > (the rest is pretty much my hardware, firewall and other irrelevant stuff) However, possibly with a degree of self-contradictory advice :-), I'd set maxusers to a specific value like 256 or so. The notion of tuning kernel variables to the amount of memory on the _build_ system-- which is not always the same as the system which will be running the kernel-- doesn't strike me as the right thing to do, so I tend to set this to 64 or so by default, and then change it on a case-by-case basis as the need arises. Also, increase NSWAPDEV to at least two, so you at least have a possibility of adding more swap to a running system or for adding some in order to take down the primary swap area for some reason. [ ... ] > Any ideas? Comments? Is my tuning decent, or would you change anything > else? This server is to be a very loaded web/database server. How many simultaneous users do you expect? WWW hits per day, or pageviews (if you can distinguish)? Are you going to be using SSL, lots of virtual domains, any special apache modules? Logfile analysis needs? Also, what are you doing with the database; that is to say, which DB software are you going to use, and what is the DB to be used for? Is the data sensitive or important? If so, having a seperate DB server which is not Internet accessible but on a small local subnet (talking to a second NIC in the WWW server) would significantly improve the security aspects, and would also help balance site load in several regards. Specificly, disk I/O contention is likely to happen if the DB is expected to see any significant usage while the drives are busy serving up user files. That's aside from the fact that you really don't want to keep database files on RAID-5 storage in the first place, either-- keep the DB on RAID-1 or RAID-1,0 if you can: use some of that 15K swapspace if you need to. -- -Chuck