From owner-freebsd-net@FreeBSD.ORG Tue Jan 4 22:22:31 2005 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8424316A4CF for ; Tue, 4 Jan 2005 22:22:31 +0000 (GMT) Received: from mail2.dbitech.ca (radius.wavefire.com [64.141.13.252]) by mx1.FreeBSD.org (Postfix) with SMTP id D850543D1F for ; Tue, 4 Jan 2005 22:22:30 +0000 (GMT) (envelope-from darcy@wavefire.com) Received: (qmail 10522 invoked from network); 4 Jan 2005 23:35:47 -0000 Received: from dbitech.wavefire.com (HELO ?64.141.15.253?) (darcy@64.141.15.253) by radius.wavefire.com with SMTP; 4 Jan 2005 23:35:47 -0000 From: Darcy Buskermolen Organization: Wavefire Technologies Corp. To: freebsd-net@freebsd.org Date: Tue, 4 Jan 2005 14:22:29 -0800 User-Agent: KMail/1.6.2 References: <7632915A8F000C4FAEFCF272A880344165164F@Ehost067.exch005intermedia.net> <69A7A6D4-5E85-11D9-ACFF-000A95C705DC@chittenden.org> In-Reply-To: <69A7A6D4-5E85-11D9-ACFF-000A95C705DC@chittenden.org> MIME-Version: 1.0 Content-Disposition: inline Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <200501041422.29131.darcy@wavefire.com> cc: Jeff Behl cc: freebsd-performance@freebsd.org Subject: Re: %cpu in system - squid performance in FreeBSD 5.3 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Jan 2005 22:22:31 -0000 On January 4, 2005 11:18 am, Sean Chittenden wrote: > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU CPU > > COMMAND > > 474 squid 96 0 68276K 62480K select 0 53:38 16.80% 16.80% > > squid > > 311 bind 20 0 10628K 6016K kserel 0 12:28 0.00% 0.00% > > named > > > > It's actually so good that one machine can now handle all traffic > > (around 180 Mb/s) at < %50 cpu utilization. Seems like something in > > the > > network stack is responsible for the high %system cpu util... > > OH!!!! Wow, I should've noticed that earlier. Your hint about someone > on Linux having the same problem tipped me off to look at the process > state again. Anyway, you nearly answered your own question, save you > probably aren't familiar with select(2)'s lack of scalability. Read > these: > > http://www.kegel.com/c10k.html > > > Specifically the four methods mentioned here: > > http://www.kegel.com/c10k.html#nb.select > > > Then look at the benchmarks done using libevent(3): > > http://www.monkey.org/~provos/libevent/ > > > Dime to dollar you're spending all of your time copying file descriptor > arrays in and out of the kernel because squid uses select(2) instead of > kqueue(2). Might be an interesting project for someone to take up to > convert that to kqueue(2). Until then, any local TCP load balancer > that uses kqueue(2) would also solve your problem (I'm not aware of any > off the top of my head... pound(8) does, but it is only used for HTTP > and is not a reverse proxy) and would likely prevent you from having > your problems. -sc squid-dev has kqueue support already. ./configure --disable-select --disable-poll --enable-kqueue -- Darcy Buskermolen Wavefire Technologies Corp. ph: 250.717.0200 fx: 250.763.1759 http://www.wavefire.com