From owner-freebsd-performance@FreeBSD.ORG Thu May 26 18:27:52 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E3EA16A41C for ; Thu, 26 May 2005 18:27:52 +0000 (GMT) (envelope-from pfak@telus.net) Received: from priv-edtnes56.telusplanet.net (outbound01.telus.net [199.185.220.220]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD06443D1D for ; Thu, 26 May 2005 18:27:51 +0000 (GMT) (envelope-from pfak@telus.net) Received: from localhost ([199.185.220.240]) by priv-edtnes56.telusplanet.net (InterMail vM.6.01.04.04 201-2131-118-104-20050224) with ESMTP id <20050526182751.DSDA16662.priv-edtnes56.telusplanet.net@localhost>; Thu, 26 May 2005 12:27:51 -0600 Received: from 64.180.103.26 ( [64.180.103.26]) as user a1b26634@192.168.200.1 by webmail.telus.net with HTTP; Thu, 26 May 2005 11:27:50 -0700 Message-ID: <1117132070.42961526e0922@webmail.telus.net> Date: Thu, 26 May 2005 11:27:50 -0700 From: Peter Kieser To: Benjamin Krueger References: <1117129668.42960bc4b751b@webmail.telus.net> <20050526181747.GF62730@strawberry.seattlefenix.net> In-Reply-To: <20050526181747.GF62730@strawberry.seattlefenix.net> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit User-Agent: Internet Messaging Program (IMP) 3.1-cvs X-Originating-IP: 64.180.103.26 Cc: freebsd-performance@freebsd.org Subject: Re: Performance/lockup issues with FreeBSD as a router X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 May 2005 18:27:52 -0000 -- Peter Kieser pfak@telus.net Quoting Benjamin Krueger : > * Peter Kieser (pfak@telus.net) [050526 10:49]: > > Hello guys, > > > > I'm not quite sure if this is the right list to address this to, as it's > partly > > a performance problem and partly otherwise. > > > > I have a FreeBSD machine acting as a router (doing approx. 15-25Mbit/s of > > traffic (lot's of small packets, about 45,000 pps)), however I'm currently > > running into issues where one, or both of the NICs will stop transmitting > > traffic. When I go onto the machine, and try to ping something I get "No > buffer > > space available" > > > > The nics are if_dc, this is a stock FreeBSD 5.4-RELEASE installation (no > > firewall or anything): > > > > Here's my /etc/sysctl.conf: > > > > net.inet.ip.rtexpire=1800 > > net.inet.ip.rtminexpire=1800 > > kern.maxfiles=32768 > > kern.maxfilesperproc=32768 > > kern.ipc.somaxconn=32767 > > net.inet.tcp.sendspace=256000 > > net.inet.tcp.recvspace=256000 > > kern.ipc.maxsockbuf=2097152 > > net.inet.ip.fastforwarding=1 > > net.inet.tcp.rfc1323=1 > > net.link.ether.inet.max_age=600 > > net.inet.tcp.msl=7500 > > net.inet.ip.fw.dyn_udp_lifetime=10 > > net.inet.ip.fw.dyn_buckets=1024 > > > > And here's my /boot/loader.conf: > > > > kern.ipc.maxsockets="163840" > > kern.maxusers="2048" > > > > Is there anythign I'm overlooking that would be causing the machine to > lockup > > like this? > > > > --Peter > > What does 'netstat -m' say? Is there any reason you explicitely set maxusers > instead of > letting the kernel auto-tune the setting? I would watch your mbuf usage and > when you > find your average and peak usage, configure your mbuf allocation at boot and > see if that > helps. I'd also consider letting FreeBSD auto-tune your MAXUSERS. > > -- > Benjamin Krueger > SysAdmin, CarDomain Network > 92 Toyota Turbo MR2 > 78 Datsun B-210 > 91 freakin Geo Prizm > I forgot to mention that mbuf's aren't exhausted, here's the reporting on mbuf's when the machine's NICs fail to work: $ netstat -m 450 mbufs in use 448/132096 mbuf clusters in use (current/max) 0/2/33280 sfbufs in use (current/peak/max) 1008 KBytes allocated to network 0 requests for sfbufs denied 0 requests for sfbufs delayed 0 requests for I/O initiated by sendfile 0 calls to protocol drain routines