From owner-freebsd-net Mon Mar 3 15: 1:55 2003 Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 22FC537B401 for ; Mon, 3 Mar 2003 15:01:53 -0800 (PST) Received: from tesla.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5318043FAF for ; Mon, 3 Mar 2003 15:01:52 -0800 (PST) (envelope-from bmilekic@unixdaemons.com) Received: (from bmilekic@localhost) by tesla.distributel.net (8.11.6/8.11.6) id h23N0EG07348; Mon, 3 Mar 2003 18:00:14 -0500 (EST) (envelope-from bmilekic@unixdaemons.com) Date: Mon, 3 Mar 2003 18:00:14 -0500 From: Bosko Milekic To: CHOI Junho Cc: net@freebsd.org Subject: Re: Performance tuning hints of gigabit networking? Message-ID: <20030303180014.A7317@unixdaemons.com> References: <20030226.220551.10329540.cjh@kr.FreeBSD.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <20030226.220551.10329540.cjh@kr.FreeBSD.org>; from cjh@kr.FreeBSD.org on Wed, Feb 26, 2003 at 10:05:51PM +0900 Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org You're not running out of mbufs or clusters, you're out of RAM. Don't bump up nmbclusters anymore because you don't need to; instead, add more RAM. On Wed, Feb 26, 2003 at 10:05:51PM +0900, CHOI Junho wrote: > > Hi, > > I am looking for a good resource for kernel tuning on very high > bandwidth HTTP servers(avg 500Mbit/sec, peak 950Mbit/sec). Today I > faced very unusual situation with 950Mbit/sec bandwidth! > > > netstat -m > 16962/93488/262144 mbufs in use (current/peak/max): > 16962 mbufs allocated to data > 16952/65536/65536 mbuf clusters in use (current/peak/max) > 154444 Kbytes allocated to network (14% of mb_map in use) > 512627 requests for memory denied > 2614 requests for memory delayed > 0 calls to protocol drain routines > > I set kern.ipc.nmbclusters=65536, but it overflowed. This is P-IV Xeon > 1.8G, 2GB RAM, and one Intel 1000baseSX(em driver) machine running > 4.7-RELEASE-pX. This server is running only one service, HTTP. I use > thttpd, since apache doesn't work in such a high load. thttpd is highly > amazing, just give <1 load in any time. > > Once I tried to increase kern.ipc.nmbclusters to 131072 or > higher(multiple of 65536 or 32768, tuning(7) only cites about 32768 > case..), it fails to boot kernel when 262144, or kernel panic in > somewhat higher load when 131072, so I gave up other changes and fall > back to 65536. > > What is a good way to calcurate this value safely? Here is another > hint, /etc/sysctl.conf: > > net.link.ether.inet.log_arp_wrong_iface=0 > kern.ipc.maxsockbuf=2048000 > kern.ipc.somaxconn=4096 > kern.ipc.maxsockets=60000 > kern.maxfiles=65536 > kern.maxfilesperproc=32768 > net.inet.tcp.rfc1323=1 > net.inet.tcp.delayed_ack=0 > net.inet.tcp.sendspace=65535 > net.inet.tcp.recvspace=65535 > net.inet.udp.recvspace=65535 > net.inet.udp.maxdgram=57344 > net.inet.icmp.drop_redirect=1 > net.inet.icmp.log_redirect=1 > net.inet.ip.redirect=0 > net.inet6.ip6.redirect=0 > net.link.ether.inet.max_age=1200 > net.inet.ip.sourceroute=0 > net.inet.ip.accept_sourceroute=0 > net.inet.icmp.bmcastecho=0 > net.inet.icmp.maskrepl=0 > net.inet.tcp.inflight_enable=1 > > kernel configuration is not specially tuned, except DEVICE_POLLING and > HZ=2000. > > -- > CHOI Junho KFUG > FreeBSD Project Web Data Bank > Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-net" in the body of the message > -- Bosko Milekic * bmilekic@unixdaemons.com * bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message