Date: Mon, 3 Mar 2003 18:00:14 -0500 From: Bosko Milekic <bmilekic@unixdaemons.com> To: CHOI Junho <cjh@kr.FreeBSD.org> Cc: net@freebsd.org Subject: Re: Performance tuning hints of gigabit networking? Message-ID: <20030303180014.A7317@unixdaemons.com> In-Reply-To: <20030226.220551.10329540.cjh@kr.FreeBSD.org>; from cjh@kr.FreeBSD.org on Wed, Feb 26, 2003 at 10:05:51PM %2B0900 References: <20030226.220551.10329540.cjh@kr.FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
You're not running out of mbufs or clusters, you're out of RAM. Don't bump up nmbclusters anymore because you don't need to; instead, add more RAM. On Wed, Feb 26, 2003 at 10:05:51PM +0900, CHOI Junho wrote: > > Hi, > > I am looking for a good resource for kernel tuning on very high > bandwidth HTTP servers(avg 500Mbit/sec, peak 950Mbit/sec). Today I > faced very unusual situation with 950Mbit/sec bandwidth! > > > netstat -m > 16962/93488/262144 mbufs in use (current/peak/max): > 16962 mbufs allocated to data > 16952/65536/65536 mbuf clusters in use (current/peak/max) > 154444 Kbytes allocated to network (14% of mb_map in use) > 512627 requests for memory denied > 2614 requests for memory delayed > 0 calls to protocol drain routines > > I set kern.ipc.nmbclusters=65536, but it overflowed. This is P-IV Xeon > 1.8G, 2GB RAM, and one Intel 1000baseSX(em driver) machine running > 4.7-RELEASE-pX. This server is running only one service, HTTP. I use > thttpd, since apache doesn't work in such a high load. thttpd is highly > amazing, just give <1 load in any time. > > Once I tried to increase kern.ipc.nmbclusters to 131072 or > higher(multiple of 65536 or 32768, tuning(7) only cites about 32768 > case..), it fails to boot kernel when 262144, or kernel panic in > somewhat higher load when 131072, so I gave up other changes and fall > back to 65536. > > What is a good way to calcurate this value safely? Here is another > hint, /etc/sysctl.conf: > > net.link.ether.inet.log_arp_wrong_iface=0 > kern.ipc.maxsockbuf=2048000 > kern.ipc.somaxconn=4096 > kern.ipc.maxsockets=60000 > kern.maxfiles=65536 > kern.maxfilesperproc=32768 > net.inet.tcp.rfc1323=1 > net.inet.tcp.delayed_ack=0 > net.inet.tcp.sendspace=65535 > net.inet.tcp.recvspace=65535 > net.inet.udp.recvspace=65535 > net.inet.udp.maxdgram=57344 > net.inet.icmp.drop_redirect=1 > net.inet.icmp.log_redirect=1 > net.inet.ip.redirect=0 > net.inet6.ip6.redirect=0 > net.link.ether.inet.max_age=1200 > net.inet.ip.sourceroute=0 > net.inet.ip.accept_sourceroute=0 > net.inet.icmp.bmcastecho=0 > net.inet.icmp.maskrepl=0 > net.inet.tcp.inflight_enable=1 > > kernel configuration is not specially tuned, except DEVICE_POLLING and > HZ=2000. > > -- > CHOI Junho <http://www.kr.FreeBSD.org/~cjh>; KFUG <cjh at kr.FreeBSD.org> > FreeBSD Project <cjh at FreeBSD.org> Web Data Bank <cjh at wdb.co.kr> > Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-net" in the body of the message > -- Bosko Milekic * bmilekic@unixdaemons.com * bmilekic@FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030303180014.A7317>