From owner-freebsd-net Wed Feb 26 5: 6:39 2003 Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5F4737B401 for ; Wed, 26 Feb 2003 05:06:37 -0800 (PST) Received: from daemon.kr.FreeBSD.org (daemon.kr.freebsd.org [211.176.62.31]) by mx1.FreeBSD.org (Postfix) with ESMTP id 66FC343F93 for ; Wed, 26 Feb 2003 05:06:34 -0800 (PST) (envelope-from cjh@kr.FreeBSD.org) Received: from gradius.wdb.co.kr (daemon [211.176.62.31]) by daemon.kr.FreeBSD.org (Postfix) with ESMTP id E6F368F60B for ; Wed, 26 Feb 2003 22:06:31 +0900 (KST) Received: from localhost (localhost [127.0.0.1]) by gradius.wdb.co.kr (8.12.7/8.12.7) with ESMTP id h1QD5p8x017628 for ; Wed, 26 Feb 2003 22:05:52 +0900 (KST) (envelope-from cjh@kr.FreeBSD.org) Date: Wed, 26 Feb 2003 22:05:51 +0900 (KST) Message-Id: <20030226.220551.10329540.cjh@kr.FreeBSD.org> To: net@freebsd.org Subject: Performance tuning hints of gigabit networking? From: CHOI Junho Organization: Korea FreeBSD Users Group X-URL: http://www.kr.FreeBSD.org/~cjh X-Mailer: Mew version 3.2rc1 on Emacs 21.2 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Hi, I am looking for a good resource for kernel tuning on very high bandwidth HTTP servers(avg 500Mbit/sec, peak 950Mbit/sec). Today I faced very unusual situation with 950Mbit/sec bandwidth! > netstat -m 16962/93488/262144 mbufs in use (current/peak/max): 16962 mbufs allocated to data 16952/65536/65536 mbuf clusters in use (current/peak/max) 154444 Kbytes allocated to network (14% of mb_map in use) 512627 requests for memory denied 2614 requests for memory delayed 0 calls to protocol drain routines I set kern.ipc.nmbclusters=65536, but it overflowed. This is P-IV Xeon 1.8G, 2GB RAM, and one Intel 1000baseSX(em driver) machine running 4.7-RELEASE-pX. This server is running only one service, HTTP. I use thttpd, since apache doesn't work in such a high load. thttpd is highly amazing, just give <1 load in any time. Once I tried to increase kern.ipc.nmbclusters to 131072 or higher(multiple of 65536 or 32768, tuning(7) only cites about 32768 case..), it fails to boot kernel when 262144, or kernel panic in somewhat higher load when 131072, so I gave up other changes and fall back to 65536. What is a good way to calcurate this value safely? Here is another hint, /etc/sysctl.conf: net.link.ether.inet.log_arp_wrong_iface=0 kern.ipc.maxsockbuf=2048000 kern.ipc.somaxconn=4096 kern.ipc.maxsockets=60000 kern.maxfiles=65536 kern.maxfilesperproc=32768 net.inet.tcp.rfc1323=1 net.inet.tcp.delayed_ack=0 net.inet.tcp.sendspace=65535 net.inet.tcp.recvspace=65535 net.inet.udp.recvspace=65535 net.inet.udp.maxdgram=57344 net.inet.icmp.drop_redirect=1 net.inet.icmp.log_redirect=1 net.inet.ip.redirect=0 net.inet6.ip6.redirect=0 net.link.ether.inet.max_age=1200 net.inet.ip.sourceroute=0 net.inet.ip.accept_sourceroute=0 net.inet.icmp.bmcastecho=0 net.inet.icmp.maskrepl=0 net.inet.tcp.inflight_enable=1 kernel configuration is not specially tuned, except DEVICE_POLLING and HZ=2000. -- CHOI Junho KFUG FreeBSD Project Web Data Bank Key fingerprint = 1369 7374 A45F F41A F3C0 07E3 4A01 C020 E602 60F5 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message