From owner-freebsd-hackers@FreeBSD.ORG Sat Feb 28 20:17:53 2004 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 06C2C16A4CF for ; Sat, 28 Feb 2004 20:17:53 -0800 (PST) Received: from mail.sandvine.com (sandvine.com [199.243.201.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 20E1D43D2D for ; Sat, 28 Feb 2004 20:17:52 -0800 (PST) (envelope-from don@sandvine.com) Received: by mail.sandvine.com with Internet Mail Service (5.5.2657.72) id <15P457G1>; Sat, 28 Feb 2004 23:17:47 -0500 Message-ID: From: Don Bowman To: 'Deepak Jain' Date: Sat, 28 Feb 2004 23:17:44 -0500 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2657.72) Content-Type: text/plain; charset="iso-8859-1" cc: freebsd-hackers@freebsd.org Subject: RE: em0, polling performance, P4 2.8ghz FSB 800mhz X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 29 Feb 2004 04:17:53 -0000 > It was kindly pointed out that I didn't including the symptoms of the > problem: > > > Without polling on, I get 70+% interrupt load, and I get live lock. > > With polling on, I start getting huge amounts of input errors, packet > loss, and general unresponsiveness to the network. The web > server on it > doesn't respond though it occassionally will open the > connection, just > not respond. accept_filter on/off makes no difference. I have > read other > posts that say em systems can more >200kpps without serious incident. > > Thanks in advance, > > DJ You may need to increase the MAX_RXD inside your em driver to e.g. 512. With similar system, em can handle ~800Kpps of bridging. Your earlier email showed a very large number of RST messages, which makes me suspect the blackhole actually wasn't enabled. Not exactly sure what you're trying to do here. It sounds like you are trying to generate a SYN flood on port 80, and your listen queue is backing up. You've increased kern.ipc.somaxconn? Does your application specify a fixed listen queue depth? Could it be increased? Are you using apache as the server? Could you use a kqueue-enabled one like thttpd? Have you checked net.inet.ip.intr_queue_drops? If its showing >0, check net.inet.ip.intr_queue_maxlen, perhaps increase it. Have you sufficient mbufs and clusters? netstat -m. If you want to spend more time in kernel, perhaps change kern.polling.user_frac to 10? I might have HZ @ 2500 as well. You could use ipfw to limit the damage of a syn flood, e.g. a keep-state rule with a limit of ~2-5 per source IP, lower the timeouts, increase the hash buckets in ipfw, etc. This would use a mask on src-ip of all bits. something like: allow tcp from any to any setup limit src-addr 2 this would only allow 2 concurrent TCP sessions per unique source address. Depends on the syn flood you are expecting to experience. You could also use dummynet to shape syn traffic to a fixed level i suppose. now... this will switch the DoS condition to elsewhere in the kernel, and it might not win you anything. net.inet.ip.fw.dyn_buckets=16384 net.inet.ip.fw.dyn_syn_lifetime=5 net.inet.ip.fw.dyn_max=32000 might be called for if you try that approach.