From owner-freebsd-net@FreeBSD.ORG Wed Jan 14 01:59:12 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4EF016A4CE; Wed, 14 Jan 2004 01:59:12 -0800 (PST) Received: from king.suceava.rdsnet.ro (king.suceava.rdsnet.ro [62.231.118.3]) by mx1.FreeBSD.org (Postfix) with ESMTP id 07E5743D53; Wed, 14 Jan 2004 01:58:59 -0800 (PST) (envelope-from ady@freebsd.ady.ro) Received: from datacenter.office.suceava.rdsnet.ro (datacenter.office.suceava.rdsnet.ro [217.156.25.194])i0E9wuMS019036 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 14 Jan 2004 11:58:56 +0200 Received: from sungoku.home.ady.ro ([82.208.147.127]) id i0EA8cK6007387 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 14 Jan 2004 12:08:46 +0200 (EET) (envelope-from ady@freebsd.ady.ro) Received: from localhost (sunny.home.ady.ro [10.0.0.2]) by sungoku.home.ady.ro (8.12.9/8.12.9) with ESMTP id i0EAEBGg036997; Wed, 14 Jan 2004 12:14:26 +0200 (EET) (envelope-from ady@freebsd.ady.ro) Date: Wed, 14 Jan 2004 11:58:07 +0200 (E. Europe Standard Time) From: Adrian Penisoara To: freebsd-isp@freebsd.org Message-ID: X-X-Sender: ady@datacenter.office.suceava.rdsnet.ro MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-RAVMilter-Version: 8.4.2(snapshot 20021212) (datacenter.office.suceava.rdsnet.ro) cc: freebsd-net@freebsd.org Subject: Handling 100.000 packets/sec or more X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Jan 2004 09:59:13 -0000 Hi, At one site that I administer we have a gateway server which services a large SOHO LAN (more than 300 stations) and I'm facing a serious issue: very often we see strong spoofed floods (variable source IP and port, variable destination IP, destination port 80) which can go as far as 100 000 packets/sec! Of course, the server (FreeBSD 5.2-REL, PIII 733Mhz, 256Mb RAM, 3COM 3C905B-TX aka xl0 with checksum offloading support) has a hard time swallowing this kind of traffic. The main issue are the IRQ interrupts: over 15000 interrupts/sec which consume more than 90% of the CPU time. We got ingress filtering so the packets go no further than the firewall (which, BTW, is not the issue, even disabling it it's the same problem). The system is still responsive but the load average goes as high as 10 and the interface is losing packets (input errors) which dramatically affects legitimate traffic, besides mbuf(9) starvation. We are taking down the culprit clients, but this takes time and we need the other clients not to be affected by it. What can I do to make the system better handle this kind of traffic ? Could device polling(8) or just increasing the kernel frequency clock to 1000Hz or more improve the situation ? What kind of network cards could face a lot better this burden ? Are there any other solutions ? On a side note: what would be a adequate formula to calculate the NMBCLUSTERS and MBUFS we should set on this server (via boot-time kern.ipc.nmbclusters and kern.ipc.nmbufs) ? Thank you. -- Adrian Penisoara Ady (@freebsd.ady.ro)