From owner-freebsd-net@FreeBSD.ORG Fri Nov 26 12:19:22 2004 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 880FB16A4CE for ; Fri, 26 Nov 2004 12:19:22 +0000 (GMT) Received: from borgtech.ca (borgtech.ca [216.187.106.216]) by mx1.FreeBSD.org (Postfix) with ESMTP id 45B1443D41 for ; Fri, 26 Nov 2004 12:19:22 +0000 (GMT) (envelope-from asegu@borgtech.ca) Received: from mojlaptop (unknown [161.53.212.202]) by borgtech.ca (Postfix) with ESMTP id 66A0454A5 for ; Fri, 26 Nov 2004 12:33:32 +0000 (GMT) Message-ID: <007f01c4d3b2$12597af0$cad435a1@mojlaptop> From: "Andrew Seguin" To: Date: Fri, 26 Nov 2004 13:18:41 +0100 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1437 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441 Subject: FreeBSD 5.3 Networking performance problem X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Nov 2004 12:19:22 -0000 *Problem: Poor performance for freebsd transparent gateway. *Situation: I need to install a simple firewall for a school network I am administering. We have about 100 computers active, generating a stream of approximately 80-90K packets per minute for a load I estimate* to be a little under 10Mbps. Overall the firewall will need to filter for a /24 subnet. *Configuration: Hardware: The firewall is a Celeron 900Mhz with 128MB ram (more on the way) with one rl and one sis based network cards. The firewall is to be the bridge between the main switch and the router. Software: I built up the firewall with FreeBSD 5.3, with a recompiled kernel using options BRIDGE, IPFIREWALL, IPFIREWALL_VERBOSE, IPFIREWALL_VERBOSE_LIMIT, IPFIREWALL_DEFAULT_TO_ACCEPT and IPSTEALTH. No software is running. IPFW is left with only it's default rule of allow all. *Testing: I tested with the firewall bridging for a single computer: ping time to the router was a stable 2ms. I then tested with the whole school going through the firewall: very bad. packets were being droped and ping times were around 600ms. Internet was pretty much unuseable. I googled around and read a bit, discovered polling. I Rebuilt the kernel for it and HZ set to 1000. I set the appropriate sysctl's and saw on ifconfig polling was indicated for both network cards. I retried using the firewall for the whole school, but again it wasn't working. I disconnected the secondary switches (which is for the offices, student residence, computer lab, etc) and kept a computer on the main switch. Ping times remained stable up to a bandwith I estimated later to be of approximately 20MB/min. The last switch I added, having a trafic of 5MB/min seemed to kill the box. During my testing with the poling kernel, interupt time went up to 10% for the whole school, with 90% idle. Memory remained unchanged with 86MB free. Conclusion: I don't know what could be causing what seems to me as simply low performance under increased load. I've heard of people with higher loads then I have here**. If somebody on the list could give me some clues of what could be the problem here and pointers as to what to look at next, I would appreciate it greatly. The only idea I have here is to try and rebuild to 4.10 and see if the performance is there... is 4.10 much more performant then 5.3 ? * I have yet to get access to the router (SNMP or otherwise). I estimated the school load by using my firewall to test the traffic from each individual switch's uplink. I then extrapolated approximate traffic for our web and email servers in the very unscientific manor of comparing the lights on the main switch. **In particular the post on Nov 17 by Yar Tikhiy "polling(4) rocks!" had a claim of about 9kpps vs my load of about 1.5kpps