From owner-freebsd-net@FreeBSD.ORG Sat Oct 21 01:17:45 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7FD1516A47C for ; Sat, 21 Oct 2006 01:17:45 +0000 (UTC) (envelope-from lists@codeangels.com) Received: from mail.codeangels.com (monkey.codeangels.com [62.2.169.19]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0086C43D53 for ; Sat, 21 Oct 2006 01:17:42 +0000 (GMT) (envelope-from lists@codeangels.com) Received: (qmail-ldap/ctrl 15337 invoked from network); 21 Oct 2006 01:17:40 -0000 Received: from monkey.codeangels.com (HELO www.codeangels.com) (nglvdz@[192.168.5.6]) (envelope-sender ) by monkey.codeangels.com (qmail-ldap-1.03) with SMTP for ; 21 Oct 2006 01:17:40 -0000 Message-ID: <2108.192.168.1.6.1161393460.squirrel@www.codeangels.com> Date: Sat, 21 Oct 2006 03:17:40 +0200 (CEST) From: "Kirill Ponazdyr" To: freebsd-net@freebsd.org User-Agent: SquirrelMail/Codeangels_GEN MIME-Version: 1.0 Content-Type: text/plain;charset=iso-8859-1 X-Priority: 3 Importance: Normal Subject: Gigabit performance test X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: lists@codeangels.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Oct 2006 01:17:45 -0000 Hello, I am preparing a test of different FreeBSD firewalls in our lab, before doing so I am trying to push maximum 2 gbps of traffic through the machine with a simple routed on it in the most optimal way. The lab setup is as following: 4 x traffic generators machines: Dual Opteron, generic FreeBSD 6.1 / AMD64 kernel + iperf 2.02. The iperf between the machines directly is almost always around ~930 megabit, which is fine (See table referenced later for detailed results). 1 x Firewall machine, which is a Dell 2650 Server, for detailed specs please see: dmesg: http://www.codeangels.com/misc/fwtest/first/fw_dmesg.txt pciconf: http://www.codeangels.com/misc/fwtest/first/fw_pciconf.txt sysctl: http://www.codeangels.com/misc/fwtest/first/fw_sysctl.txt kernel: http://www.codeangels.com/misc/fwtest/first/fw_kern.txt HZ and Pooling values in those config files have been changed by me during test several times as you will see in results table. The kernels have pf compiled in but it is not turned on at this time. The network topo is: http://www.codeangels.com/misc/fwtest/first/topo.gif And here are the results: http://www.codeangels.com/misc/fwtest/first/results.htm My questions are: * Single stream / single thread is always slower then in direct machine to machine communication, full throughput is reached only with multiple threads. Why? * In polling mode, there seems to be a "magic wall" between 1.3 and 1.7gbps where INT CPU usage suddenly jumps up from almost nothing to over 45+ and throughput stops there, Why? Can this be changed? * Any other ideas on improving performance of this box? Thanks ahead for help! Kirill