From owner-freebsd-net@FreeBSD.ORG Thu Nov 9 22:01:20 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4601C16A40F; Thu, 9 Nov 2006 22:01:20 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1EC3A43D6D; Thu, 9 Nov 2006 22:00:53 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kA9M0rXt038408; Thu, 9 Nov 2006 17:00:53 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kA9M0q1E020473 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 9 Nov 2006 17:00:52 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611092200.kA9M0q1E020473@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Thu, 09 Nov 2006 17:00:56 -0500 To: Scott Long From: Mike Tancsa In-Reply-To: <45534E76.6020906@samsco.org> References: <2a41acea0611081719h31be096eu614d2f2325aff511@mail.gmail.com> <200611091536.kA9FaltD018819@lava.sentex.ca> <45534E76.6020906@samsco.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Cc: freebsd-net , freebsd-stable@freebsd.org, Jack Vogel Subject: Re: Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Nov 2006 22:01:20 -0000 At 10:51 AM 11/9/2006, Scott Long wrote: >Mike Tancsa wrote: >>At 08:19 PM 11/8/2006, Jack Vogel wrote: >> >>>BUT, I've added the FAST_INTR changes back into the code, so >>>if you go into your Makefile and add -DEM_FAST_INTR you will >>>then get the taskqueue stuff. >>It certainly does make a difference performance wise. I did some >>quick testing with netperf and netrate. Back to back boxes, using >>an AMD x2 with bge nic and one intel box >>CPU: AMD Athlon(tm) 64 X2 Dual Core Processor 3800+ (2009.27-MHz >>686-class CPU) >>CPU: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz (2144.01-MHz >>686-class CPU) >>The intel is a DG965SS with integrated em nic, the AMD a Tyan with >>integrated bge. Both running SMP kernels with pf built in, no inet6. >> >>Intel box as sender. In this test its with the patch from >>yesterday. The first set with the patch as is, the second test with >>-DEM_FAST_INTR. > >Thanks for the tests. One thing to note is that Gleb reported a higher >rate of dropped packets with INTR_FAST. He is the only one who has >reported this, so I'd like to find out if there is something unique to >his environment, or if there is a larger problem to be addressed. There >are ways that we can change the driver to not drop any packets at all >for Gleb, but they expose the system to risk if there is ever an >accidental (or malicious) RX flood on the interface. With a high rate of packets, I am able to live lock the box. I setup the following b1a ------| b2a -----R1 ------------b1b |-------------b2b R1 has PCIe dual port em PRO/1000 PT em0: port 0x9000-0x901f mem 0xd7020000-0xd703ffff,0xd7000000-0xd701ffff irq 1 7 at device 0.0 on pci6 em0: Ethernet address: 00:15:17:0b:70:98 em0: [FAST] em1: port 0x9400-0x941f mem 0xd7040000-0xd705ffff,0xd7060000-0xd707ffff irq 1 8 at device 0.1 on pci6 em1: Ethernet address: 00:15:17:0b:70:99 em1: [FAST] b1a = 192.168.44.1 onboard bge0 b1b = 192.168.88.218 - onboard em 82547EI Gigabit Ethernet Controller b2a = 192.168.88.176 single port PCIe em0 b2b = 192.168.44.244 onboard em0 (DG965SS) R1 has 192.168.44.223 and 192.168.88.223. Routing across R1 with b1a blasting b1b and b2a blastin b2b with netrate will lock up R1 even though the total throughput is only 500Mb. While on b1a # ./netblast 192.168.88.218 500 10 1000 I see the following on R1 (bge1 is my management interface) R1 # ifstat -b em0 em1 bge1 Kbps in Kbps out Kbps in Kbps out Kbps in Kbps out 273770.1 0.00 0.00 237269.1 1.40 3.51 273509.8 0.00 0.00 237040.2 1.73 2.76 273694.9 0.00 0.00 237202.6 0.94 2.34 274258.6 0.00 0.00 237690.4 1.40 2.34 273623.8 0.00 0.00 237140.7 0.94 2.34 If I start up the netblast on b2b or on b2a (either direction, doesnt matter) R1 locks up. This was with R1 in an SMP config. Without INTR_FAST, it doesnt work as fast, but R1 does not lock up, or at least lock me out of my management interface. ---Mike