From owner-freebsd-net@FreeBSD.ORG Sat Nov 11 12:05:18 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 345F016A47C; Sat, 11 Nov 2006 12:05:18 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [205.211.164.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6FD1D43D5D; Sat, 11 Nov 2006 12:05:12 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost2.sentex.ca (8.13.8/8.13.8) with ESMTP id kABC5BfX013224; Sat, 11 Nov 2006 07:05:11 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kABC59gP032197 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 11 Nov 2006 07:05:09 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611111205.kABC59gP032197@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sat, 11 Nov 2006 07:05:17 -0500 To: Scott Long From: Mike Tancsa In-Reply-To: <455570D8.6070000@samsco.org> References: <2a41acea0611081719h31be096eu614d2f2325aff511@mail.gmail.com> <200611091536.kA9FaltD018819@lava.sentex.ca> <45534E76.6020906@samsco.org> <200611092200.kA9M0q1E020473@lava.sentex.ca> <200611102004.kAAK4iO9027778@lava.sentex.ca> <2a41acea0611101400w5b8cef40ob84ed6de181f3e2c@mail.gmail.com> <200611102221.kAAML6ol028630@lava.sentex.ca> <455570D8.6070000@samsco.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Cc: freebsd-net , freebsd-stable@freebsd.org, Jack Vogel Subject: Re: Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 11 Nov 2006 12:05:18 -0000 At 01:42 AM 11/11/2006, Scott Long wrote: >surprised by your results. I'm still a bit unclear on the exact >topology of your setup, so if could explain it some more in private >email, I'd appreciate it. Hi, I made a quick diagram of the test setup that should make it more clear http://www.tancsa.com/blast.jpg Basically 5 boxes (plus my workstation for out of band access), the main one being tested is the box marked R2 which has a 2 port PCIe em NIC (Pro 1000PT) in the motherboard's 4X slot. I have 2 test boxes as UDP senders and 2 test boxes as UDP receivers, and all the packets flow through the 2 interfaces of R2. With one stream of packets being blasted across, the box is dropping some packets even on its OOB management interface. With 2, its totally unresponsive. Only with polling am I able to continue to work on the box via the OOB interface while one and even 2 streams of UDP packets are blasting across. However, in polling mode some amount of packets are being dropped and I guess I need to better understand how many. My goal in all this is to have a firewall / router that can withstand a high pps workload that will still be reachable OOB when under attack or even under high workload. To measure how many packets are dropped I was looking at making a modified netreceive to count the packets it gets so I can test to see if polling mode will be adequate for my needs. Lets say the max pps the box can handle is X, either in polling or non polling modes. As the box approaches X and gets pushed beyond X, I guess the ideal situation for my needs would be that it drops some packets on the busiest interface so that it can still function and service its other needs, be that network, disk, whatever. But my question is, is X the same for polling and non polling modes. >For the short term, I don't think that there is anything that can be >magically tweaked that will safely give better results. I know that >Gleb has some ideas on a fairly simple change for the non-INTR_FAST, >non-POLLING case, but I and several others worry that it's not robust >in the face of real-world network problems. > >For the long term, I have a number of ideas for improving both the RX >and TX paths in the driver. Some of it is specific to the if_em driver, >some involve improvements in the FFWD and PFIL_HOOKS code as well as the >driver. What will help me is if you can hook up a serial console to >your machine and see if it can be made to drop to the debugger while it >is under load and otherwise unresponsive. If you can, getting a process >dump might help confirm where each CPU is spending its time. Yes, I will see what I can do over the weekend. I have some changes to babysit again tomorrow night and will see what I can do between cycles. ---Mike