From owner-freebsd-net@FreeBSD.ORG Mon Nov 13 01:58:41 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1FF6A16A403; Mon, 13 Nov 2006 01:58:41 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3D9E43D5E; Mon, 13 Nov 2006 01:58:40 +0000 (GMT) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.13.6/8.13.6) with ESMTP id kAD1wdnL060097; Sun, 12 Nov 2006 20:58:40 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.6/8.13.3) with ESMTP id kAD1wdKE040908 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sun, 12 Nov 2006 20:58:39 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200611130158.kAD1wdKE040908@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Sun, 12 Nov 2006 20:58:49 -0500 To: Scott Long From: Mike Tancsa In-Reply-To: <4557CECD.2000609@samsco.org> References: <2a41acea0611081719h31be096eu614d2f2325aff511@mail.gmail.com> <200611091536.kA9FaltD018819@lava.sentex.ca> <45534E76.6020906@samsco.org> <200611092200.kA9M0q1E020473@lava.sentex.ca> <200611102004.kAAK4iO9027778@lava.sentex.ca> <2a41acea0611101400w5b8cef40ob84ed6de181f3e2c@mail.gmail.com> <200611102221.kAAML6ol028630@lava.sentex.ca> <455570D8.6070000@samsco.org> <200611120412.kAC4CuIB035746@lava.sentex.ca> <45574ECA.4080207@samsco.org> <200611130040.kAD0etbp040637@lava.sentex.ca> <4557CECD.2000609@samsco.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Virus-Scanned: ClamAV version 0.88.3, clamav-milter version 0.88.3 on clamscanner2 X-Virus-Status: Clean Cc: freebsd-net , freebsd-stable@freebsd.org, Jack Vogel Subject: Re: Proposed 6.2 em RELEASE patch X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Nov 2006 01:58:41 -0000 At 08:47 PM 11/12/2006, Scott Long wrote: >>>2. Try compiling in WITNESS and running the test as before, then break >>>into the debugger as before. Run 'show locks'. I'm not sure how >>>fruitful this will be, WITNESS might make it unbearably slow. >>It was in that kernel already > >So you're seeing the livelock under load while also having WITNESS >enabled? Have you tried your test without WITNESS? What about INVARIANTS? Sorry, by it was in the kernel already, it was from last nights tests where you asked me to break into the debugger to see what was going on. I figured you would want it in there. My original tests were all without WITNESS and INVARIANTS and they were showing the same behaviour (locking up on the management interface) with and without WITNESS and INVARIANTS Removing ADAPTIVE_GIANT seems to be very key in this! If I remove it, the box no longer locks up under a high packet rate! Its still a little sluggish on the bge OOB interface, but its not totally locking up like before Running ifstat -b from the management interface, here is the traffic pattern first with one blast running and then the second kicks in. The Kbps/s out on em1 drops in half, but the box is still responsive and bge1... Certainly responsive enough to see the output of ifstat. 218028.4 0.00 0.00 216389.9 0.47 1.17 218971.2 0.00 0.00 217123.1 0.94 2.34 217754.7 0.00 0.00 215677.2 0.94 1.17 215763.1 0.00 0.00 214578.5 0.47 1.17 217850.3 0.00 0.00 216138.7 1.40 2.34 217668.0 0.00 0.00 215888.3 0.47 1.17 em0 em1 bge1 Kbps in Kbps out Kbps in Kbps out Kbps in Kbps out 217382.3 0.00 0.00 216293.8 1.26 1.59 218247.9 118916.5 234167.3 124972.1 0.47 2.17 215973.4 220459.9 393350.0 54875.64 1.40 2.34 217697.7 210984.5 398543.3 58165.20 0.94 3.09 217183.6 200424.9 399096.5 61845.74 0.94 1.17 217788.7 221212.9 400129.8 53716.38 0.47 1.17 217040.0 204934.2 396628.7 61247.34 1.40 2.34 217105.9 201295.5 399274.1 61710.76 0.94 3.09 215605.6 225012.4 397533.5 53237.31 0.94 1.17 217933.3 200346.4 395949.3 62734.52 0.47 1.17 216783.7 205114.1 395923.0 61042.17 0.94 1.17 217885.7 88513.41 159056.4 151418.9 0.47 1.17 216927.2 0.00 0.00 215450.4 0.94 1.17 217454.1 0.00 0.00 215466.6 0.47 1.17 216810.6 0.00 0.00 215564.2 1.40 1.17 217608.3 0.00 0.00 216061.8 0.47 1.17 217452.8 0.00 0.00 215627.1 0.94 1.17 217185.0 0.00 0.00 216079.7 0.47 1.17 However, if I turn on fastforwarding, its back to the old behavior with it locking up. This was with the stock driver. I will try the same test with #define EM_FAST_INTR 1 as well as taking out the nfs option from the kernel driver. Anything else to tune with ? ---Mike >Scott