From owner-freebsd-stable@FreeBSD.ORG Wed Feb 27 20:45:04 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC5FB106566B for ; Wed, 27 Feb 2008 20:45:04 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.159]) by mx1.freebsd.org (Postfix) with ESMTP id 6D2708FC24 for ; Wed, 27 Feb 2008 20:45:02 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so2256238fgg.35 for ; Wed, 27 Feb 2008 12:45:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; bh=ktdjqVC4559Nb8of7lanUg4wD0T+J7uqa/Xg0r2lZr0=; b=Gzy9nICQVb+qcbDh54J4hiIXYDbJAiMLEE7ZWVD7mbfcNE4+ZJKKMpJv9EFLrQ+4jxVkF/gSv/s8P3i2+si+MIC0Aw/UhvUtoDxqxuhQfXJiq+EcXEkdUP244ayhIDzr8Y8MbIK2dXZBspIJqdWXIC3i9yT4CPeC6Wdr54fPLLI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=gqdkYpI6Lf+FNzNAwx+pbL87p2C1uwIlzP6gfyRNdBIRF7/jGQx6phvewaHS5vr1wbIRteJNDEWseiT84eKgsZUWeaBN9WpPvzRZrwi4icDb5EiZbsbQEcWTlrHgDKDGluICXYH7LYfZZiXK8CzGE6LyKRfx4OvH3PPlQITWJV8= Received: by 10.86.31.18 with SMTP id e18mr6587646fge.68.1204143387958; Wed, 27 Feb 2008 12:16:27 -0800 (PST) Received: by 10.86.97.10 with HTTP; Wed, 27 Feb 2008 12:16:27 -0800 (PST) Message-ID: <2a41acea0802271216m2796f90fnb7650bc06c86ed25@mail.gmail.com> Date: Wed, 27 Feb 2008 12:16:27 -0800 From: "Jack Vogel" To: "Holger Kipp" In-Reply-To: <20080227195058.GA27997@intserv.int1.b.intern> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <20080227104942.GA14530@intserv.int1.b.intern> <200802271452.m1REqJ6H059687@lava.sentex.ca> <20080227195058.GA27997@intserv.int1.b.intern> Cc: freebsd-stable@freebsd.org Subject: Re: [solved/workaround?] Re: em very slow, shared irq... on 6.3p8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2008 20:45:05 -0000 On Wed, Feb 27, 2008 at 11:50 AM, Holger Kipp wrote: > On Wed, Feb 27, 2008 at 09:50:16AM -0500, Mike Tancsa wrote: > > more details below. as it currently is, polling seems to do > the trick, however handling several em-interfaces with the > same irq (mind you, it is pci) shouldn't cause delays of > up to 1.5 seconds for a simple ping... Therefore I consider > using polling for a nearly idle system more a workaround > than a solution to this problem :-( > > > At 05:49 AM 2/27/2008, Holger Kipp wrote: > > > > >I therefore assume that the problem is between receiving the > > >irq from em and getting the data from the interface on the firewall > > >itself. > > > > I would try upgrading to 6.3R (there are several em driver bug fixes) > done. system is now 6.3-RELEASE-p1 which also gave me -c option for pciconf > and msi syscontrols (were missing in the old 6.2). > > > > and then try the box with > > % cat /boot/loader.conf > > hw.pci.enable_msi=1 > > > > ...if the cards support msi. > > > > I think pciconf -lvc should tell you if the cards and slots support it or > > not. > > pciconf -lvc says for all em: > cap 05[d0] = MSI supports 1 message, 64 bit > so I assume they do support MSI. > > with msi disabled I get > > 38 packets transmitted, 38 packets received, 0% packet loss > round-trip min/avg/max/stddev = 4.833/228.022/1539.337/339.768 ms > > with msi enabled (via sysctl) I get > > 33 packets transmitted, 33 packets received, 0% packet loss > round-trip min/avg/max/stddev = 1.865/156.421/1339.841/239.375 ms > > so looks equally bad (I don't consider 30-40 packets a meaningful sample). > I don't know if it makes any differences if switched on directly in > loader.conf, though. > > enabling polling (withous MSI) gives > > 30 packets transmitted, 30 packets received, 0% packet loss > round-trip min/avg/max/stddev = 0.366/0.790/1.339/0.290 ms > > (maybe I should have used HZ=2000 to keep it below 0.6ms ;-) > > > Also, if you dont need IPV6, use FAST_IPSEC. It does not need > > mpsafe. If you do need IPSEC and IPV6, 7.0R got rid of that restriction. > > I think this enough changes in one go for a production system ;-) Hmmm, something is really broken here that POLL just is just bypassing, what was the adapter type exactly (pciconf -l). Sorry, but I must have missed this email earlier. Jack