From owner-freebsd-stable@FreeBSD.ORG Wed Feb 27 21:11:46 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5884D106566C for ; Wed, 27 Feb 2008 21:11:46 +0000 (UTC) (envelope-from hk@alogis.com) Received: from alogis.com (firewall.solit-ag.de [212.184.102.1]) by mx1.freebsd.org (Postfix) with ESMTP id E2E8A8FC18 for ; Wed, 27 Feb 2008 21:11:45 +0000 (UTC) (envelope-from hk@alogis.com) Received: from alogis.com (localhost [127.0.0.1]) by alogis.com (8.13.4/8.13.1) with ESMTP id m1RLBi3T030139; Wed, 27 Feb 2008 22:11:44 +0100 (CET) (envelope-from hk@alogis.com) Received: (from hk@localhost) by alogis.com (8.13.4/8.13.1/Submit) id m1RLBiB4030138; Wed, 27 Feb 2008 22:11:44 +0100 (CET) (envelope-from hk) Date: Wed, 27 Feb 2008 22:11:44 +0100 From: Holger Kipp To: Jack Vogel Message-ID: <20080227211144.GB28902@intserv.int1.b.intern> References: <20080227104942.GA14530@intserv.int1.b.intern> <200802271452.m1REqJ6H059687@lava.sentex.ca> <20080227195058.GA27997@intserv.int1.b.intern> <2a41acea0802271216m2796f90fnb7650bc06c86ed25@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2a41acea0802271216m2796f90fnb7650bc06c86ed25@mail.gmail.com> User-Agent: Mutt/1.4.2.1i Cc: freebsd-stable@freebsd.org Subject: Re: [solved/workaround?] Re: em very slow, shared irq... on 6.3p8 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Feb 2008 21:11:46 -0000 On Wed, Feb 27, 2008 at 12:16:27PM -0800, Jack Vogel wrote: > > Hmmm, something is really broken here that POLL just is just bypassing, > what was the adapter type exactly (pciconf -l). Sorry, but I must have > missed this email earlier. Problem is with em0-em13 that many share an IRQ 16/17, and because I was using 6.2-p8 and therefore did not have MSI enabled/available. Polling was also not compiled into the kernel. Polling alone will give times between 0.2 and 1.3 ms (with HZ=1000), and MSI alone (after activated during boot within loader.conf) will give best results (around 0.4 to 0.5 ms for a ping through firewall and back across two lans). This was then with 6.3-p1 Still, 6.3-p1 without msi and polling is still very slow in handling IRQs from the em-nics. I still don't know exactly why this took up to 1.5 seconds (sometimes even more) without MSI or POLLING. Please see my previous emails for dmesg. pciconf -l gives: em0@pci4:4:0: class=0x020000 card=0x11998086 chip=0x10b58086 rev=0x03 hdr=0x00 em1@pci4:4:1: class=0x020000 card=0x11998086 chip=0x10b58086 rev=0x03 hdr=0x00 em2@pci4:6:0: class=0x020000 card=0x11998086 chip=0x10b58086 rev=0x03 hdr=0x00 em3@pci4:6:1: class=0x020000 card=0x11998086 chip=0x10b58086 rev=0x03 hdr=0x00 em4@pci12:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em5@pci12:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em6@pci13:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em7@pci13:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em8@pci16:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em9@pci16:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em10@pci17:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em11@pci17:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 em12@pci18:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00 em13@pci19:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 We have one quad PCI-X 64bit card (the first four) and two quad PCI-Express cards. The last two devices are on-board ports. device = '82546GB PRO/1000 GT Quad Port Server Adapter' device = '82546GB PRO/1000 GT Quad Port Server Adapter' device = '82546GB PRO/1000 GT Quad Port Server Adapter' device = '82546GB PRO/1000 GT Quad Port Server Adapter' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82571EB Gigabit Ethernet Controller' device = '82573E Intel Corporation 82573E Gigabit Ethernet Controller (Copper)' device = '82573L Intel PRO/1000 PL Network Adaptor' Is this helpful? Please let me know if you need anything else. Best regards, Holger Kipp