From owner-freebsd-stable@FreeBSD.ORG Wed Sep 27 09:17:29 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 699C916A403 for ; Wed, 27 Sep 2006 09:17:29 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 70EDD43D7F for ; Wed, 27 Sep 2006 09:17:23 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [192.168.254.14] (imini.samsco.home [192.168.254.14]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k8R9GvbU017794; Wed, 27 Sep 2006 03:17:02 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <451A4189.5020906@samsco.org> Date: Wed, 27 Sep 2006 03:16:57 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.7.7) Gecko/20050416 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Oliver Brandmueller References: <451A1375.5080202@gneto.com> <20060927071538.GF22229@e-Gitt.NET> In-Reply-To: <20060927071538.GF22229@e-Gitt.NET> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=3.8 tests=ALL_TRUSTED autolearn=failed version=3.1.1 X-Spam-Checker-Version: SpamAssassin 3.1.1 (2006-03-10) on pooker.samsco.org Cc: freebsd-stable@freebsd.org Subject: Re: 6.2 SHOWSTOPPER - em completely unusable on 6.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Sep 2006 09:17:29 -0000 Oliver Brandmueller wrote: > Hi, > > On Wed, Sep 27, 2006 at 08:00:21AM +0200, Martin Nilsson wrote: > >>I get tons of these: >>em0: watchdog timeout -- resetting >>em0: link state changed to DOWN >>em0: link state changed to UP >> >>mailbox# pciconf -lv >>em0@pci13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 >>hdr=0x00 >> vendor = 'Intel Corporation' >> device = 'PRO/1000 PM' >> class = network >> subclass = ethernet >>em1@pci14:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 >>hdr=0x00 >> vendor = 'Intel Corporation' >> class = network >> subclass = ethernet >> > > [...] > >>I have only seen them on em0. Yesterday I tried sysutils/cpuburn on >>similar boxes that are netbooted with NFS mounted drives and everytime I >>loaded the two CPU cores the network went down. > > > I see the same. > > Very much on this one, where I workaround the problem by using polling, > it's a UP machine. > > FreeBSD nessie 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #3: Fri Sep 15 09:48:36 CEST 2006 root@nessie:/usr/obj/usr/src/sys/NESSIE i386 > > em0@pci2:1:0: class=0x020000 card=0x10198086 chip=0x10198086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82547EI Gigabit Ethernet Controller (LOM)' > class = network > subclass = ethernet > > irq18: em0 uhci2 3319 0 > > > Another machine, also UP, but with two interfaces. The problem is not as > apparent as on the first machine, but it's there. This machine is not as > loaded usually (CPU wise) as the first machine. The problem is ONLY on > em1: > > FreeBSD hudson 6.2-PRERELEASE FreeBSD 6.2-PRERELEASE #48: Thu Sep 14 10:19:46 CEST 2006 root@hudson:/usr/obj/usr/src/sys/NFS-32-FBSD6 i386 > > em0@pci1:1:0: class=0x020000 card=0x10758086 chip=0x10758086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82547EI Gigabit Ethernet Controller' > class = network > subclass = ethernet > > em1@pci3:2:0: class=0x020000 card=0x10768086 chip=0x10768086 rev=0x00 hdr=0x00 > vendor = 'Intel Corporation' > device = '82547EI Gigabit Ethernet Controller' > class = network > subclass = ethernet > > irq17: em1 ichsmb0 950121879 855 > irq18: em0 71437344 64 > > > The problem appeared after the em updates during the last weeks in the > kernel and has not been observed before this. em is always loaded as a > module in my kernels. The problem seems to occur more often if the > machine's CPU is busy. > > > I have several SMP machines with the following em interfaces, which > DON'T show the problem, but they also have different chipset on the em > interface. Most of the kernels were built between Sep 7 and Sep 19. > > 3 times this: > em0@pci4:5:0: class=0x020000 card=0x34248086 chip=0x10108086 rev=0x01 hdr=0x00 > em1@pci4:5:1: class=0x020000 card=0x34248086 chip=0x10108086 rev=0x01 hdr=0x00 > irq23: em0 970303432 750 > > > > 3 times this: > em0@pci4:5:0: class=0x020000 card=0x34258086 chip=0x100e8086 rev=0x02 hdr=0x00 > irq23: em0 292477376 435 > > > So I can observe at least 3 interesting differences: > > - the interface showing the problems shares the interrupt > - for me it happens on UP machines only > - the chips are different > > What I can't do: moving the interfaces between machines, these are > onboard interfaces. > > What I could do: I could try to unload the USB driver or the ichsmb > driver on the machiens, where the interrupts are shared. Anyway, the USB > is not used currently (I have it enabled to be prepared to hook up a USB > Mass Storage device, which never happend since the problem occured). The > ichsmb also is usually not queried. > > Any suggestions on how I could help? > > - Olli > > Well, the best I can say at the moment is, "Wow." =-( I guess the thing to do here is to figure out if the problem lies with the em interrupt handler not getting run, or the taskqueue not getting run. Since you've stated that it seems to be related to shared interrupts, the first possibility is more likely. However, I'm not sure why the symptom would only be showing up now. The Intel docs say that the 82547EI are a bit interesting, and I wonder if assumptions that we make about PCI ordering aren't true (or if there are bugs that make our assumptions invalid). Does this happen after there has been a lot of disk activity, like a large tar extraction? Are you using the SMBus interface at all, or is it sitting completely idle? Scott