From owner-freebsd-net@FreeBSD.ORG Wed Jan 5 03:58:53 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9553A1065673 for ; Wed, 5 Jan 2011 03:58:53 +0000 (UTC) (envelope-from crapsh@monkeybrains.net) Received: from lavash.monkeybrains.net (mail.monkeybrains.net [208.69.40.9]) by mx1.freebsd.org (Postfix) with ESMTP id 78FE68FC13 for ; Wed, 5 Jan 2011 03:58:53 +0000 (UTC) Received: from Computer-of-Penelope.local (adsl-76-199-97-132.dsl.pltn13.sbcglobal.net [76.199.97.132]) (authenticated bits=0) by lavash.monkeybrains.net (8.14.4/8.14.4) with ESMTP id p053wrYk098980 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 4 Jan 2011 19:58:53 -0800 (PST) (envelope-from crapsh@monkeybrains.net) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=monkeybrains.net; s=monkey; t=1294199933; bh=5EoNd1cUccOPEsRsqHH2iiKI7nUyBOciZiXiX9ceZmY=; h=Message-ID:Date:From:MIME-Version:To:CC:Subject:References: In-Reply-To:Content-Type:Content-Transfer-Encoding; b=DIkymxPOIF5tgxO8DjgpFTt8npwAOYXxCgh9TprxEhdFZaUi5cqtEtxIgpuoAhPMK TC+8HR4eIhC35aKCeTljG2H5GX7UAnFxBD+xEHXnK0WFeFygNa9ZZmGIjUACHHEU25 Aa9cJfmSWo1dZCn/wOmFb2WbB0GZdpXRWSp8DiyI= Message-ID: <4D23EC16.9040005@monkeybrains.net> Date: Tue, 04 Jan 2011 19:57:10 -0800 From: "Rudy (bulk)" User-Agent: Thunderbird 2.0.0.24 (Macintosh/20100228) MIME-Version: 1.0 To: Robin Sommer References: <20100729215649.GB2615@icir.org> <20110103210209.GA13091@icir.org> In-Reply-To: <20110103210209.GA13091@icir.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: clamav-milter 0.96.5 at lavash.monkeybrains.net X-Virus-Status: Clean Cc: freebsd-net Subject: Re: igb watchdog timeouts X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Jan 2011 03:58:53 -0000 I get them as well... on my em devices. I was just thinking of upgrading to a dual port igb I have kicking around, but your email is not encouraging. :) # grep watchdog /var/log/message Jan 2 21:13:38 turtle kernel: em5: watchdog timeout -- resetting Jan 3 04:31:37 turtle kernel: em4: watchdog timeout -- resetting Jan 3 09:41:09 turtle kernel: em4: watchdog timeout -- resetting Jan 3 12:05:05 turtle kernel: em4: watchdog timeout -- resetting Jan 4 10:29:58 turtle kernel: em1: watchdog timeout -- resetting Jan 4 15:36:19 turtle kernel: em5: watchdog timeout -- resetting Jan 4 16:09:51 turtle kernel: em5: watchdog timeout -- resetting Jan 4 18:28:48 turtle kernel: em5: watchdog timeout -- resetting Jan 4 18:33:41 turtle kernel: em5: watchdog timeout -- resetting Jan 4 19:18:35 turtle kernel: em4: watchdog timeout -- resetting Jan 4 19:26:21 turtle kernel: em5: watchdog timeout -- resetting Jan 4 19:27:25 turtle kernel: em5: watchdog timeout -- resetting Jan 4 19:33:33 turtle kernel: em5: watchdog timeout -- resetting Jan 4 19:40:54 turtle kernel: em5: watchdog timeout -- resetting Jan 4 19:50:45 turtle kernel: em5: watchdog timeouts = 129 Jan 4 19:54:33 turtle kernel: em0: watchdog timeouts = 0 Jan 4 19:54:39 turtle kernel: em1: watchdog timeouts = 13 Jan 4 19:54:41 turtle kernel: em2: watchdog timeouts = 0 Jan 4 19:54:44 turtle kernel: em3: watchdog timeouts = 0 Jan 4 19:54:46 turtle kernel: em4: watchdog timeouts = 121 stats info: Jan 4 19:50:45 turtle kernel: em5: Excessive collisions = 0 Jan 4 19:50:45 turtle kernel: em5: Sequence errors = 0 Jan 4 19:50:45 turtle kernel: em5: Defer count = 0 Jan 4 19:50:45 turtle kernel: em5: Missed Packets = 2280947 Jan 4 19:50:45 turtle kernel: em5: Receive No Buffers = 0 Jan 4 19:50:45 turtle kernel: em5: Receive Length Errors = 0 Jan 4 19:50:45 turtle kernel: em5: Receive errors = 0 Jan 4 19:50:45 turtle kernel: em5: Crc errors = 0 Jan 4 19:50:45 turtle kernel: em5: Alignment errors = 0 Jan 4 19:50:45 turtle kernel: em5: Collision/Carrier extension errors = 0 Jan 4 19:50:45 turtle kernel: em5: RX overruns = 115 Jan 4 19:50:45 turtle kernel: em5: watchdog timeouts = 129 Jan 4 19:50:45 turtle kernel: em5: RX MSIX IRQ = 0 TX MSIX IRQ = 0 LINK MSIX IRQ = 0 Jan 4 19:50:45 turtle kernel: em5: XON Rcvd = 0 Jan 4 19:50:45 turtle kernel: em5: XON Xmtd = 0 Jan 4 19:50:45 turtle kernel: em5: XOFF Rcvd = 0 Jan 4 19:50:45 turtle kernel: em5: XOFF Xmtd = 0 Jan 4 19:50:45 turtle kernel: em5: Good Packets Rcvd = 23188157965 Jan 4 19:50:45 turtle kernel: em5: Good Packets Xmtd = 42184614153 Jan 4 19:50:45 turtle kernel: em5: TSO Contexts Xmtd = 52043 Jan 4 19:50:45 turtle kernel: em5: TSO Contexts Failed = 0 debug info: Jan 4 19:51:15 turtle kernel: em5: Adapter hardware address = 0xc51ca420 Jan 4 19:51:15 turtle kernel: em5: CTRL = 0x140248 RCTL = 0x8002 Jan 4 19:51:15 turtle kernel: em5: Packet buffer = Tx=20k Rx=12k Jan 4 19:51:15 turtle kernel: em5: Flow control watermarks high = 10240 low = 8740 Jan 4 19:51:15 turtle kernel: em5: tx_int_delay = 66, tx_abs_int_delay = 66 Jan 4 19:51:15 turtle kernel: em5: rx_int_delay = 32, rx_abs_int_delay = 66 Jan 4 19:51:15 turtle kernel: em5: fifo workaround = 0, fifo_reset_count = 0 Jan 4 19:51:15 turtle kernel: em5: hw tdh = 180, hw tdt = 180 Jan 4 19:51:15 turtle kernel: em5: hw rdh = 708, hw rdt = 707 Jan 4 19:51:15 turtle kernel: em5: Num Tx descriptors avail = 2048 Jan 4 19:51:15 turtle kernel: em5: Tx Descriptors not avail1 = 0 Jan 4 19:51:15 turtle kernel: em5: Tx Descriptors not avail2 = 0 Jan 4 19:51:15 turtle kernel: em5: Std mbuf failed = 0 Jan 4 19:51:15 turtle kernel: em5: Std mbuf cluster failed = 0 Jan 4 19:51:15 turtle kernel: em5: Driver dropped packets = 0 Jan 4 19:51:15 turtle kernel: em5: Driver tx dma failure in encap = 0 # pciconf -lv em0@pci0:5:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller' class = network subclass = ethernet em1@pci0:5:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller' class = network subclass = ethernet em2@pci0:6:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller' class = network subclass = ethernet em3@pci0:6:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06 hdr=0x00 vendor = 'Intel Corporation' device = '82571EB Gigabit Ethernet Controller' class = network subclass = ethernet em4@pci0:13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82573E Intel Corporation 82573E Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet em5@pci0:15:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086 rev=0x00 hdr=0x00 vendor = 'Intel Corporation' device = '82573L Intel PRO/1000 PL Network Adaptor' class = network subclass = ethernet # uname -a FreeBSD turtle 7.2-STABLE FreeBSD 7.2-STABLE #0: Thu Sep 24 08:43:29 PDT 2009 root@turtle:/usr/obj/usr/src/sys/TURTLE i386 I know... old kernel, but the box has been pretty stable, and in 2009 I asked the list about these watchdogs, upgraded from 7.0 and the problem didn't go away, so I don't think kernel upgrades help... Rudy Robin Sommer wrote: > Hello all, > > quite a while ago I asked about the problem below. Unfortunately, I > haven't found a solution yet and I'm actually still seeing these > timeouts after just upgrading to 8.2-RC1. Any further ideas on what > could be triggering them, or how I could track down the cause? > > Thanks, > > Robin > > On Thu, Jul 29, 2010 at 14:56 -0700, I wrote: > > >> Since upgrading from 8.0 to 8.1-RELEASE, I'm seeing lots of messages >> like those below on all my SuperMicro SBI-7425C-T3 blades. There's >> almost no traffic on those interfaces. >> >> Any idea? >> >> Thanks, >> >> Robin >> >> Jul 29 13:01:18 blade0 kernel: igb1: Watchdog timeout -- resetting >> Jul 29 13:01:18 blade0 kernel: igb1: Queue(0) tdh = 256, hw tdt = 266 >> Jul 29 13:01:18 blade0 kernel: igb1: TX(0) desc avail = 1013,Next TX to Clean = 255 >> Jul 29 13:01:18 blade0 kernel: igb1: link state changed to DOWN >> Jul 29 13:01:18 blade0 kernel: igb1: link state changed to UP >> Jul 29 13:01:29 blade0 kernel: igb1: Watchdog timeout -- resetting >> Jul 29 13:01:29 blade0 kernel: igb1: Queue(0) tdh = 0, hw tdt = 10 >> Jul 29 13:01:29 blade0 kernel: igb1: TX(0) desc avail = 1014,Next TX to Clean = 0 >> Jul 29 13:01:29 blade0 kernel: igb1: link state changed to DOWN >> Jul 29 13:01:29 blade0 kernel: igb1: link state changed to UP >> Jul 29 13:01:46 blade0 kernel: igb1: Watchdog timeout -- resetting >> Jul 29 13:01:46 blade0 kernel: igb1: Queue(0) tdh = 32, hw tdt = 33 >> Jul 29 13:01:46 blade0 kernel: igb1: TX(0) desc avail = 1022,Next TX to Clean = 31 >> Jul 29 13:01:46 blade0 kernel: igb1: link state changed to DOWN >> Jul 29 13:01:46 blade0 kernel: igb1: link state changed to UP >> Jul 29 13:01:57 blade0 kernel: igb1: Watchdog timeout -- resetting >> Jul 29 13:01:57 blade0 kernel: igb1: Queue(0) tdh = 0, hw tdt = 10 >> Jul 29 13:01:57 blade0 kernel: igb1: TX(0) desc avail = 1014,Next TX to Clean = 0 >> Jul 29 13:01:57 blade0 kernel: igb1: link state changed to DOWN >> Jul 29 13:01:58 blade0 kernel: igb1: link state changed to UP >> Jul 29 13:02:13 blade0 kernel: igb1: Watchdog timeout -- resetting >> >> >>> grep igb /var/run/dmesg.boot >>> >> igb0: port 0x2000-0x201f mem 0xfc940000-0xfc95ffff,0xfc920000-0xfc93ffff,0xfc900000-0xfc903fff irq 16 at device 0.0 on pci4 >> igb0: [FILTER] >> igb0: Ethernet address: 00:30:48:9e:22:00 >> igb1: port 0x2020-0x203f mem 0xfc980000-0xfc99ffff,0xfc960000-0xfc97ffff,0xfc904000-0xfc907fff irq 17 at device 0.1 on pci4 >> igb1: [FILTER] >> igb1: Ethernet address: 00:30:48:9e:22:01 >> >> >>> pciconf -lv >>> >> [...] >> igb0@pci0:4:0:0: class=0x020000 card=0x10a915d9 >> chip=0x10a98086 rev=0x02 hdr=0x00 >> vendor = 'Intel Corporation' >> device = '82575EB Gigabit Backplane Connection' >> class = network >> subclass = ethernet >> igb1@pci0:4:0:1: class=0x020000 card=0x10a915d9 >> chip=0x10a98086 rev=0x02 hdr=0x00 >> vendor = 'Intel Corporation' >> device = '82575EB Gigabit Backplane Connection' >> class = network >> subclass = ethernet >> [...] >> > >