Date: Tue, 04 Jan 2011 19:57:10 -0800 From: "Rudy (bulk)" <crapsh@monkeybrains.net> To: Robin Sommer <robin@icir.org> Cc: freebsd-net <freebsd-net@freebsd.org> Subject: Re: igb watchdog timeouts Message-ID: <4D23EC16.9040005@monkeybrains.net> In-Reply-To: <20110103210209.GA13091@icir.org> References: <20100729215649.GB2615@icir.org> <20110103210209.GA13091@icir.org>
next in thread | previous in thread | raw e-mail | index | archive | help
I get them as well... on my em devices. I was just thinking of
upgrading to a dual port igb I have kicking around, but your email is
not encouraging. :)
# grep watchdog /var/log/message
Jan 2 21:13:38 turtle kernel: em5: watchdog timeout -- resetting
Jan 3 04:31:37 turtle kernel: em4: watchdog timeout -- resetting
Jan 3 09:41:09 turtle kernel: em4: watchdog timeout -- resetting
Jan 3 12:05:05 turtle kernel: em4: watchdog timeout -- resetting
Jan 4 10:29:58 turtle kernel: em1: watchdog timeout -- resetting
Jan 4 15:36:19 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 16:09:51 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 18:28:48 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 18:33:41 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 19:18:35 turtle kernel: em4: watchdog timeout -- resetting
Jan 4 19:26:21 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 19:27:25 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 19:33:33 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 19:40:54 turtle kernel: em5: watchdog timeout -- resetting
Jan 4 19:50:45 turtle kernel: em5: watchdog timeouts = 129
Jan 4 19:54:33 turtle kernel: em0: watchdog timeouts = 0
Jan 4 19:54:39 turtle kernel: em1: watchdog timeouts = 13
Jan 4 19:54:41 turtle kernel: em2: watchdog timeouts = 0
Jan 4 19:54:44 turtle kernel: em3: watchdog timeouts = 0
Jan 4 19:54:46 turtle kernel: em4: watchdog timeouts = 121
stats info:
Jan 4 19:50:45 turtle kernel: em5: Excessive collisions = 0
Jan 4 19:50:45 turtle kernel: em5: Sequence errors = 0
Jan 4 19:50:45 turtle kernel: em5: Defer count = 0
Jan 4 19:50:45 turtle kernel: em5: Missed Packets = 2280947
Jan 4 19:50:45 turtle kernel: em5: Receive No Buffers = 0
Jan 4 19:50:45 turtle kernel: em5: Receive Length Errors = 0
Jan 4 19:50:45 turtle kernel: em5: Receive errors = 0
Jan 4 19:50:45 turtle kernel: em5: Crc errors = 0
Jan 4 19:50:45 turtle kernel: em5: Alignment errors = 0
Jan 4 19:50:45 turtle kernel: em5: Collision/Carrier extension errors = 0
Jan 4 19:50:45 turtle kernel: em5: RX overruns = 115
Jan 4 19:50:45 turtle kernel: em5: watchdog timeouts = 129
Jan 4 19:50:45 turtle kernel: em5: RX MSIX IRQ = 0 TX MSIX IRQ = 0 LINK
MSIX IRQ = 0
Jan 4 19:50:45 turtle kernel: em5: XON Rcvd = 0
Jan 4 19:50:45 turtle kernel: em5: XON Xmtd = 0
Jan 4 19:50:45 turtle kernel: em5: XOFF Rcvd = 0
Jan 4 19:50:45 turtle kernel: em5: XOFF Xmtd = 0
Jan 4 19:50:45 turtle kernel: em5: Good Packets Rcvd = 23188157965
Jan 4 19:50:45 turtle kernel: em5: Good Packets Xmtd = 42184614153
Jan 4 19:50:45 turtle kernel: em5: TSO Contexts Xmtd = 52043
Jan 4 19:50:45 turtle kernel: em5: TSO Contexts Failed = 0
debug info:
Jan 4 19:51:15 turtle kernel: em5: Adapter hardware address = 0xc51ca420
Jan 4 19:51:15 turtle kernel: em5: CTRL = 0x140248 RCTL = 0x8002
Jan 4 19:51:15 turtle kernel: em5: Packet buffer = Tx=20k Rx=12k
Jan 4 19:51:15 turtle kernel: em5: Flow control watermarks high = 10240
low = 8740
Jan 4 19:51:15 turtle kernel: em5: tx_int_delay = 66, tx_abs_int_delay = 66
Jan 4 19:51:15 turtle kernel: em5: rx_int_delay = 32, rx_abs_int_delay = 66
Jan 4 19:51:15 turtle kernel: em5: fifo workaround = 0,
fifo_reset_count = 0
Jan 4 19:51:15 turtle kernel: em5: hw tdh = 180, hw tdt = 180
Jan 4 19:51:15 turtle kernel: em5: hw rdh = 708, hw rdt = 707
Jan 4 19:51:15 turtle kernel: em5: Num Tx descriptors avail = 2048
Jan 4 19:51:15 turtle kernel: em5: Tx Descriptors not avail1 = 0
Jan 4 19:51:15 turtle kernel: em5: Tx Descriptors not avail2 = 0
Jan 4 19:51:15 turtle kernel: em5: Std mbuf failed = 0
Jan 4 19:51:15 turtle kernel: em5: Std mbuf cluster failed = 0
Jan 4 19:51:15 turtle kernel: em5: Driver dropped packets = 0
Jan 4 19:51:15 turtle kernel: em5: Driver tx dma failure in encap = 0
# pciconf -lv
em0@pci0:5:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06
hdr=0x00
vendor = 'Intel Corporation'
device = '82571EB Gigabit Ethernet Controller'
class = network
subclass = ethernet
em1@pci0:5:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06
hdr=0x00
vendor = 'Intel Corporation'
device = '82571EB Gigabit Ethernet Controller'
class = network
subclass = ethernet
em2@pci0:6:0:0: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06
hdr=0x00
vendor = 'Intel Corporation'
device = '82571EB Gigabit Ethernet Controller'
class = network
subclass = ethernet
em3@pci0:6:0:1: class=0x020000 card=0x10a48086 chip=0x10a48086 rev=0x06
hdr=0x00
vendor = 'Intel Corporation'
device = '82571EB Gigabit Ethernet Controller'
class = network
subclass = ethernet
em4@pci0:13:0:0: class=0x020000 card=0x108c15d9 chip=0x108c8086
rev=0x03 hdr=0x00
vendor = 'Intel Corporation'
device = '82573E Intel Corporation 82573E Gigabit Ethernet
Controller (Copper)'
class = network
subclass = ethernet
em5@pci0:15:0:0: class=0x020000 card=0x109a15d9 chip=0x109a8086
rev=0x00 hdr=0x00
vendor = 'Intel Corporation'
device = '82573L Intel PRO/1000 PL Network Adaptor'
class = network
subclass = ethernet
# uname -a
FreeBSD turtle 7.2-STABLE FreeBSD 7.2-STABLE #0: Thu Sep 24 08:43:29 PDT
2009 root@turtle:/usr/obj/usr/src/sys/TURTLE i386
I know... old kernel, but the box has been pretty stable, and in 2009 I
asked the list about these watchdogs, upgraded from 7.0 and the problem
didn't go away, so I don't think kernel upgrades help...
Rudy
Robin Sommer wrote:
> Hello all,
>
> quite a while ago I asked about the problem below. Unfortunately, I
> haven't found a solution yet and I'm actually still seeing these
> timeouts after just upgrading to 8.2-RC1. Any further ideas on what
> could be triggering them, or how I could track down the cause?
>
> Thanks,
>
> Robin
>
> On Thu, Jul 29, 2010 at 14:56 -0700, I wrote:
>
>
>> Since upgrading from 8.0 to 8.1-RELEASE, I'm seeing lots of messages
>> like those below on all my SuperMicro SBI-7425C-T3 blades. There's
>> almost no traffic on those interfaces.
>>
>> Any idea?
>>
>> Thanks,
>>
>> Robin
>>
>> Jul 29 13:01:18 blade0 kernel: igb1: Watchdog timeout -- resetting
>> Jul 29 13:01:18 blade0 kernel: igb1: Queue(0) tdh = 256, hw tdt = 266
>> Jul 29 13:01:18 blade0 kernel: igb1: TX(0) desc avail = 1013,Next TX to Clean = 255
>> Jul 29 13:01:18 blade0 kernel: igb1: link state changed to DOWN
>> Jul 29 13:01:18 blade0 kernel: igb1: link state changed to UP
>> Jul 29 13:01:29 blade0 kernel: igb1: Watchdog timeout -- resetting
>> Jul 29 13:01:29 blade0 kernel: igb1: Queue(0) tdh = 0, hw tdt = 10
>> Jul 29 13:01:29 blade0 kernel: igb1: TX(0) desc avail = 1014,Next TX to Clean = 0
>> Jul 29 13:01:29 blade0 kernel: igb1: link state changed to DOWN
>> Jul 29 13:01:29 blade0 kernel: igb1: link state changed to UP
>> Jul 29 13:01:46 blade0 kernel: igb1: Watchdog timeout -- resetting
>> Jul 29 13:01:46 blade0 kernel: igb1: Queue(0) tdh = 32, hw tdt = 33
>> Jul 29 13:01:46 blade0 kernel: igb1: TX(0) desc avail = 1022,Next TX to Clean = 31
>> Jul 29 13:01:46 blade0 kernel: igb1: link state changed to DOWN
>> Jul 29 13:01:46 blade0 kernel: igb1: link state changed to UP
>> Jul 29 13:01:57 blade0 kernel: igb1: Watchdog timeout -- resetting
>> Jul 29 13:01:57 blade0 kernel: igb1: Queue(0) tdh = 0, hw tdt = 10
>> Jul 29 13:01:57 blade0 kernel: igb1: TX(0) desc avail = 1014,Next TX to Clean = 0
>> Jul 29 13:01:57 blade0 kernel: igb1: link state changed to DOWN
>> Jul 29 13:01:58 blade0 kernel: igb1: link state changed to UP
>> Jul 29 13:02:13 blade0 kernel: igb1: Watchdog timeout -- resetting
>>
>>
>>> grep igb /var/run/dmesg.boot
>>>
>> igb0: <Intel(R) PRO/1000 Network Connection version - 1.9.5> port 0x2000-0x201f mem 0xfc940000-0xfc95ffff,0xfc920000-0xfc93ffff,0xfc900000-0xfc903fff irq 16 at device 0.0 on pci4
>> igb0: [FILTER]
>> igb0: Ethernet address: 00:30:48:9e:22:00
>> igb1: <Intel(R) PRO/1000 Network Connection version - 1.9.5> port 0x2020-0x203f mem 0xfc980000-0xfc99ffff,0xfc960000-0xfc97ffff,0xfc904000-0xfc907fff irq 17 at device 0.1 on pci4
>> igb1: [FILTER]
>> igb1: Ethernet address: 00:30:48:9e:22:01
>>
>>
>>> pciconf -lv
>>>
>> [...]
>> igb0@pci0:4:0:0: class=0x020000 card=0x10a915d9
>> chip=0x10a98086 rev=0x02 hdr=0x00
>> vendor = 'Intel Corporation'
>> device = '82575EB Gigabit Backplane Connection'
>> class = network
>> subclass = ethernet
>> igb1@pci0:4:0:1: class=0x020000 card=0x10a915d9
>> chip=0x10a98086 rev=0x02 hdr=0x00
>> vendor = 'Intel Corporation'
>> device = '82575EB Gigabit Backplane Connection'
>> class = network
>> subclass = ethernet
>> [...]
>>
>
>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4D23EC16.9040005>