From owner-freebsd-stable@FreeBSD.ORG Wed Feb 28 09:20:01 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 8CC5516A403 for ; Wed, 28 Feb 2007 09:20:01 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.192.81]) by mx1.freebsd.org (Postfix) with ESMTP id 3F7F413C4A6 for ; Wed, 28 Feb 2007 09:20:01 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from icarus.home.lan (c-71-198-0-135.hsd1.ca.comcast.net[71.198.0.135]) by comcast.net (rwcrmhc11) with ESMTP id <20070228092000m1100jdchie>; Wed, 28 Feb 2007 09:20:00 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 3C0321FA03D; Wed, 28 Feb 2007 01:20:00 -0800 (PST) Date: Wed, 28 Feb 2007 01:20:00 -0800 From: Jeremy Chadwick To: Dimuthu Parussalla Message-ID: <20070228092000.GA51292@icarus.home.lan> Mail-Followup-To: Dimuthu Parussalla , 'Glen Van Lehn' , freebsd-stable@freebsd.org References: <20070215043533.GA3293@icarus.home.lan> <000b01c750bd$5a2d55b0$d801a8c0@dimuthu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <000b01c750bd$5a2d55b0$d801a8c0@dimuthu> X-PGP-Key: http://jdc.parodius.com/pubkey.asc User-Agent: Mutt/1.5.13 (2006-08-11) Cc: freebsd-stable@freebsd.org, 'Glen Van Lehn' Subject: Re: Intermittent network issues with Freebsd 6.2 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Feb 2007 09:20:01 -0000 On Thu, Feb 15, 2007 at 03:54:18PM +1100, Dimuthu Parussalla wrote: > Hi, > > Dmesg output related to bge as follows. > > miibus0: on bge0 > brgphy0: on miibus0 > brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, > 1000baseTX-FDX, auto > bge0: Ethernet address: 00:11:25:e9:7f:58 > bge0: [GIANT-LOCKED] > pcib6: at device 5.0 on pci0 > pci8: on pcib6 > bge1: mem 0xc6ff0000-0xc6ffffff irq > 16 at device 0.0 on pci8 > miibus1: on bge1 > brgphy1: on miibus1 > brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, > 1000baseTX-FDX, auto > bge1: Ethernet address: 00:11:25:e9:7f:59 > bge1: [GIANT-LOCKED] Interestingly enough, this problem just started haunting us too (out of no where), on one of our Supermicro systems. There haven't been any changes to the network in literally months (no one's been to the datacenter since December). Here's our details: * Upstream switch is an HP ProCurve 2626 . All ports used are 100mbit, with auto-select enabled (speed/duplex neg) * Speed/duplex negotiation is being done correctly. We have no throughput problems (either direction) or otherwise * netstat -i -n shows no errors, except for two output errors, which are probably due to the interface being brought down and back up rudely (see below) * Switch shows no errors on either interface * Cabling is good (CAT6 none the less) * Uniprocessor system; kernel not built with SMP What we see: Feb 17 11:22:00 eos kernel: bge0: watchdog timeout -- resetting Feb 17 11:22:00 eos kernel: bge0: link state changed to DOWN Feb 17 11:22:01 eos kernel: bge0: link state changed to UP Feb 24 11:20:56 eos kernel: bge0: watchdog timeout -- resetting Feb 24 11:20:56 eos kernel: bge0: link state changed to DOWN Feb 24 11:20:58 eos kernel: bge0: link state changed to UP These timestamps are awfully suspicious; exactly 7 days apart, almost to the hour? And no, we have no cronjobs or anything else that runs at that time (this box is hardly used for anything). Applicable system information: (I'm including ichsmb/smbus because it shares an IRQ with bge1; nothing shares an IRQ with bge0) bge0: mem 0xd0100000-0xd010ffff irq 18 at device 0.0 on pci4 miibus0: on bge0 brgphy0: on miibus0 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge0: Ethernet address: 00:30:48:81:fc:8a pcib5: irq 19 at device 28.3 on pci0 pci5: on pcib5 bge1: mem 0xd0200000-0xd020ffff irq 19 at device 0.0 on pci5 miibus1: on bge1 brgphy1: on miibus1 brgphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX, 1000baseTX-FDX, auto bge1: Ethernet address: 00:30:48:81:fc:8b ichsmb0: port 0x500-0x51f irq 19 at device 31.3 on pci0 ichsmb0: [GIANT-LOCKED] smbus0: on ichsmb0 smb0: on smbus0 Odd that pciconf -lv shows this as a BCM5750 A1 while the kernel shows this as a BCM5750 B1. Is this indicative of anything? bge0@pci4:0:0: class=0x020000 card=0x02c615d9 chip=0x165914e4 rev=0x11 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5750A1 NetXtreme Gigabit Ethernet PCI Express' class = network subclass = ethernet bge1@pci5:0:0: class=0x020000 card=0x02c615d9 chip=0x165914e4 rev=0x11 hdr=0x00 vendor = 'Broadcom Corporation' device = 'BCM5750A1 NetXtreme Gigabit Ethernet PCI Express' class = network subclass = ethernet [jdc@eos ~]$ vmstat -i interrupt total rate irq4: sio0 6 0 irq6: fdc0 14 0 irq14: ata0 520782 0 irq15: ata1 58 0 irq18: bge0 21839717 11 irq19: bge1+ 32914 0 cpu0: timer 3638265059 1968 Total 3660658550 1981 [jdc@eos ~]$ netstat -in Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll bge0 1500 00:30:48:81:fc:8a 13841423 0 10349370 2 0 bge0 1500 72.20.106/25 72.20.106.2 3590195 - 10348720 - - bge0 1500 72.20.106.3/3 72.20.106.3 2075045 - 0 - - bge0 1500 72.20.106.4/3 72.20.106.4 2003973 - 0 - - bge0 1500 72.20.106.5/3 72.20.106.5 2328549 - 0 - - bge0 1500 72.20.106.6/3 72.20.106.6 2006174 - 0 - - bge1 1500 00:30:48:81:fc:8b 3888 0 29600 0 0 bge1 1500 10 10.72.0.1 2605 - 2605 - - lo0 16384 641 0 641 0 0 lo0 16384 127 127.0.0.1 641 - 641 - - bridg 1500 86:ec:97:73:50:03 26993 0 30885 0 0 tap0 1500 00:bd:ed:13:00:00 25712 0 1286 0 0 If a developer wants access to this box, I can provide it. No serial console at this time (soon, soon...), but can provide root. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |