From owner-freebsd-stable@FreeBSD.ORG Tue Jun 15 03:09:11 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 643341065675; Tue, 15 Jun 2010 03:09:11 +0000 (UTC) (envelope-from artem_kim@inbox.ru) Received: from mx34.mail.ru (mx34.mail.ru [94.100.176.48]) by mx1.freebsd.org (Postfix) with ESMTP id EBE7A8FC14; Tue, 15 Jun 2010 03:09:10 +0000 (UTC) Received: from [79.142.107.13] (port=51530 helo=tt.tt.tt) by mx34.mail.ru with asmtp id 1OOMWT-0006CF-00; Tue, 15 Jun 2010 07:09:09 +0400 From: Artem Kim To: freebsd-stable@freebsd.org, pyunyh@gmail.com Date: Tue, 15 Jun 2010 07:09:27 +0400 User-Agent: KMail/1.12.4 (FreeBSD/8.1-PRERELEASE; KDE/4.3.5; amd64; ; ) References: <201006131934.11389.artem_kim@inbox.ru> <20100614210343.GA4257@michelle.cdnetworks.com> In-Reply-To: <20100614210343.GA4257@michelle.cdnetworks.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201006150709.27396.artem_kim@inbox.ru> X-Spam: Not detected X-Mras: Ok Cc: yongari@freebsd.org Subject: Re: Problems with bge (possibly related to r208993) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 15 Jun 2010 03:09:11 -0000 On Tuesday 15 June 2010 01:03:43 Pyun YongHyeon wrote: > On Sun, Jun 13, 2010 at 07:34:11PM +0400, Artem Kim wrote: > > Hi, > > > > I have two routers (HP DL140G3): > > > > NAS3 FreeBSD 8.1-PRERELEASE # 0: Thu Jun 3 04:13:07 MSD 2010 i386 > > NAS2 FreeBSD 8.1-PRERELEASE # 0: Sat Jun 12 16:42:19 UTC 2010 i386 > > (r208993 included) > > > > bge0 @ pci0: 19:0:0: class = 0x020000 card = 0x3260103c chip = 0x165914e4 > > rev = 0x11 hdr = 0x00 > > vendor = 'Broadcom Corporation' > > device = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > > class = network > > subclass = ethernet > > bge1 @ pci0: 20:0:0: class = 0x020000 card = 0x3260103c chip = 0x165914e4 > > rev = 0x11 hdr = 0x00 > > vendor = 'Broadcom Corporation' > > device = 'NetXtreme Gigabit Ethernet PCI Express (BCM5721)' > > class = network > > subclass = ethernet > > > > > > I have some problems with bge on NAS2. > > > > After some time (about 15 hours) bge1 stops flowing traffic. > > NAS3 NAS3 - pppoe server. Through bge1 passes only ip traffic through > > bge0 no ip-traffic. > > Problems occur only with the bge1 interface on NAS2. > > > > > > Traffic through bge1 not pass until I will not do "ifconfig bge1 down > > ifconfig bge1 up". > > > > When I do "ifconfig bge0 down" NIC does not shutdown: > > > > nas2 # ifconfig bge1 down > > nas2 # > > nas2 # ifconfig bge1 > > bge1: flags = 8843 metric 0 mtu > > 1500 options = 8009b > > > > ether XXXXXXXXXXXXX > > inet YYYYYYYYYYY netmask 0xffffffc0 broadcast YYYYYYYYYYYY > > media: Ethernet autoselect (1000baseT ) > > status: active > > > > LED also indicates that the NIC is active. > > > > I left the NAS in a state of "frozen bge1" - and can provide additional > > information for diagnosis. > > Try run tcpdump on bge1 and see whether driver still see incoming > traffic. Also show me the output of "netstat -ndI bge1" and output > of "sysctl dev.bge.1.stats". Verbose dmesg output also would be > helpful. nas2 # netstat-ndI bge1 Name Mtu Network Address Ipkts Ierrs Idrop Opkts Oerrs Coll Drop bge1 1500 00:1 b: 78: a3: 3c: 01 418543876 1972918 0 446063237 0 0 0 bge1 1500 XX.XX.6.12 XX.XX.6.133 890,306 - - 1,076,833 - - - Should I add additional debugging options? nas2 # sysctl dev.bge.1.stats sysctl: unknown oid 'dev.bge.1.stats' nas2 # sysctl dev.bge.1 dev.bge.1.% desc: Broadcom NetXtreme Gigabit Ethernet Controller, ASIC rev. 0x004101 dev.bge.1.% driver: bge dev.bge.1.% location: slot = 0 function = 0 dev.bge.1.% pnpinfo: vendor = 0x14e4 device = 0x1659 subvendor = 0x103c subdevice = 0x3260 class = 0x020000 dev.bge.1.% parent: pci20 dev.bge.1.forced_collapse: 0 I can show verbose dmesg, but this requires a reboot so bge1 come out of the current state. I looked tcpdump on NAS2 - and I only saw the ARP requests from NAS2 (NAS2 - XX.XX.6.133): nas2 # tcpdump -i bge1 tcpdump: verbose output suppressed, use-v or-vv for full protocol decode listening on bge1, link-type EN10MB (Ethernet), capture size 96 bytes 01:23:43.063238 ARP, Request who-has XX.XX.6.129 tell XX.XX.6.133, length 28 01:23:43.162257 ARP, Request who-has XX.XX.6.129 tell XX.XX.6.133, length 28 01:23:43.935016 ARP, Request who-has XX.XX.6.129 tell XX.XX.6.133, length 28 XX.XX.6.129 is l3-switch(AT-X900) default router for NAS2; bge1 is directly connected to the x900. I looked tcpdump on the x900: awplus # tcpdump -ni vlanXX host XX.XX.6.133 05:36:30.455642 arp who-has XX.XX.6.129 tell XX.XX.6.133 05:36:30.455898 arp reply XX.XX.6.129 is-at 00:00: cd: 29:6 e: 09 05:36:31.483353 arp who-has XX.XX.6.129 tell XX.XX.6.133 05:36:31.483505 arp reply XX.XX.6.129 is-at 00:00: cd: 29:6 e: 09 05:36:32.511260 arp who-has XX.XX.6.129 tell XX.XX.6.133 05:36:32.511353 arp reply XX.XX.6.129 is-at 00:00: cd: 29:6 e: 09 05:36:33.539163 arp who-has XX.XX.6.129 tell XX.XX.6.133 ARP requests from NAS2 (XX.XX.6.133). But on NAS2 I can _only_ see ARP- requests from NAS2. I added static arp-entry on NAS2 and do ping XX.XX.6.129. Then I looked again at tcpdump on XX.XX.6.129: awplus # tcpdump -nei vlanXX host XX.XX.6.133 06:13:03.472539 00:00: cd: 29:6 e: 09> ff: ff: ff: ff: ff: ff, ethertype ARP (0x0806), length 42: arp who-has XX.XX.6.133 tell XX. XX.6.129 06:13:03.526768 00:1 b: 78: a3: 3c: 01> 00:00: cd: 29:6 e: 09, ethertype IPv4 (0x0800), length 98: XX.XX.6.133> XX.XX.6.129: ICMP echo request, id 6958, seq 1920, length 64 06:13:04.553728 00:1 b: 78: a3: 3c: 01> 00:00: cd: 29:6 e: 09, ethertype IPv4 (0x0800), length 98: XX.XX.6.133> XX.XX.6.129: ICMP echo request, id 6958, seq 1921, length 64 06:13:04.554495 00:00: cd: 29:6 e: 09> ff: ff: ff: ff: ff: ff, ethertype ARP (0x0806), length 42: arp who-has XX.XX.6.133 tell XX. XX.6.129 06:13:05.554486 00:00: cd: 29:6 e: 09> ff: ff: ff: ff: ff: ff, ethertype ARP (0x0806), length 42: arp who-has XX.XX.6.133 tell XX. XX.6.129 06:13:05.581488 00:1 b: 78: a3: 3c: 01> 00:00: cd: 29:6 e: 09, ethertype IPv4 (0x0800), length 98: XX.XX.6.133> XX.XX.6.129: ICMP echo request, id 6958, seq 1922, length 64 Traffic comes from XX.XX.6.133 to XX.XX.6.129. I tried to add arp-entry on XX.XX.6.129 (XX.XX.6.133 -> 001b.78a3.3c01) but communication does not restore.