From owner-freebsd-net@FreeBSD.ORG Sun Nov 28 08:16:21 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 55A1B106566C for ; Sun, 28 Nov 2010 08:16:21 +0000 (UTC) (envelope-from jhay@meraka.csir.co.za) Received: from zibbi.meraka.csir.co.za (unknown [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 58BBE8FC08 for ; Sun, 28 Nov 2010 08:16:19 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id 2D05E39824; Sun, 28 Nov 2010 10:16:17 +0200 (SAST) Date: Sun, 28 Nov 2010 10:16:17 +0200 From: John Hay To: Jack Vogel Message-ID: <20101128081617.GA90332@zibbi.meraka.csir.co.za> References: <201011261037105152721@yahoo.com.cn> <201011270946271408828@yahoo.com.cn> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , Ryan Stone Subject: Re: Re: 82599 receiving packets with vlan tag=0 (vlan strip problem)? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Nov 2010 08:16:21 -0000 Hi Jack, On Sat, Nov 27, 2010 at 09:59:10AM -0800, Jack Vogel wrote: > Well, that will be cool if so, its not usually FreeBSD that > finds bugs, and if it really is hardware then they need to > know. Thanks Ryan! > > Jack > > > On Sat, Nov 27, 2010 at 4:52 AM, Ryan Stone wrote: > > > 2010/11/26 Jack Vogel : > > > Just for the record, I was not aware of a hardware bug, and > > > for right now no one is around :) I will ask around next week > > > to see if something was known that I missed. > > > > I doubt that anybody at Intel will know about this one. To my > > knowledge, it's not in the errata for the 82599. This is just > > something that I discovered independently over the summer, fixed in my > > local tree and promptly forgot about. I don't think your solution / workaround work in every case. To debug my other vlan + ipv6 problem, I have configured a mirrored port on the switch and connected that to one of the ixgbe interfaces with no vlans configured on it. Packets dumped on it looks like this: ######################################### # tcpdump -i ix3 -n -s 0 -e ether host 00:23:ae:a5:00:ef tcpdump: WARNING: ix3: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ix3, link-type EN10MB (Ethernet), capture size 65535 bytes 10:00:58.215870 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 0, length 16 10:00:59.207090 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 1, length 16 10:01:00.206506 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 2, length 16 10:01:01.206923 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 3, length 16 10:01:02.206378 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 4, length 16 10:01:03.168219 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 86: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor advertisement, tgt is fe80::21b:21ff:fe57:b420, length 24 10:01:03.206846 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 5, length 16 10:01:03.215452 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 94: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor solicitation, who has fe80::223:aeff:fea5:ef, length 32 10:01:04.206331 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 6, length 16 10:01:05.206760 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 7, length 16 10:01:06.206203 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 8, length 16 10:01:07.206794 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 9, length 16 ######################################## On the base interface that actually sent the packets out, it looked like this: ######################################## # tcpdump -i ix2 -n -s 0 -e ether host 00:23:ae:a5:00:ef tcpdump: WARNING: ix2: no IPv4 address assigned tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ix2, link-type EN10MB (Ethernet), capture size 65535 bytes 10:00:58.215838 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 0, length 16 10:00:58.215861 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 0, length 16 10:00:59.207067 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 1, length 16 10:00:59.207081 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 1, length 16 10:01:00.206484 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 2, length 16 10:01:00.206497 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 2, length 16 10:01:01.206899 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 3, length 16 10:01:01.206913 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 3, length 16 10:01:02.206355 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 4, length 16 10:01:02.206368 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 4, length 16 10:01:03.168187 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 90: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, neighbor solicitation, who has fe80::21b:21ff:fe57:b420, length 32 10:01:03.168210 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 82: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor advertisement, tgt is fe80::21b:21ff:fe57:b420, length 24 10:01:03.206828 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 5, length 16 10:01:03.206838 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 5, length 16 10:01:03.215445 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 90: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor solicitation, who has fe80::223:aeff:fea5:ef, length 32 10:01:03.215604 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 82: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, neighbor advertisement, tgt is fe80::223:aeff:fea5:ef, length 24 10:01:04.206307 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 6, length 16 10:01:04.206321 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 6, length 16 10:01:05.206739 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 7, length 16 10:01:05.206751 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 7, length 16 10:01:06.206181 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 8, length 16 10:01:06.206194 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 8, length 16 10:01:07.206772 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 9, length 16 ######################################## You can see that there is only one vlan tag. You can also see that all of these were sent out on vlan 1, while by the time they arrive on the mirrored port some have been mangled to vlan 8. This is after updating to the new driver that was merged today. It seems that I do not need routing anymore for this condition to happen. Just a ping to the link-local address triggers it. I did this from another machine: ######################################## # ping6 fe80::21b:21ff:fe57:b420%em0 PING6(56=40+8+8 bytes) fe80::223:aeff:fea5:ef%em0 --> fe80::21b:21ff:fe57:b420%em0 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=5 hlim=64 time=0.180 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=6 hlim=64 time=0.134 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=7 hlim=64 time=0.145 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=8 hlim=64 time=0.169 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=33 hlim=64 time=0.135 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=34 hlim=64 time=0.145 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=35 hlim=64 time=0.151 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=36 hlim=64 time=0.166 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=37 hlim=64 time=0.159 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=38 hlim=64 time=0.147 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=39 hlim=64 time=0.146 ms 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=40 hlim=64 time=0.213 ms ^C --- fe80::21b:21ff:fe57:b420%em0 ping6 statistics --- 41 packets transmitted, 12 packets received, 70.7% packet loss round-trip min/avg/max/std-dev = 0.134/0.158/0.213/0.021 ms ######################################## Currently the configs look like this: ######################################## mr3# ifconfig ix2 ix2: flags=8843 metric 0 mtu 1500 options=1b8 ether 00:1b:21:57:ef:7c inet6 fe80::21b:21ff:fe57:ef7c%ix2 prefixlen 64 scopeid 0x3 nd6 options=3 media: Ethernet autoselect (10Gbase-SR ) status: active mr3# ifconfig ix3 ix3: flags=8843 metric 0 mtu 1500 options=1bb ether 00:1b:21:57:ef:7d inet6 fe80::21b:21ff:fe57:ef7d%ix3 prefixlen 64 scopeid 0x4 nd6 options=3 media: Ethernet autoselect (10Gbase-SR ) status: active mr3# ifconfig ix2.1 ix2.1: flags=8843 metric 0 mtu 1500 ether 00:1b:21:57:ef:7c inet 146.64.28.2 netmask 0xffffff00 broadcast 146.64.28.255 inet6 fe80::21b:21ff:fe57:b420%ix2.1 prefixlen 64 scopeid 0xa inet6 2001:4200:7000:3:21b:21ff:fe57:b420 prefixlen 64 inet6 2001:4200:7000:3:: prefixlen 64 anycast nd6 options=3 media: Ethernet autoselect (10Gbase-SR ) status: active vlan: 1 parent interface: ix2 mr3# ifconfig ix2.8 ix2.8: flags=8843 metric 0 mtu 1500 ether 00:1b:21:57:ef:7c inet 146.64.8.50 netmask 0xffffff00 broadcast 146.64.8.255 inet6 fe80::21b:21ff:fe57:b420%ix2.8 prefixlen 64 scopeid 0xb inet6 2001:4200:7000:1:21b:21ff:fe57:b420 prefixlen 64 inet6 2001:4200:7000:1:: prefixlen 64 anycast nd6 options=3 media: Ethernet autoselect (10Gbase-SR ) status: active vlan: 8 parent interface: ix2 ######################################## John -- John Hay -- jhay@meraka.csir.co.za / jhay@FreeBSD.org