From owner-freebsd-net@FreeBSD.ORG Sun Nov 28 10:27:35 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E6724106566B for ; Sun, 28 Nov 2010 10:27:35 +0000 (UTC) (envelope-from jhay@meraka.csir.co.za) Received: from zibbi.meraka.csir.co.za (unknown [IPv6:2001:4200:7000:2::1]) by mx1.freebsd.org (Postfix) with ESMTP id 5B1658FC15 for ; Sun, 28 Nov 2010 10:27:34 +0000 (UTC) Received: by zibbi.meraka.csir.co.za (Postfix, from userid 3973) id C1DB239824; Sun, 28 Nov 2010 12:17:02 +0200 (SAST) Date: Sun, 28 Nov 2010 12:17:02 +0200 From: John Hay To: Jack Vogel Message-ID: <20101128101702.GA1574@zibbi.meraka.csir.co.za> References: <201011270946271408828@yahoo.com.cn> <20101128081617.GA90332@zibbi.meraka.csir.co.za> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: freebsd-net , Ryan Stone Subject: Re: Re: 82599 receiving packets with vlan tag=0 (vlan strip problem)? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 28 Nov 2010 10:27:36 -0000 On Sun, Nov 28, 2010 at 01:19:07AM -0800, Jack Vogel wrote: > Well, its not solving this odd vlan tag change problem, that isn't what the > workaround was about, at least I dont see how to connect them. Yes, but I still see the double vlan tags on the mirror port that I connected to ix3, which is what Ryan reported? > So you arent using HW Filters? or does that just not show up in the ifconfig > display? Which HW Filters? I have -rxcsum -lro from previous debugging. I can turn it on again. Or are there others that I do not know about? John > > Jack > > > On Sun, Nov 28, 2010 at 12:16 AM, John Hay wrote: > > > Hi Jack, > > > > On Sat, Nov 27, 2010 at 09:59:10AM -0800, Jack Vogel wrote: > > > Well, that will be cool if so, its not usually FreeBSD that > > > finds bugs, and if it really is hardware then they need to > > > know. Thanks Ryan! > > > > > > Jack > > > > > > > > > On Sat, Nov 27, 2010 at 4:52 AM, Ryan Stone wrote: > > > > > > > 2010/11/26 Jack Vogel : > > > > > Just for the record, I was not aware of a hardware bug, and > > > > > for right now no one is around :) I will ask around next week > > > > > to see if something was known that I missed. > > > > > > > > I doubt that anybody at Intel will know about this one. To my > > > > knowledge, it's not in the errata for the 82599. This is just > > > > something that I discovered independently over the summer, fixed in my > > > > local tree and promptly forgot about. > > > > I don't think your solution / workaround work in every case. To debug my > > other vlan + ipv6 problem, I have configured a mirrored port on the switch > > and connected that to one of the ixgbe interfaces with no vlans configured > > on it. Packets dumped on it looks like this: > > > > ######################################### > > # tcpdump -i ix3 -n -s 0 -e ether host 00:23:ae:a5:00:ef > > tcpdump: WARNING: ix3: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on ix3, link-type EN10MB (Ethernet), capture size 65535 bytes > > 10:00:58.215870 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 0, length 16 > > 10:00:59.207090 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 1, length 16 > > 10:01:00.206506 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 2, length 16 > > 10:01:01.206923 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 3, length 16 > > 10:01:02.206378 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 4, length 16 > > 10:01:03.168219 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 86: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor > > advertisement, tgt is fe80::21b:21ff:fe57:b420, length 24 > > 10:01:03.206846 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 5, length 16 > > 10:01:03.215452 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 94: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, neighbor > > solicitation, who has fe80::223:aeff:fea5:ef, length 32 > > 10:01:04.206331 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 6, length 16 > > 10:01:05.206760 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 7, length 16 > > 10:01:06.206203 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 8, length 16 > > 10:01:07.206794 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, ethertype > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo reply, > > seq 9, length 16 > > ######################################## > > > > On the base interface that actually sent the packets out, it looked like > > this: > > > > ######################################## > > # tcpdump -i ix2 -n -s 0 -e ether host 00:23:ae:a5:00:ef > > tcpdump: WARNING: ix2: no IPv4 address assigned > > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > > listening on ix2, link-type EN10MB (Ethernet), capture size 65535 bytes > > 10:00:58.215838 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 0, length 16 > > 10:00:58.215861 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 0, length 16 > > 10:00:59.207067 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 1, length 16 > > 10:00:59.207081 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 1, length 16 > > 10:01:00.206484 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 2, length 16 > > 10:01:00.206497 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 2, length 16 > > 10:01:01.206899 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 3, length 16 > > 10:01:01.206913 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 3, length 16 > > 10:01:02.206355 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 4, length 16 > > 10:01:02.206368 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 4, length 16 > > 10:01:03.168187 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 90: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, neighbor solicitation, who has > > fe80::21b:21ff:fe57:b420, length 32 > > 10:01:03.168210 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 82: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, neighbor advertisement, tgt is > > fe80::21b:21ff:fe57:b420, length 24 > > 10:01:03.206828 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 5, length 16 > > 10:01:03.206838 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 5, length 16 > > 10:01:03.215445 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 90: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, neighbor solicitation, who has > > fe80::223:aeff:fea5:ef, length 32 > > 10:01:03.215604 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 82: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, neighbor advertisement, tgt is > > fe80::223:aeff:fea5:ef, length 24 > > 10:01:04.206307 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 6, length 16 > > 10:01:04.206321 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 6, length 16 > > 10:01:05.206739 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 7, length 16 > > 10:01:05.206751 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 7, length 16 > > 10:01:06.206181 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 8, length 16 > > 10:01:06.206194 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::21b:21ff:fe57:b420 > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 8, length 16 > > 10:01:07.206772 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, fe80::223:aeff:fea5:ef > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 9, length 16 > > ######################################## > > > > You can see that there is only one vlan tag. > > > > You can also see that all of these were sent out on vlan 1, while by the > > time they arrive on the mirrored port some have been mangled to vlan 8. > > > > This is after updating to the new driver that was merged today. It seems > > that I do not need routing anymore for this condition to happen. Just a > > ping to the link-local address triggers it. > > > > I did this from another machine: > > > > ######################################## > > # ping6 fe80::21b:21ff:fe57:b420%em0 > > PING6(56=40+8+8 bytes) fe80::223:aeff:fea5:ef%em0 --> > > fe80::21b:21ff:fe57:b420%em0 > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=5 hlim=64 time=0.180 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=6 hlim=64 time=0.134 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=7 hlim=64 time=0.145 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=8 hlim=64 time=0.169 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=33 hlim=64 time=0.135 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=34 hlim=64 time=0.145 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=35 hlim=64 time=0.151 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=36 hlim=64 time=0.166 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=37 hlim=64 time=0.159 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=38 hlim=64 time=0.147 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=39 hlim=64 time=0.146 > > ms > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=40 hlim=64 time=0.213 > > ms > > ^C > > --- fe80::21b:21ff:fe57:b420%em0 ping6 statistics --- > > 41 packets transmitted, 12 packets received, 70.7% packet loss > > round-trip min/avg/max/std-dev = 0.134/0.158/0.213/0.021 ms > > ######################################## > > > > Currently the configs look like this: > > > > ######################################## > > mr3# ifconfig ix2 > > ix2: flags=8843 metric 0 mtu 1500 > > options=1b8 > > ether 00:1b:21:57:ef:7c > > inet6 fe80::21b:21ff:fe57:ef7c%ix2 prefixlen 64 scopeid 0x3 > > nd6 options=3 > > media: Ethernet autoselect (10Gbase-SR ) > > status: active > > mr3# ifconfig ix3 > > ix3: flags=8843 metric 0 mtu 1500 > > > > options=1bb > > ether 00:1b:21:57:ef:7d > > inet6 fe80::21b:21ff:fe57:ef7d%ix3 prefixlen 64 scopeid 0x4 > > nd6 options=3 > > media: Ethernet autoselect (10Gbase-SR ) > > status: active > > mr3# ifconfig ix2.1 > > ix2.1: flags=8843 metric 0 mtu 1500 > > ether 00:1b:21:57:ef:7c > > inet 146.64.28.2 netmask 0xffffff00 broadcast 146.64.28.255 > > inet6 fe80::21b:21ff:fe57:b420%ix2.1 prefixlen 64 scopeid 0xa > > inet6 2001:4200:7000:3:21b:21ff:fe57:b420 prefixlen 64 > > inet6 2001:4200:7000:3:: prefixlen 64 anycast > > nd6 options=3 > > media: Ethernet autoselect (10Gbase-SR ) > > status: active > > vlan: 1 parent interface: ix2 > > mr3# ifconfig ix2.8 > > ix2.8: flags=8843 metric 0 mtu 1500 > > ether 00:1b:21:57:ef:7c > > inet 146.64.8.50 netmask 0xffffff00 broadcast 146.64.8.255 > > inet6 fe80::21b:21ff:fe57:b420%ix2.8 prefixlen 64 scopeid 0xb > > inet6 2001:4200:7000:1:21b:21ff:fe57:b420 prefixlen 64 > > inet6 2001:4200:7000:1:: prefixlen 64 anycast > > nd6 options=3 > > media: Ethernet autoselect (10Gbase-SR ) > > status: active > > vlan: 8 parent interface: ix2 > > > > ######################################## > > > > John > > -- > > John Hay -- jhay@meraka.csir.co.za / jhay@FreeBSD.org > >