From owner-freebsd-net@FreeBSD.ORG Mon Nov 29 20:08:00 2010 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3AF041065672 for ; Mon, 29 Nov 2010 20:08:00 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-ey0-f182.google.com (mail-ey0-f182.google.com [209.85.215.182]) by mx1.freebsd.org (Postfix) with ESMTP id 67A938FC08 for ; Mon, 29 Nov 2010 20:07:59 +0000 (UTC) Received: by eyb7 with SMTP id 7so2352112eyb.13 for ; Mon, 29 Nov 2010 12:07:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type; bh=SvYhynHlrWIOs7FEukKKrdhptTMDDlsTySsHBNRFIxI=; b=noKs6cB/bpIqMe1KKvwNagDcrGg3k1MFfcefgZ3DEnjd0P3Nbh1GqNsxuAYpoy5Boy /KFFiYi+XuQaGwKRpwnFk9KA3Lt4vbiZEhIWSkuLY84lED4ReIJytGbbg2ByY8cxiPqD WkpvzakvuJ9JpK+ZDxj2SofKNq49DecuZ3lJE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; b=KrrFlokL7wZOY+hmpOZyBgHJq40P0TGUIPhZjQR1rmydTAedtav47p93uOjNa1arUG aXpnF9XH5zHxwkdX7hfhbc3tOv7s36MmJ/OoZW5bKfa1Td45NBJeGgDDhmmgoXi86ND0 VjzrLCoRpTgMcHQneYhbZcCJXvTxEOOQ7bbjI= MIME-Version: 1.0 Received: by 10.216.179.210 with SMTP id h60mr1371396wem.42.1291061278011; Mon, 29 Nov 2010 12:07:58 -0800 (PST) Received: by 10.216.2.206 with HTTP; Mon, 29 Nov 2010 12:07:57 -0800 (PST) In-Reply-To: <20101128101702.GA1574@zibbi.meraka.csir.co.za> References: <201011270946271408828@yahoo.com.cn> <20101128081617.GA90332@zibbi.meraka.csir.co.za> <20101128101702.GA1574@zibbi.meraka.csir.co.za> Date: Mon, 29 Nov 2010 12:07:57 -0800 Message-ID: From: Jack Vogel To: John Hay Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-net , Ryan Stone Subject: Re: Re: 82599 receiving packets with vlan tag=0 (vlan strip problem)? X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Nov 2010 20:08:00 -0000 You use `ifconfig ix0 vlanhwfilter`, its off by default, I assume you're using the new driver code from either HEAD or STABLE/8 now, right? Try another experiment please: at line 4232 is a condition: if ((adapter->num_vlans) &&... Change that to: if ((vtag) && .... See if either of those change things. Jack On Sun, Nov 28, 2010 at 2:17 AM, John Hay wrote: > On Sun, Nov 28, 2010 at 01:19:07AM -0800, Jack Vogel wrote: > > Well, its not solving this odd vlan tag change problem, that isn't what > the > > workaround was about, at least I dont see how to connect them. > > Yes, but I still see the double vlan tags on the mirror port that I > connected to ix3, which is what Ryan reported? > > > So you arent using HW Filters? or does that just not show up in the > ifconfig > > display? > > Which HW Filters? I have -rxcsum -lro from previous debugging. I can > turn it on again. Or are there others that I do not know about? > > John > > > > > Jack > > > > > > On Sun, Nov 28, 2010 at 12:16 AM, John Hay wrote: > > > > > Hi Jack, > > > > > > On Sat, Nov 27, 2010 at 09:59:10AM -0800, Jack Vogel wrote: > > > > Well, that will be cool if so, its not usually FreeBSD that > > > > finds bugs, and if it really is hardware then they need to > > > > know. Thanks Ryan! > > > > > > > > Jack > > > > > > > > > > > > On Sat, Nov 27, 2010 at 4:52 AM, Ryan Stone > wrote: > > > > > > > > > 2010/11/26 Jack Vogel : > > > > > > Just for the record, I was not aware of a hardware bug, and > > > > > > for right now no one is around :) I will ask around next week > > > > > > to see if something was known that I missed. > > > > > > > > > > I doubt that anybody at Intel will know about this one. To my > > > > > knowledge, it's not in the errata for the 82599. This is just > > > > > something that I discovered independently over the summer, fixed in > my > > > > > local tree and promptly forgot about. > > > > > > I don't think your solution / workaround work in every case. To debug > my > > > other vlan + ipv6 problem, I have configured a mirrored port on the > switch > > > and connected that to one of the ixgbe interfaces with no vlans > configured > > > on it. Packets dumped on it looks like this: > > > > > > ######################################### > > > # tcpdump -i ix3 -n -s 0 -e ether host 00:23:ae:a5:00:ef > > > tcpdump: WARNING: ix3: no IPv4 address assigned > > > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode > > > listening on ix3, link-type EN10MB (Ethernet), capture size 65535 bytes > > > 10:00:58.215870 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 0, length 16 > > > 10:00:59.207090 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 1, length 16 > > > 10:01:00.206506 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 2, length 16 > > > 10:01:01.206923 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 3, length 16 > > > 10:01:02.206378 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 4, length 16 > > > 10:01:03.168219 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 86: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, > neighbor > > > advertisement, tgt is fe80::21b:21ff:fe57:b420, length 24 > > > 10:01:03.206846 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 5, length 16 > > > 10:01:03.215452 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 94: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, > neighbor > > > solicitation, who has fe80::223:aeff:fea5:ef, length 32 > > > 10:01:04.206331 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 6, length 16 > > > 10:01:05.206760 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 7, length 16 > > > 10:01:06.206203 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 1, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 8, length 16 > > > 10:01:07.206794 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 78: vlan 0, p 0, ethertype 802.1Q, vlan 8, p 0, > ethertype > > > IPv6, fe80::21b:21ff:fe57:b420 > fe80::223:aeff:fea5:ef: ICMP6, echo > reply, > > > seq 9, length 16 > > > ######################################## > > > > > > On the base interface that actually sent the packets out, it looked > like > > > this: > > > > > > ######################################## > > > # tcpdump -i ix2 -n -s 0 -e ether host 00:23:ae:a5:00:ef > > > tcpdump: WARNING: ix2: no IPv4 address assigned > > > tcpdump: verbose output suppressed, use -v or -vv for full protocol > decode > > > listening on ix2, link-type EN10MB (Ethernet), capture size 65535 bytes > > > 10:00:58.215838 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 0, length 16 > > > 10:00:58.215861 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 0, length 16 > > > 10:00:59.207067 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 1, length 16 > > > 10:00:59.207081 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 1, length 16 > > > 10:01:00.206484 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 2, length 16 > > > 10:01:00.206497 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 2, length 16 > > > 10:01:01.206899 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 3, length 16 > > > 10:01:01.206913 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 3, length 16 > > > 10:01:02.206355 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 4, length 16 > > > 10:01:02.206368 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 4, length 16 > > > 10:01:03.168187 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 90: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, neighbor solicitation, who has > > > fe80::21b:21ff:fe57:b420, length 32 > > > 10:01:03.168210 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 82: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, neighbor advertisement, tgt is > > > fe80::21b:21ff:fe57:b420, length 24 > > > 10:01:03.206828 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 5, length 16 > > > 10:01:03.206838 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 5, length 16 > > > 10:01:03.215445 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 90: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, neighbor solicitation, who has > > > fe80::223:aeff:fea5:ef, length 32 > > > 10:01:03.215604 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 82: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, neighbor advertisement, tgt is > > > fe80::223:aeff:fea5:ef, length 24 > > > 10:01:04.206307 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 6, length 16 > > > 10:01:04.206321 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 6, length 16 > > > 10:01:05.206739 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 7, length 16 > > > 10:01:05.206751 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 7, length 16 > > > 10:01:06.206181 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 8, length 16 > > > 10:01:06.206194 00:1b:21:57:ef:7c > 00:23:ae:a5:00:ef, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::21b:21ff:fe57:b420 > > > > fe80::223:aeff:fea5:ef: ICMP6, echo reply, seq 8, length 16 > > > 10:01:07.206772 00:23:ae:a5:00:ef > 00:1b:21:57:ef:7c, ethertype 802.1Q > > > (0x8100), length 74: vlan 1, p 0, ethertype IPv6, > fe80::223:aeff:fea5:ef > > > > fe80::21b:21ff:fe57:b420: ICMP6, echo request, seq 9, length 16 > > > ######################################## > > > > > > You can see that there is only one vlan tag. > > > > > > You can also see that all of these were sent out on vlan 1, while by > the > > > time they arrive on the mirrored port some have been mangled to vlan 8. > > > > > > This is after updating to the new driver that was merged today. It > seems > > > that I do not need routing anymore for this condition to happen. Just a > > > ping to the link-local address triggers it. > > > > > > I did this from another machine: > > > > > > ######################################## > > > # ping6 fe80::21b:21ff:fe57:b420%em0 > > > PING6(56=40+8+8 bytes) fe80::223:aeff:fea5:ef%em0 --> > > > fe80::21b:21ff:fe57:b420%em0 > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=5 hlim=64 > time=0.180 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=6 hlim=64 > time=0.134 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=7 hlim=64 > time=0.145 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=8 hlim=64 > time=0.169 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=33 hlim=64 > time=0.135 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=34 hlim=64 > time=0.145 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=35 hlim=64 > time=0.151 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=36 hlim=64 > time=0.166 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=37 hlim=64 > time=0.159 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=38 hlim=64 > time=0.147 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=39 hlim=64 > time=0.146 > > > ms > > > 16 bytes from fe80::21b:21ff:fe57:b420%em0, icmp_seq=40 hlim=64 > time=0.213 > > > ms > > > ^C > > > --- fe80::21b:21ff:fe57:b420%em0 ping6 statistics --- > > > 41 packets transmitted, 12 packets received, 70.7% packet loss > > > round-trip min/avg/max/std-dev = 0.134/0.158/0.213/0.021 ms > > > ######################################## > > > > > > Currently the configs look like this: > > > > > > ######################################## > > > mr3# ifconfig ix2 > > > ix2: flags=8843 metric 0 mtu > 1500 > > > options=1b8 > > > ether 00:1b:21:57:ef:7c > > > inet6 fe80::21b:21ff:fe57:ef7c%ix2 prefixlen 64 scopeid 0x3 > > > nd6 options=3 > > > media: Ethernet autoselect (10Gbase-SR ) > > > status: active > > > mr3# ifconfig ix3 > > > ix3: flags=8843 metric 0 mtu > 1500 > > > > > > > options=1bb > > > ether 00:1b:21:57:ef:7d > > > inet6 fe80::21b:21ff:fe57:ef7d%ix3 prefixlen 64 scopeid 0x4 > > > nd6 options=3 > > > media: Ethernet autoselect (10Gbase-SR ) > > > status: active > > > mr3# ifconfig ix2.1 > > > ix2.1: flags=8843 metric 0 mtu > 1500 > > > ether 00:1b:21:57:ef:7c > > > inet 146.64.28.2 netmask 0xffffff00 broadcast 146.64.28.255 > > > inet6 fe80::21b:21ff:fe57:b420%ix2.1 prefixlen 64 scopeid 0xa > > > inet6 2001:4200:7000:3:21b:21ff:fe57:b420 prefixlen 64 > > > inet6 2001:4200:7000:3:: prefixlen 64 anycast > > > nd6 options=3 > > > media: Ethernet autoselect (10Gbase-SR ) > > > status: active > > > vlan: 1 parent interface: ix2 > > > mr3# ifconfig ix2.8 > > > ix2.8: flags=8843 metric 0 mtu > 1500 > > > ether 00:1b:21:57:ef:7c > > > inet 146.64.8.50 netmask 0xffffff00 broadcast 146.64.8.255 > > > inet6 fe80::21b:21ff:fe57:b420%ix2.8 prefixlen 64 scopeid 0xb > > > inet6 2001:4200:7000:1:21b:21ff:fe57:b420 prefixlen 64 > > > inet6 2001:4200:7000:1:: prefixlen 64 anycast > > > nd6 options=3 > > > media: Ethernet autoselect (10Gbase-SR ) > > > status: active > > > vlan: 8 parent interface: ix2 > > > > > > ######################################## > > > > > > John > > > -- > > > John Hay -- jhay@meraka.csir.co.za / jhay@FreeBSD.org > > > >