Date: Mon, 25 Feb 2008 12:56:17 +0900 From: Pyun YongHyeon <pyunyh@gmail.com> To: Sam Leffler <sam@errno.com> Cc: Ian FREISLICH <ianf@clue.co.za>, Robert Backhaus <robbak@robbak.com>, FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Packet corruption in re0 [checksum offloading] Message-ID: <20080225035617.GF42733@cdnetworks.co.kr> In-Reply-To: <47C0678D.20905@errno.com> References: <20080222042700.GB30497@cdnetworks.co.kr> <E1JSTV4-0000l2-EE@clue.co.za> <20080222094742.GF30497@cdnetworks.co.kr> <47C0678D.20905@errno.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Feb 23, 2008 at 10:35:57AM -0800, Sam Leffler wrote: > Pyun YongHyeon wrote: > >On Fri, Feb 22, 2008 at 10:43:22AM +0200, Ian FREISLICH wrote: > > > Pyun YongHyeon wrote: > > > > On Thu, Feb 21, 2008 at 01:18:18PM +0200, Ian FREISLICH wrote: > > > > > Pyun YongHyeon wrote: > > > > > > On Thu, Feb 21, 2008 at 02:47:43PM +1000, Robert Backhaus wrote: > > > > > > > On Thu, Feb 21, 2008 at 1:50 PM, Pyun YongHyeon > > <pyunyh@gmail.com> wr > > > ote: > > > > > > > > On Thu, Feb 21, 2008 at 11:03:02AM +1000, Robert Backhaus > > wrote: > > > > > > > > > I am experiencing roughly 15% packet corruption on the > > re inter > > > face > > > > > on > > > > > > > > > my freebsd 7/amd64 box. > > > > > > > > > > > > > > > > > > FreeBSD gw.flexi.robbak.com 7.0-PRERELEASE FreeBSD > > 7.0-PRERELEA > > > SE #8 > > > > > : > > > > > > > > > Tue Feb 5 09:49:55 EST 2008 > > > > > > > > > root@gw.flexi.robbak.com:/usr/obj/usr/src/sys/GW amd64 > > > > > > > > > > > > > > > > > > Just to make troubleshooting difficult, this problem > > only shows > > > up > > > > > > > > > after the system has been up for roughly 36 hours, > > depending on > > > the > > > > > > > > > amount of traffic. > > > > > > > > > > > > > > > > > > > > > > > > > I didn't take a look attached tcpdump files but I guess the > > > > > > > > instability issue was fixed in HEAD. It's not yet MFCed but > > > > > > > > I'll handle it in a week. > > > > > > > > > > > > > > > > Would you try re(4) in HEAD? > > > > > > > > > > > > > > > > > > > > > > OK, I'll do that. What is the best way to do that? csupping > > to "." se > > > ems a > > > > > > > bit drastic, and I don't do much with cvs proper. I take it > > that I sh > > > ould > > > > > use > > > > > > > anon-cvs to grab the directory, but I don't quite know how. > > > > > > > > > > > > > > > > > > > Copy sys/dev/re/if_re.c, sys/pci/if_rlreg.h in HEAD to your box. > > > > > > Due to lack of m_defrag(9) in 7-PRERELEASE/RC, you also have to > > add > > > > > > that function to if_re.c(Copy m_defrag() in sys/kern/uipc_mbuf.c > > on > > > > > > HEAD/RELENG_7 to if_re.c). That would make it build on your box. > > > > > > > > > > This doesn't solve the problem that I'm seeing on re(4) interfaces. > > > > > It basically shows up as quagga establishing OSPF neighours as > > > > > "Exchange/DR" when VLAN hardware tagging is enabled. I'm running > > > > > OSPF over 802.1Q vlans. Neighbours are correctly negotiated once > > > > > VLAN hardware tagging is disabled on the interface. > > > > > > > > > > I'll do more debugging. > > > > > > > > > > > > > Hmm. That sounds like different issue to me. I guess I din't change > > > > any semantics in VLAN H/W tagging. Do you still the same VLAN H/W > > > > tagging related issues on RELENG_7? > > > > > > > > To narrow down the issue it would be even better to know which parts > > > > of H/W assistance was broken. For example, > > > > - Disable checksum offload for VLAN interface first and check > > > > whether quagga works. > > > > > > You can only disable offload on the parent interface. > > > > > > >Hmm... I thought it should work. > >I have no idea why ioctl handler of vlan(4) rejects checksum > >offload configutation. I guess vlan(4) should be teached to handle > >this. If parent interface have IFCAP_VLAN_HWCSUM capability and > >IFCAP_VLAN_HWTAGGING, ifconfig(4) should be able to control checksum > >offload for vlan(4) interface. CCed to yar to get his opinions on > >controlling checksum offload on vlan(4). > > > > > > - Disable checksum offload for parent interface and check again. > > > > If you can post tcpdump output for broken conntection it may help a > > > > lot to diagnose the issue. > > > > > > The only flag affecting this behaviour is vlanhwtag. Various > > > permutations of the interface flags make no difference to this > > > behaviour as long as hardware tagging is enabled. > > > > > > >Disabling VLAN HW tagging also turns off checksum offload on vlan(4) > >interface. > > > > > > This reminds me that there are several places in the system where h/w > checksum offload needs to be specially handled but instead is disabled > as a WAR. In particular I'm thinking of the bridge where txcsum is > muted on devices while they are plumbed. But this can be a big loss and > the better approach (IMO) is to fill in the missing capability in s/w. > Agreed. > Not sure what components there are besides bridge and vlan; maybe lagg? > netgraph? > I'm not familiar with lagg(4) and netgraph(4). But lagg(4) should disable Tx checksum offload if one of interface is not capable of hardware checksum offload. > Note there are other capabilities besides checksum offload, TSO can be > done in s/w with good effect. > AFAIK bridge(4) blindly disables Tx checksum offload for all members of a bridge. If all members of a bridge can do checksum offload/TSO with hardware assistance I guess there is no reason to disable these capabilities in bridge environments. The same apply to lagg(4) too. S/W checksum offload/TSO emulation for intefaces without these hardware capabilities would greatly enhance Tx performance when other member of interface of a bridage can make use of hardware offload capability. > Sam > -- Regards, Pyun YongHyeon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080225035617.GF42733>