Date: Sun, 16 Jan 2011 08:54:59 -0500 (EST) From: Rick Macklem <rmacklem@uoguelph.ca> To: freebsd-net@freebsd.org Cc: Ronald Klop <ronald-freebsd8@klop.yi.org> Subject: bogus 0 len IP packet, was: Hang in VOP_LOCK1_APV on 8-STABLE with NFS. Message-ID: <708793006.292748.1295186099577.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <op.vpekz9uz8527sy@212-123-145-58.ip.telfort.nl>
next in thread | previous in thread | raw e-mail | index | archive | help
Ronald has reported having a problem with the FreeBSD NFS client using 8.2-prerelease. I've redirected it here, since it looks like there is a TCP/IP issue that is causing it. > > >> > >> These are the links to the dumps: > >> http://klop.ws/~ronald/nfs-problem/procstat.nolockd > >> http://klop.ws/~ronald/nfs-problem/ps.nolockd > >> http://klop.ws/~ronald/nfs-problem/linux5.nfs.nolockd.dump.gz > >> http://klop.ws/~ronald/nfs-problem/linux5.nfs.with_rpc_patch.dump.gz > >> I looked at the last of these via wireshark and it seems that the FreeBSD client is sending bogus TCP/IP packets with a IP length == 0. (If you look at the above dump (...with_rpc_patch.dump.gz), the first one is at packet #46, then #3024, then persistently starting at #3234.) Basically the packet looks like: frame len: 4446 MAC: dst 00:0d:56:70:b7:6c src b8:ac:6f:47:73:6e type: 08 00 (IP) IP version: 4 header length: 20 differentiated services field: 0x00 total length: 0 - followed by what looks like a legitimate TCP/IP packet Here's the first bytes of the raw packet data: 00 0d 56 70 b7 6c b8 ac 6f 47 73 6e 08 00 45 00 00 00 ... After this packet is sent to the Linux server, it replies with a TCP ack, which gets ACK'd from FreeBSD as well. For the persistent case, it just keeps doing this (bogus 0 length packet from FreeBSD -> Linux server, followed by the two TCP ack packets) over and over and over again, to the end of the dump. So, does anyone have an idea why the IP length field would be set to 0 for these TCP/IP packets? Here's some info from Ronald w.r.t. his hardware. (All I can think of is that he could try disabling TSO, etc?) Thanks in advance for any help with this, rick > > I just looked at the last dump and there seems to be a network > > issue. > > (It first shows up at packet #46, then again at #3025, then > > persistently > > starting at #3234.) > > > > I'd like to post on freebsd-net@ to see if anyone more conversant > > with > > TCP/IP can look, but first I'd like to get a little more info on > > your > > hardware/software config. > > > > In particular, what network hardware does the FreeBSD client use? > > > > And I assume the server is some variant of Linux? > > > > Thanks for creating the tcpdumps, rick > > ps: If you look on wireshark, the problem seems to start with a > > badly formed IP datagram that then causes acks in both > > directions. > > We are getting off list now. I don't know how good that is. > I've redirected it to freebsd-net@ in the hopes that networking folks can help. > But here is some info. I also noticed the bad packets. And my computer > is > resending the same info at quite a high rate (MB/s). > The server is an up-to-date Linux Debian 5 with a 2.6.26-1-686-bigmem > kernel. Colleagues using Linux clients don't have these problems. > > dmesg | grep em0 > em0: <Intel(R) PRO/1000 Network Connection 7.1.8> port 0xece0-0xecff > mem > 0xf7fe0000-0xf7ffffff,0xf7fd9000-0xf7fd9fff irq 21 at device 25.0 on > pci0 > em0: Using an MSI interrupt > em0: [FILTER] > em0: Ethernet address: b8:ac:6f:47:73:6e > > pciconf -lv > em0@pci0:0:25:0: class=0x020000 card=0x02761028 chip=0x10de8086 > rev=0x02 hdr=0x00 > vendor = 'Intel Corporation' > device = 'Intel Gigabit network connection (82567LM-3 )' > class = network > subclass = ethernet > > > [root@ronald ~]# ifconfig em0 > em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu > 1500 > options=219b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,TSO4,WOL_MAGIC> > ether b8:ac:6f:47:73:6e > inet 10.1.20.49 netmask 0xffffff00 broadcast 10.1.20.255 > media: Ethernet autoselect (100baseTX <full-duplex>) > status: active > > Thanks for looking into this. If you need more just let me know. I can > also reproduce the problem and send nfsstat or netstat output or .... > > Thanks, > Ronald.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?708793006.292748.1295186099577.JavaMail.root>