Date: Thu, 1 Jul 2004 22:48:29 -0400 From: Jonathan Chen <jon@freebsd.org> To: freebsd-hackers@freebsd.org Subject: bge on 64-bit pci: corrupt packets Message-ID: <20040701224829.A20882@porthos.spock.org>
next in thread | raw e-mail | index | archive | help
Hi all, I've been experiencing a problem with using bge on 64-bit pci slots. After looking at this for the whole day, I still have no idea what the problem is. Perhaps one of you can help... The setup: SuperMicro P4SCT+ motherboard (E7210 Canterwood ES) 3Com 3c996-SX (Broadcom BCM5701 rev 0x105) FreeBSD 5.2.1 GENERIC kernel Using vlan over the bge card The problem: TCP packets sent over the vlan would get corrupted while sending. Other packet types may or may not be corrupted -- I've only been able to catch TCP so far. The content of the packet would get corrupted, while the checksum generated remains valid for the non-corrupted packet. tcpdump on the sending side does not show any errors, while tcpdump on the receive side shows the problem. (See tcpdump output at bottom of message, bytes 0x210-0x214 should be the string "8901" and not "0123") The corrupted 4 bytes appear to be at the end of the buffer of a write() call. The problem appears to go away when a 32-bit PCI slot is used. I am also unable to reproduce the problem on a 64-bit PCI slot without the use of vlans. Not only were there no tcp checksum errors, datastreams appears intact after the transfer (using md5). In the course of trying to track down the problem, I turned off hardware TCP checksuming for the card (by removing CSUM_TCP from BGE_CSUM_FEATURES). This appears to make matters worse. Now, not only is TCP over vlans broken, TCP traffic over the non-vlaned bge is also corrupt in the same manner. Note that turning off checksuming through ifconfig -txcsum did not have the same effect. (NB: I think turning on/off bge checksuming through ifconfig/ioctl is broken? Shouldn't it set ifp->if_hwassist?) Since the card works fine in a 32-bit slot but breaks in a 64-bit slot, I'm guessing the problem is somehow related to alignment. However, I have no idea how I would go about realigning things, especially since the card uses DMA. Help? BTW, am I correct in assuming that transmit checksum offloading is not used with vlans? -Jon 192.168.1.1.49178 > 192.168.1.2.5001: . [bad tcp cksum 606!] 97779257:97780701(1444) ack 1 win 33212 <nop,nop,timestamp 23564240 23581272> (DF) (ttl 64, id 19704, len 1496) 0x0000 4500 05d8 4cf8 4000 4006 64d4 c0a8 0101 E...L.@.@.d..... 0x0010 c0a8 0102 c01a 1389 affe cfe1 060c 82ee ................ 0x0020 8010 81bc 4f61 0000 0101 080a 0167 8fd0 ....Oa.......g.. 0x0030 0167 d258 3233 3435 3637 3839 3031 3233 .g.X234567890123 0x0040 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0050 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0060 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0070 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0080 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0090 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x00a0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x00b0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x00c0 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x00d0 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x00e0 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x00f0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0100 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0110 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0120 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0130 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0140 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0150 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0160 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0170 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0180 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0190 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x01a0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x01b0 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x01c0 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x01d0 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x01e0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x01f0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0200 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0210 3031 3233 0000 0000 0000 0001 0000 1389 0123............ 0x0220 0000 0000 0000 0000 ffff fc18 3435 3637 ............4567 0x0230 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0240 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0250 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0260 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0270 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0280 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0290 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x02a0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x02b0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x02c0 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x02d0 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x02e0 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x02f0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0300 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0310 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0320 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0330 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0340 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0350 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0360 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0370 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0380 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0390 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x03a0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x03b0 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x03c0 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x03d0 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x03e0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x03f0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0400 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0410 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0420 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0430 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0440 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0450 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0460 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0470 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0480 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0490 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x04a0 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x04b0 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x04c0 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x04d0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x04e0 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x04f0 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0500 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0510 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0520 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0530 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0540 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x0550 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x0560 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x0570 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x0580 3637 3839 3031 3233 3435 3637 3839 3031 6789012345678901 0x0590 3233 3435 3637 3839 3031 3233 3435 3637 2345678901234567 0x05a0 3839 3031 3233 3435 3637 3839 3031 3233 8901234567890123 0x05b0 3435 3637 3839 3031 3233 3435 3637 3839 4567890123456789 0x05c0 3031 3233 3435 3637 3839 3031 3233 3435 0123456789012345 0x05d0 3637 3839 3031 3233 67890123 [packet courtasy of iperf] -Jon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040701224829.A20882>