Date: Mon, 11 Jan 2010 17:22:49 -0800 From: David Ehrmann <ehrmann@gmail.com> To: pyunyh@gmail.com Cc: freebsd-current@freebsd.org Subject: Re: vge traffic problem Message-ID: <4B4BCEE9.1060600@gmail.com> In-Reply-To: <20100112000859.GD1228@michelle.cdnetworks.com> References: <4B40AFFA.6090706@gmail.com> <20100103221630.GV1166@michelle.cdnetworks.com> <4B47B4F6.8030106@gmail.com> <20100109013145.GG18529@michelle.cdnetworks.com> <4B4ACD68.5030907@gmail.com> <20100111203557.GB1228@michelle.cdnetworks.com> <4B4BB679.2060500@gmail.com> <20100112000859.GD1228@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Pyun YongHyeon wrote:
>> I ran the test, again, and had tcpdump capture the packets (on the host
>> with the vge interface).
>>
>> tcpdump -i vge0 -w dump.cap -K -s 0 host 10.0.1.2
>>
>> When I opened it up in Wireshark, it's reporting that the outgoing UDP
>> checksums are incorrect; they're always 0x1ae3. That said, maybe the
>>
>
> Because bpf(4) see the frame before controller inserts checksum
> value it always reports incorrect checksum. It's normal for TX
> checksum offload capable controller.
> In order to verify the hardware assisted checksum you should check
> that on receiver side. If the checksum was invalid, receiver
> dropped them. See "netstat -s | grep bad" on receiver host.
>
>
I did this one between two similar FreeBSD systems (8.0 and 8.0 RC1, I
think). There wasn't an error in netstat -s on the receiver, but I ran
tcpdump, anyway. The checksums were good, but iperf ended with "read
failed: connection refused." On the non-vge box, once the vge box is
done sending, it sends four identical UDP packets to the vge host, I
think to ask it for sending statistics. Instead of statistics, it got
back an ICMP packet saying "port unreachable." This happens with a
freebsd client, but not a linux client (I didn't confirm the ICMP
message with tcpdump, though). All iperf versions are the same (2.0.4),
but the linux one doesn't have any threading (which shouldn't be an
issue when I'm not using that option).
>> checksums are done in hardware AFTER tcpdump sees them.
>>
>> I set net.inet.udp.checksum to 0. The bad checksums are gone, but I
>> still see dropped packets.
>>
>>
>
> This doesn't disable TX checksum offload of driver. Instead use
> #ifconfig vge0 -txcsum
>
>
I'm still losing packets, but now it's interesting. The other FreeBSD
machine (a VMware VM on a fairly fast machine with another gigabit
connection) loses just a few, now, and sometimes none. The linux box
(200mhz, runs OpenWrt, 100mbps ethernet) still loses more than 1% of
packets. I cut iperf back to 100kbps. The results were much better (0
packets lost), but this did happen one time:
[ 3] WARNING: did not receive ack of last datagram after 10 tries.
Maybe this is related to "port unreachable," and maybe to my nfs
problem. The drops seem to be related to iperf not being able to do
much processing at 200 mhz (which is a little surprising; I was only
asking for 1Mbps).
>> It's on the motherboard, probably wired directly to a PCI-E connection,
>> but yes, it is a VT6130.
>>
>
> Show me "pciconf -lcv" output.
>
vge0@pci0:3:0:0: class=0x020000 card=0x01101106 chip=0x31191106
rev=0x82 hdr=0x00
vendor = 'VIA Technologies Inc'
device = ''Velocity' Gigabit Ethernet Controllers
(VT6120/VT6121/VT6122)'
class = network
subclass = ethernet
cap 01[50] = powerspec 3 supports D0 D1 D2 D3 current D0
cap 10[90] = PCI-Express 1 endpoint max data 128(128) link x1(x1)
cap 05[c0] = MSI supports 1 message, 64 bit, vector masks
It might not be vge, after all, and might just be its speed...but that
shouldn't cause a tcp nfs issue.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4B4BCEE9.1060600>