Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 24 Oct 2012 11:55:25 -0700
From:      Jeremy Chadwick <jdc@koitsu.org>
To:        Harald Schmalzbauer <h.schmalzbauer@omnilan.de>
Cc:        Adrian Chadd <adrian@freebsd.org>, FreeBSD Stable <freebsd-stable@freebsd.org>
Subject:   Re: every 2nd echo-request malformed when ping -s >4067
Message-ID:  <20121024185525.GA6426@icarus.home.lan>
In-Reply-To: <20121024181239.GA5755@icarus.home.lan>
References:  <20121024154017.GA3167@icarus.home.lan> <5088163E.2090506@omnilan.de> <20121024165148.GA4250@icarus.home.lan> <50881EC7.9030400@omnilan.de> <20121024174425.GA4699@icarus.home.lan> <50882D3B.5050704@omnilan.de> <20121024181239.GA5755@icarus.home.lan>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Oct 24, 2012 at 11:12:39AM -0700, Jeremy Chadwick wrote:
> On Wed, Oct 24, 2012 at 08:02:35PM +0200, Harald Schmalzbauer wrote:
> > Please find attached the requested info.
> 
> Thanks, got 'em!  I'll reply in a follow-up mail with the decoded
> results.

As promised, here are the decoded results.  Took me longer than I
expected since I started going down the road of IP options and then was
like, "no, wait a minute, this is ICMP gah!".  Opinions are at the
bottom.  Gosh I hope I didn't botch a copy-paste on this one...

17:58:08.481888 IP 10.5.49.126 > 10.5.49.65: ICMP echo request, id 49423, seq 0, length 4076
	0x0000:  4500 1000 1fff 4000 4001 9435 0a05 317e
	0x0010:  0a05 3141 0800 a352 c10f 0000 5088 2c30
	0x0020:  0007 5a3b {...snip...}

0x45       = bits 7-4: IPv4 protocol
           = bits 3-0: header length: 20 bytes
0x00       = DSF / RFC 2474 stuff
0x1000     = datagram length: 4096 bytes
0x1fff     = fragment id
0x4000     = bits 15-13: %010 = reserved bit (0), DF bit (1), MF bit (0)
           = bits 12-0:  fragment offset: 0
0x40       = TTL: 64
0x01       = protocol: 1 (ICMP)
0x9435     = header checksum
0x0a05317e = source IP
0x0a053141 = destination IP
0x08       = ICMP type: 8 = Echo Request
0x00       = ICMP code: 0 = always zero for ICMP type 8
0xa352     = ICMP header checksum
0xc10f     = ICMP identifier
0x0000     = ICMP sequence number
0x5088     = timestamp from ICMP data
0x2c30     = timestamp from ICMP data
0x0007     = timestamp from ICMP data
0x5a3b     = timestamp from ICMP data


17:58:09.488461 IP 10.5.49.126 > 10.5.49.65: icmp
	0x0000:  4500 1000 1fff 0040 4001 d3f5 0a05 317e
	0x0010:  0a05 3141 0800 8998 c10f 0001 5088 2c31
	0x0020:  0007 73f3 {...snip...}

0x45       = bits 7-4: IPv4 protocol
           = bits 3-0: header length: 20 bytes
0x00       = DSF / RFC 2474 stuff
0x1000     = datagram length: 4096 bytes
0x1fff     = fragment id
0x0040     = bits 15-13: %000 = reserved bit (0), DF bit (0), MF bit (0)
           = bits 12-0:  fragment offset: 64
0x40       = TTL: 64
0x01       = protocol: 1 (ICMP)
0xd3f5     = header checksum
0x0a05317e = source IP
0x0a053141 = destination IP
0x08       = ICMP type: 8 = Echo Request
0x00       = ICMP code: 0 = always zero for ICMP type 8
0x8998     = ICMP header checksum
0xc10f     = ICMP identifier
0x0001     = ICMP sequence number
0x5088     = timestamp from ICMP data
0x2c31     = timestamp from ICMP data
0x0007     = timestamp from ICMP data
0x73f3     = timestamp from ICMP data


Summary: I don't see anything anomalous EXCEPT the ordeal regarding the
fragment offset going from 0->64 and the DF bit going from 1->0.
Possibly this makes tcpdump throw a fit in some way, I'm not sure.

If I had to take a guess?  I'd say some code somewhere is incorrectly
shifting bits (I'm thinking along the lines of a hardware latch counter)
or possibly has a byte-ordering problem.

I'd love to say "big vs. little endian issue" except I'd expect that to
manifest itself constantly time and not just every other packet.  But
heck if I know what NIC drivers and their IP stack tie-ins are doing
under the hood.

The good part is that you can reproduce this reliably.

P.S. -- Just saw the mail from Barney -- I think he's referring to IP,
TCP, and UDP checksums (you can disable that calculation check using
tcpdump -K), but that's only in tcpdump.  There is also the checksum
offloading features on NICs (see ifconfig).  I have not the slightest
idea if these NIC features could cause this problem / make it manifest
in this way.

P.P.S. -- Just saw Adrian's comment about zero copy sockets.  No
idea if it's relevant or not, but disable it anyway because it's going
to bite you in the butt come FreeBSD 10.x anyway:

http://www.freshbsd.org/commit/freebsd/r241955
http://lists.freebsd.org/pipermail/freebsd-current/2012-October/037354.html

-- 
| Jeremy Chadwick                                   jdc@koitsu.org |
| UNIX Systems Administrator                http://jdc.koitsu.org/ |
| Mountain View, CA, US                                            |
| Making life hard for others since 1977.             PGP 4BD6C0CB |



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121024185525.GA6426>