From owner-freebsd-net@freebsd.org Sat Jan 18 11:50:07 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3D7FC1F1C6D for ; Sat, 18 Jan 2020 11:50:07 +0000 (UTC) (envelope-from vas@sibptus.ru) Received: from admin.sibptus.ru (admin.sibptus.ru [IPv6:2001:19f0:5001:21dc::10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 480GSt3nfZz3P8X; Sat, 18 Jan 2020 11:50:06 +0000 (UTC) (envelope-from vas@sibptus.ru) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=sibptus.ru; s=20181118; h=In-Reply-To:Message-ID:Subject:To:From:Date; bh=ZBMyhG4RIznGFOrUGxV9Os0zziVgIxGUF/t6y3qhT68=; b=TFgqfljT20JQXFBNWGKhVRXWpN 0OWdeTnMMckdF63S0wuzNP5I+NJMmYdHf3f6+C0KoL8ClJt0CLKeAlf20PTYfDbuTQNQz4TpRXbQr 2x0b8ccSnkaFjUYVqJ9t2Qvb+Fd8H7UBmXez4JabBF18JoAr22eXquF0BW1mOc4NxP94=; Received: from vas by admin.sibptus.ru with local (Exim 4.92.3 (FreeBSD)) (envelope-from ) id 1ismc8-0003Zg-Lj; Sat, 18 Jan 2020 18:50:04 +0700 Date: Sat, 18 Jan 2020 18:50:04 +0700 From: Victor Sudakov To: Eugene Grosbein Cc: freebsd-net@freebsd.org, "Andrey V. Elsukov" , Michael Tuexen Subject: Re: IPSec transport mode, mtu, fragmentation... Message-ID: <20200118115004.GA13555@admin.sibptus.ru> References: <20200116155305.GA465@admin.sibptus.ru> <55f7bafa-24c4-9810-0d21-f82cb332ee2d@grosbein.net> <20200116160745.GA1356@admin.sibptus.ru> <72355e03-1cf8-c58f-3aec-b0a21e631870@grosbein.net> <20200117093645.GA51899@admin.sibptus.ru> <70b0b855-189b-03c2-0712-fc1e35640702@grosbein.net> <20200117150928.GB66677@admin.sibptus.ru> <16550199-67b9-d331-0c1e-4afa0e8b361c@grosbein.net> <20200118105524.GA10042@admin.sibptus.ru> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="6TrnltStXW4iwmi0" Content-Disposition: inline In-Reply-To: X-PGP-Key: http://admin.sibptus.ru/~vas/ X-PGP-Fingerprint: 10E3 1171 1273 E007 C2E9 3532 0DA4 F259 9B5E C634 X-Rspamd-Queue-Id: 480GSt3nfZz3P8X X-Spamd-Bar: -------- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=sibptus.ru header.s=20181118 header.b=TFgqfljT; dmarc=pass (policy=none) header.from=sibptus.ru; spf=pass (mx1.freebsd.org: domain of vas@sibptus.ru designates 2001:19f0:5001:21dc::10 as permitted sender) smtp.mailfrom=vas@sibptus.ru X-Spamd-Result: default: False [-8.43 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[sibptus.ru:s=20181118]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; IP_SCORE(-3.33)[ip: (-9.89), ipnet: 2001:19f0:5000::/38(-4.94), asn: 20473(-1.75), country: US(-0.05)]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[sibptus.ru:+]; DMARC_POLICY_ALLOW(-0.50)[sibptus.ru,none]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5000::/38, country:US]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Jan 2020 11:50:07 -0000 --6TrnltStXW4iwmi0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Eugene Grosbein wrote: >=20 > >>>>> Back to the point. I've figured out that both encrypted (in transpo= rt > >>>>> mode) and unencrypted TCP segments have the same MSS=3D1460. Then I= 'm > >>>>> completely at a loss how the encrypted packets avoid being fragment= ed. > >>>>> TCP has no way to know in advance that encryption overhead will be > >>>>> added. > >=20 > > Here: http://admin.sibptus.ru/~vas/ftp-pcap.tar.gz you can find two > > identical FTP sessions, the only difference being ipsec=3Doff during one > > session and ipsec=3Don during the other one. > >=20 > > As I said, in both the sessions MSS=3D1460 which is already odd, and I > > can't explain to myself why file transfer still works without MSS > > ajustment. > >=20 > > Moreover, something fishy is happening in the encrypted session: there > > are many TCP retransmissions (I was capturing on the FTP server's side, > > so there are many segments with the same sequence number). How would you > > explain this? There are almost no retransmissions in the unencrypted se= ssion. > >=20 > > All this is happening in a lab environment (one bhyve VM is an FTP > > server and the other downloads a file from the first), both VMs are on > > the same bridge interface. There are almost 19,000 packets in the > > encrypted file vs 12,000 in the plain file, I think because of those > > excessive retransmissions. > >=20 > > Could the retransmissions be some artifact of the enc(4) interface I was > > capturing the encrypted session on? >=20 > I doubt it. And I can't explain this,=20 But do you agree that the traffic dumps contain an anomaly? > but maybe it's work of PMTUD Blackhole detection? > Look at sysctl net.inet.tcp | fgrep blackhole_ On both 192.168.246.10 and 192.168.246.11: root@fbsd-test1:~vas # sysctl net.inet.tcp | fgrep blackhole_ net.inet.tcp.v6pmtud_blackhole_mss: 1220 net.inet.tcp.pmtud_blackhole_mss: 1200 net.inet.tcp.pmtud_blackhole_detection: 0 root@fbsd-test2:~ # sysctl net.inet.tcp | fgrep blackhole_ net.inet.tcp.v6pmtud_blackhole_mss: 1220 net.inet.tcp.pmtud_blackhole_mss: 1200 net.inet.tcp.pmtud_blackhole_detection: 0 root@fbsd-test2:~ #=20 --=20 Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/ --6TrnltStXW4iwmi0 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEcBAEBAgAGBQJeIvDsAAoJEA2k8lmbXsY0Pc0H/RCJZvdb3hgIRoVZlBZwD15F 6Xco2+ji/pf9LstucIjPzZXBIjjBLbkn7aAb7lADXE0e7vXzFLFNzxSDiB6LTCfB xzrjvAbPMaeQX/Bg2SPl3GlzGkorQRDEkgp5RsbTX2YUIwBzovooiSduFdNPt/8g ET9bF2d03i0DoDS8Kp0OT+RTaFpZ/u+B5p59rhCUR0cM88PmkETX3OjXeWH/WSF8 RMDH80yT19NeEa8oCke1dMf2UmL3KPpKhxuxq+iclfs4KRliq0ngREvdn3hK5jf9 khKlVWJRFkuwVwXasmop+OTsBUkQkX99m3jpoyvpLSSqrUwixUss+dDgWLlFZ/k= =Tlj/ -----END PGP SIGNATURE----- --6TrnltStXW4iwmi0--