From owner-freebsd-net@freebsd.org Sun Dec 10 19:54:53 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1BAD7E99467 for ; Sun, 10 Dec 2017 19:54:53 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx1.enfer-du-nord.net (mx1.enfer-du-nord.net [91.121.41.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id DD55B7FD4F for ; Sun, 10 Dec 2017 19:54:52 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e67:d601:d8dd:c75a:db8c:227a] (p2003008C2E67D601D8DDC75ADB8C227A.dip0.t-ipconnect.de [IPv6:2003:8c:2e67:d601:d8dd:c75a:db8c:227a]) by mx1.enfer-du-nord.net (Postfix) with ESMTPSA id 3yvxcz1Txcz7t9; Sun, 10 Dec 2017 20:54:43 +0100 (CET) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.1 \(3445.4.7\)) Subject: Re: [IPsec] Weird performance issue via IPsec/racoon tunnel From: Michael Grimm In-Reply-To: <5A2D703F.8040004@grosbein.net> Date: Sun, 10 Dec 2017 20:54:42 +0100 Cc: Eugene Grosbein Content-Transfer-Encoding: quoted-printable Message-Id: <3B480730-FF34-45B8-8636-9FCD4E97A2B9@ellael.org> References: <7A6EF712-920E-40BF-B155-113EE6C00AEA@ellael.org> <5A2D703F.8040004@grosbein.net> To: freebsd-net@FreeBSD.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3445.4.7) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 10 Dec 2017 19:54:53 -0000 Eugene Grosbein wrote: > 10.12.2017 23:55, Michael Grimm wrote: > "bad cksum 0" is pretty normal for traffic going out via interface = supporting hardware checksum offload, > so kernel skips computing checksum before passing packets to the NIC. Ok, good to know. > Your problem more likely is due to fragmented ESP packets. > It's not uncommon when cloud IP stack or ISP infrastructure drop high = percentage > of fragmented ESP packets because they are not optimized for such = packets, > e.g. router has to process them in software instead of hardware > like non-fragmented packets are processed. Thank you for this explanation.=20 I did already lower MTU: If I do configure vtnet0 to a MTU of 1490 at = boot time I do not not notice a performance loss compared to the default = 1500 setting. >> *BUT* if I do a "ifconfig vtnet0 mtu 1450 up ; ifconfig vtnet0 mtu = 1500 up" I do observe: >>=20 >> #) scp NEW to OLD via IPsec tunnel: 17.1 MB/s ! >> #) scp OLD to NEW via IPsec tunnel: 16.9 MB/s *BUT* if I do boot with the default 1500 setting, changing the MTU to = e.g. 1450 and *immediately* back to 1500 manually, I do not encounter = any performance loss at all. Why? Even when booting 1490 and immediately = setting the MTU manually to 1500 I do not see any performance loss. = Strange. > When you lower MTU of vtnet enough to make encapsulated packets = (payload+overhead) <=3D1500 bytes, > resulted ESP packets have not be fragmented and pass just fine. I will keep the MTU at 1490 and monitor that server for the time being. > To verify if it's your case, you should run two tcpdump commands, > one at sending side and another at receiving size=20 > and compare outputs to see if *every* outgoing packet reaches its = destination or not. Hmm, how would one check that? The output is to fast for me ;-) = Seriously, how should one check this? Thanks for your help, Michael