From owner-freebsd-net@freebsd.org Thu Dec 21 22:21:06 2017 Return-Path: Delivered-To: freebsd-net@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 76B50E8F1F1; Thu, 21 Dec 2017 22:21:06 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from hz.grosbein.net (hz.grosbein.net [78.47.246.247]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "hz.grosbein.net", Issuer "hz.grosbein.net" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 0CB317B59D; Thu, 21 Dec 2017 22:21:05 +0000 (UTC) (envelope-from eugen@grosbein.net) Received: from eg.sd.rdtc.ru (root@eg.sd.rdtc.ru [62.231.161.221] (may be forged)) by hz.grosbein.net (8.15.2/8.15.2) with ESMTPS id vBLMKr5Z015312 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Dec 2017 23:20:54 +0100 (CET) (envelope-from eugen@grosbein.net) X-Envelope-From: eugen@grosbein.net X-Envelope-To: trashcan@ellael.org Received: from [10.58.0.4] ([10.58.0.4]) by eg.sd.rdtc.ru (8.15.2/8.15.2) with ESMTPS id vBLMKooX012573 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Fri, 22 Dec 2017 05:20:50 +0700 (+07) (envelope-from eugen@grosbein.net) Subject: Re: performance issue within VNET jail To: Michael Grimm References: <4F5EE3F6-0163-4435-8726-56B0D4AE9FAF@ellael.org> <8102F5FD-DCFC-4EF8-A443-9E6C9EB1F467@ellael.org> <5A3C2C42.6060904@grosbein.net> <5DAD8B80-FE3C-49D2-A645-EE144474D5FE@ellael.org> Cc: Kristof Provost , freebsd-net@freebsd.org, freebsd-jail@FreeBSD.org From: Eugene Grosbein Message-ID: <5A3C33BF.9050902@grosbein.net> Date: Fri, 22 Dec 2017 05:20:47 +0700 User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <5DAD8B80-FE3C-49D2-A645-EE144474D5FE@ellael.org> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=2.2 required=5.0 tests=BAYES_00, LOCAL_FROM, RDNS_NONE autolearn=no autolearn_force=no version=3.4.1 X-Spam-Report: * -2.3 BAYES_00 BODY: Bayes spam probability is 0 to 1% * [score: 0.0000] * 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS * 2.6 LOCAL_FROM From my domains X-Spam-Level: ** X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on hz.grosbein.net X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Dec 2017 22:21:06 -0000 22.12.2017 4:59, Michael Grimm wrote: >> Make sure and double check that your ESP packets do not get fragmented. > > > Hmm, I do not know how to achieve that. May the following tcpdump excerpts answer your question, or do you want me to look somewhere else? > > At hostA while downloading from hostB/jailX and "tcpdump -i extIF esp -vv" > > 22:52:42.341023 IP (tos 0x0, ttl 64, id 40481, offset 0, flags [none], proto ESP (50), length 140) > hostA > hostB: ESP(spi=0x01d9ec34,seq=0x5fe699), length 120 > 22:52:42.341079 IP (tos 0x0, ttl 53, id 64310, offset 1480, flags [none], proto ESP (50), length 100) > hostB > hostA: ip-proto-50 It shows non-zero offsets, so your ESP packets *are* fragmented. I guess, this is the reason of your problems as fragmented ESP packets are known to cause problems due to different reasons. Simpliest way to avoid such issues is to decrease MTU of IPSEC tunnel and/or TCP MSS so that incapsulated ESP packets do not get fragmented.