From owner-freebsd-pf@freebsd.org Fri Dec 22 20:30:45 2017 Return-Path: Delivered-To: freebsd-pf@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 03872E87D9C; Fri, 22 Dec 2017 20:30:45 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from mx2.enfer-du-nord.net (mx2.enfer-du-nord.net [87.98.149.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id BCEFF65759; Fri, 22 Dec 2017 20:30:41 +0000 (UTC) (envelope-from trashcan@ellael.org) Received: from [IPv6:2003:8c:2e03:dc01:c8f8:8a2b:f09d:2a5a] (p2003008C2E03DC01C8F88A2BF09D2A5A.dip0.t-ipconnect.de [IPv6:2003:8c:2e03:dc01:c8f8:8a2b:f09d:2a5a]) by mx2.enfer-du-nord.net (Postfix) with ESMTPSA id 3z3Krs1746z3Dv; Fri, 22 Dec 2017 21:30:37 +0100 (CET) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 11.2 \(3445.5.20\)) Subject: Re: performance issue within VNET jail From: Michael Grimm In-Reply-To: <5A3D67EC.6010907@grosbein.net> Date: Fri, 22 Dec 2017 21:30:35 +0100 Cc: Eugene Grosbein Content-Transfer-Encoding: quoted-printable Message-Id: <53687746-C487-4712-AA52-DE86CE70FDEF@ellael.org> References: <4F5EE3F6-0163-4435-8726-56B0D4AE9FAF@ellael.org> <8102F5FD-DCFC-4EF8-A443-9E6C9EB1F467@ellael.org> <8C8A172B-4D4F-4066-8B94-EF5F59E2D345@ellael.org> <5A3D67EC.6010907@grosbein.net> To: freebsd-net@freebsd.org, freebsd-pf@FreeBSD.org X-Virus-Scanned: clamav-milter 0.99.2 at mail X-Virus-Status: Clean X-Mailer: Apple Mail (2.3445.5.20) X-BeenThere: freebsd-pf@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "Technical discussion and general questions about packet filter \(pf\)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Dec 2017 20:30:45 -0000 Hi =E2=80=94 [ I am including freebsd-pf@FreeBSD.org now and removing = freebsd-jail@FreeBSD.org ] [ Thread starts at = https://lists.freebsd.org/pipermail/freebsd-net/2017-December/049470.html = ] Eugene Grosbein wrote: > Michael Grimm wrote: >> Kristof Provost wrote: >>> I run a very similar setup (although on CURRENT), and see no = performance issues from my jails. >>=20 >> In utter despair I did upgrade one server to CURRENT (#327076) today, = but that hasn't been successful :-( >>=20 >> Ok, right now I do know: >>=20 >> (#) there is *no* performance loss (TCP) when: >>=20 >> (-) fetching files from outside through PF/extIF to host >> (-) fetching files from partner server host via IPSEC tunnel = bound to extIF (ESP) to host >> (-) fetching files from partner server host via IPSEC tunnel = bound to extIF (ESP) to jail via bridge >> (-) fetching files from partner server jail via bridge and then = via IPSEC tunnel bound to extIF (ESP) to host >> (-) fetching files from partner server jail via bridge and then = via IPSEC tunnel bound to extIF (ESP) and then via bridge to jail >>=20 >> (#) there is a *dramatic* performance loss (TCP) when: >>=20 >> (-) fetching files from outside through PF/extIF via bridge to = jail >>=20 >> (#) I did try to tweak the following settings *without* success: >>=20 >> (-) sysctl net.inet.tcp.tso=3D0=20 >> (-) sysctl net.link.bridge.pfil_onlyip=3D0 >> (-) sysctl net.link.bridge.pfil_bridge=3D0 >> (-) sysctl net.link.bridge.pfil_member=3D0=20 >> (-) reducing mtu to 1400 (1490 before) on all interfaces extIF, = bridge, epairXs >> (-) deactivating "scrub in all" and "scrub out on $extIF all = random-id" in /etc/pf.conf >> (-) setting "set require-order yes" and "set require-order no" = in /etc/pf.conf [1] >>=20 >> [1] I do see more a lot of out-of-order packages within a jail = "netstat -s -p tcp" after those slow downloads, but not after downloads = via IPSEC tunnel from partner host. >>=20 >> That leads me to the conclusions: >>=20 >> (#) the bridge is not to blame >> (#) it's either the PF/NATing or something else, right? >>=20 >> Thanks for your suggestions so far, but I am lost here. Any ideas? >=20 > It seems to me some kind of bug in the PF. > I personally never tried it, I use ipfw and it works just fine. Before testing IPFW (which I have never used before) I'd like to ask the = experts in freebsd-pf@FreeBSD.org about possible tests/tweaks regarding = PF. Thanks to all involved so far and regards, Michael