From owner-freebsd-questions@freebsd.org Tue Oct 4 09:39:19 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E631AF3485 for ; Tue, 4 Oct 2016 09:39:19 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from smtp.fagskolen.gjovik.no (smtp.fagskolen.gjovik.no [IPv6:2001:700:1100:1:200:ff:fe00:b]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smtp.fagskolen.gjovik.no", Issuer "Fagskolen i Gj??vik" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id CF1CB1F7C for ; Tue, 4 Oct 2016 09:39:18 +0000 (UTC) (envelope-from trond@fagskolen.gjovik.no) Received: from mail.fig.ol.no (localhost [127.0.0.1]) by mail.fig.ol.no (8.15.2/8.15.2) with ESMTPS id u949d7m7038559 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Tue, 4 Oct 2016 11:39:08 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) Received: from localhost (trond@localhost) by mail.fig.ol.no (8.15.2/8.15.2/Submit) with ESMTP id u949d7cT038556 for ; Tue, 4 Oct 2016 11:39:07 +0200 (CEST) (envelope-from trond@fagskolen.gjovik.no) X-Authentication-Warning: mail.fig.ol.no: trond owned process doing -bs Date: Tue, 4 Oct 2016 11:39:07 +0200 (CEST) From: =?ISO-8859-1?Q?Trond_Endrest=F8l?= Sender: Trond.Endrestol@fagskolen.gjovik.no To: FreeBSD questions Subject: Best practice for virtualized pf based NAT router? Message-ID: User-Agent: Alpine 2.20 (BSF 67 2015-01-07) Organization: Fagskolen Innlandet OpenPGP: url=http://fig.ol.no/~trond/trond.key MIME-Version: 1.0 X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on mail.fig.ol.no Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.23 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2016 09:39:19 -0000 Hi, I'm in the process of configuring a virtualized pf based NAT router. The NAT router is supposed be a supplement to our pool of public IPv4 addresses. FreeBSD is stable/11, r306639. XenServer 7.0.0, with all known updates, is the virtualization environment. I'm using xn0 as the external interface, and xn1 as the internal interface. The xn0 interface has a /30 IPv4 address and a /64 IPv6 address. The xn1 interface has a /20 IPv4 address (and a /64 IPv6 address for symmetry). I followed ch. 29.3.3.1 of the Handbook. In theory all is well, but with iftop(8) (net-mgmt/iftop) I only see a throughput of merely 1 Mbit/s, yes, that's one megabit per second. Running fetch(1) and ftp(1) directly on the NAT router gives me far better speeds, anything from 480 Mbit/s to 720 Mbit/s. My /etc/pf.conf file looks like this: ### 8< ###################### snip ################################ >8 # From the example in the Handbook, ch. 29.3.3.1. # Macros: ext_if="xn0" int_if="xn1" localnet = $int_if:network # Rules: nat on $ext_if from $localnet to any -> ($ext_if) block all pass from { lo0, $localnet } to any keep state # My own stuff: # Should I restrict any non-NAT44 traffic or let it all pass? pass all # Allow IPv6 everywhere. # Maybe not reasonable for a NAT44 GW, but it's not acting as an IPv6 GW. #pass inet6 all # We should allow SLAAC on $int_if. # Maybe this rule is too generous. #pass on $int_if inet6 keep state # These rules allows the GW to talk to outsiders via $ext_if. # Maybe the rules are too generous. #pass inet from $ext_if to any keep state #pass inet6 from $ext_if to any keep state ### 8< ###################### snip ################################ >8 Does anyone have any advice on how to achieve better throughput? I'm not new to FreeBSD, but pf is an unknown territory. My last attempt at doing NAT was with IPFW and natd(8) running FreeBSD 4 or 5 on a physical computer, some 15 years ago. Any advice will be highly appreciated. -- +-------------------------------+------------------------------------+ | Vennlig hilsen, | Best regards, | | Trond Endrestøl, | Trond Endrestøl, | | IT-ansvarlig, | System administrator, | | Fagskolen Innlandet, | Gjøvik Technical College, Norway, | | tlf. mob. 952 62 567, | Cellular...: +47 952 62 567, | | sentralbord 61 14 54 00. | Switchboard: +47 61 14 54 00. | +-------------------------------+------------------------------------+ From owner-freebsd-questions@freebsd.org Tue Oct 4 10:19:58 2016 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F0C33AF4295 for ; Tue, 4 Oct 2016 10:19:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from venus.codepro.be (venus.codepro.be [IPv6:2a01:4f8:162:1127::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.codepro.be", Issuer "Gandi Standard SSL CA 2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BD89CCFB for ; Tue, 4 Oct 2016 10:19:58 +0000 (UTC) (envelope-from kp@FreeBSD.org) Received: from [172.16.5.2] (vega.codepro.be [IPv6:2a01:4f8:162:1127::3]) (Authenticated sender: kp) by venus.codepro.be (Postfix) with ESMTPSA id AAE6F158C5; Tue, 4 Oct 2016 12:19:55 +0200 (CEST) From: "Kristof Provost" To: "Trond =?utf-8?q?Endrest=C3=B8l?=" Cc: "FreeBSD questions" Subject: Re: Best practice for virtualized pf based NAT router? Date: Tue, 04 Oct 2016 12:19:55 +0200 Message-ID: <2962E958-6570-4991-AC20-2A5FF39CC39C@FreeBSD.org> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit X-Mailer: MailMate (2.0BETAr6056) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 04 Oct 2016 10:19:59 -0000 On 4 Oct 2016, at 11:39, Trond Endrestøl wrote: > I'm in the process of configuring a virtualized pf based NAT router. > The NAT router is supposed be a supplement to our pool of public IPv4 > addresses. > > FreeBSD is stable/11, r306639. XenServer 7.0.0, with all known > updates, is the virtualization environment. > > I'm using xn0 as the external interface, and xn1 as the internal > interface. > > The xn0 interface has a /30 IPv4 address and a /64 IPv6 address. > The xn1 interface has a /20 IPv4 address (and a /64 IPv6 address for > symmetry). > > I followed ch. 29.3.3.1 of the Handbook. > > In theory all is well, but with iftop(8) (net-mgmt/iftop) I only see a > throughput of merely 1 Mbit/s, yes, that's one megabit per second. > There have been issues with pf and checksums in Xen before. I believe that the version you’re running has all of the relevant fixes, but it’s worth trying to disable TSO and other features on the network interfaces anyway. ifconfig xn0 -rxcsum -txcsum -rxcsum6 -txcsum6 -tso6 -tso4 -lro (and the same for xn1). If that makes a difference I’d be very interested in both network captures and further debugging. Regards, Kristof