From owner-freebsd-ipfw@freebsd.org Fri Sep 1 07:08:21 2017 Return-Path: Delivered-To: freebsd-ipfw@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 30574E0E59C for ; Fri, 1 Sep 2017 07:08:21 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from sola.nimnet.asn.au (paqi.nimnet.asn.au [115.70.110.159]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D02F16463F for ; Fri, 1 Sep 2017 07:08:19 +0000 (UTC) (envelope-from smithi@nimnet.asn.au) Received: from localhost (localhost [127.0.0.1]) by sola.nimnet.asn.au (8.14.2/8.14.2) with ESMTP id v81788h6062685; Fri, 1 Sep 2017 17:08:08 +1000 (EST) (envelope-from smithi@nimnet.asn.au) Date: Fri, 1 Sep 2017 17:08:08 +1000 (EST) From: Ian Smith To: "Andrey V. Elsukov" cc: Graham Menhennitt , freebsd-ipfw@freebsd.org Subject: Re: ipfw kernel NAT performance much worse in 11-Stable than 10-Stable In-Reply-To: <40f3bcab-5e0d-0905-ec95-8b4eec8cef89@yandex.ru> Message-ID: <20170901162808.C23641@sola.nimnet.asn.au> References: <87b38492-da4b-316f-37c2-e1043c2adee4@yandex.ru> <580bc972-7800-96ff-c190-0be176c22d77@menhennitt.com.au> <40f3bcab-5e0d-0905-ec95-8b4eec8cef89@yandex.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Content-ID: <20170901162808.V23641@sola.nimnet.asn.au> X-BeenThere: freebsd-ipfw@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: IPFW Technical Discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Sep 2017 07:08:21 -0000 On Thu, 31 Aug 2017 15:27:47 +0300, Andrey V. Elsukov wrote: > On 31.08.2017 15:10, Graham Menhennitt wrote: > > On 10-Stable, the interface is re1. The output of 'ifconfig re1 | grep > > options' is: > > options=8209b > > > > nd6 options=29 > > > > On 11-Stable (the one with the problems), it's igb1 and the output of > > 'ifconfig igb1 | grep options' is: > > options=6403bb > > > > nd6 options=29 > > > > You need to disable TSO on your interface, ipfw nat is not compatible > with TCP segmentation offloading (this is noted in ipfw(8) BUGS section). > > Try to use: > ifconfig igb1 -vlanhwtso -tso4 > > You can add these option to "ifconfig_igb1" variable in rc.conf. Specifically: Due to the architecture of libalias(3), ipfw nat is not compatible with the TCP segmentation offloading (TSO). Thus, to reliably nat your net- work traffic, please disable TSO on your NICs using ifconfig(8). Since natd also uses libalias, does not that also apply when using natd? I forget, and neither libalias(3) nor natd(8) mentions 'tso|TSO'. Since this comes up so often, including on questions@, I'm wondering if an extra test in /etc/rc.d/ipfw at ipfw_prestart() for enablement of either $natd_enable (if applicable) or $firewall_nat_enable could then and there check ifconfig $natd_interface and/or $firewall_nat_interface for the presence of TSO4 and/or VLAN_HWTSO options, and so could warn the user - or just run "ifconfig $iface -vlanhwtso -tso4" directly? While some interfaces such as ngX or pppX need not be up or even exist when starting ipfw, such interfaces should never use TSO anyway? But I'm probably missing something obvious .. cheers, Ian