Date: Fri, 14 Sep 2012 02:10:56 +1000 (EST) From: Ian Smith <smithi@nimnet.asn.au> To: Soren Dreijer <dreijer+bsd@echobit.net> Cc: freebsd-ipfw@freebsd.org Subject: Re: Significant network latency when using ipfw and in-kernel NAT Message-ID: <20120914020023.K51539@sola.nimnet.asn.au> In-Reply-To: <CALoZf3iCf1_fHgAWUXa3fgudOe66sbk35P0CYhgsneBuhCORJg@mail.gmail.com> References: <CALoZf3hfZDQQ4ZEXMrGUkYiGvb5QPoAcbpUikAq1adqVY4fLyg@mail.gmail.com> <20120913221758.E51539@sola.nimnet.asn.au> <CALoZf3iCf1_fHgAWUXa3fgudOe66sbk35P0CYhgsneBuhCORJg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 13 Sep 2012 0:48:01 -0500, Soren Dreijer wrote:
> Definitely. Since this is a server in production, I've obfuscated some
> of the IPs, etc.
>
> First off, here's the ifconfig. Our setup consists of a private (ix0)
> and a public nic (ix1) and an ip tunnel (gif0), which is what we use
> in ipfw to forward incoming packets to our internal boxes:
>
> ix0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=401bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,VLAN_HWTSO>
> ether XX:XX:XX:XX:XX:XX
> inet <private VLAN IP> netmask 0xffffffc0 broadcast xx
> inet6 xxxx::xxx:xxxx:xxxx:xxxx%ix0 prefixlen 64 scopeid 0x7
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
> status: active
> ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
> options=400bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO>
> ether XX:XX:XX:XX:XX:XX
> inet <public IP> netmask 0xfffffff8 broadcast xx
> inet6 xxxx::xxx:xxxx:xxxx:xxxx%ix1 prefixlen 64 scopeid 0x8
> inet <alias public IP> netmask 0xffffffff broadcast xx
> inet <alias public IP> netmask 0xffffffff broadcast xx
> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
> media: Ethernet autoselect (10Gbase-Twinax <full-duplex>)
> status: active
[ Soren and I had some off-list discussion which doesn't seem to have
helped matters, but I'll repost this as the only clue I had. Anybody? ]
Before anything else ..
% man ipfw | tail | head -4
Due to the architecture of libalias(3), ipfw nat is not compatible with
the TCP segmentation offloading (TSO). Thus, to reliably nat your net-
work traffic, please disable TSO on your NICs using ifconfig(8).
I don't know if this applies to VLAN_HWTSO, but likely to TSO4 on ix0?
Do things change if you try disabling all TSO?
I'd also change all 'out via ix1' to 'out xmit ix1', given the former
also applies to traffic going out anywhere that came _in_ on ix1.
cheers, Ian
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20120914020023.K51539>