From owner-freebsd-net@FreeBSD.ORG Fri Jul 2 14:32:29 2010 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9AA33106566B for ; Fri, 2 Jul 2010 14:32:29 +0000 (UTC) (envelope-from yanefbsd@gmail.com) Received: from mail-gy0-f182.google.com (mail-gy0-f182.google.com [209.85.160.182]) by mx1.freebsd.org (Postfix) with ESMTP id 52B128FC15 for ; Fri, 2 Jul 2010 14:32:29 +0000 (UTC) Received: by gyd8 with SMTP id 8so473162gyd.13 for ; Fri, 02 Jul 2010 07:32:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=+kLBdSjvUnAHuubAvFYboXCB3hNyOc4gA9McEvhMSZQ=; b=QwerhVa57KqYf/9cyxVYkuQLYkz4RFmO8OrHjFDR3aySP1YF3WmmosOKWbHt2+C3pN 5in5uU5dECBC2UZT5/gzAZnMcub5yWdRTx+TWmswatdGKPS/iQx1njA66EAfDNBuOGku psWm5L3nJlMUc+1lzkEDlaKLG0hIyW/osZWps= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=qersaNnHJpfl/lPKUwXgVQetUysPXD3eXp9wKCs7BhF7FyjqK2YsCiVMk8ezbGHSn6 ERGI2m4c4wUM85hR/at+tuS+FR1/U65mzgkvHdTEvuxFHPfLOcQE+MhFotmDB8JL0Rbo P89qqiWAbIxxC5NQgodTMeC3OuPpjT6RNKC9s= MIME-Version: 1.0 Received: by 10.229.230.65 with SMTP id jl1mr576796qcb.15.1278081145412; Fri, 02 Jul 2010 07:32:25 -0700 (PDT) Received: by 10.229.192.201 with HTTP; Fri, 2 Jul 2010 07:32:25 -0700 (PDT) In-Reply-To: References: <20100701235447.GJ7090@michelle.cdnetworks.com> <20100702022058.GA32531@babolo.ru> Date: Fri, 2 Jul 2010 07:32:25 -0700 Message-ID: From: Garrett Cooper To: Aleksandr A Babaylov <.@babolo.ru> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: Pyun YongHyeon , net@freebsd.org Subject: Re: Poor performance with natd/ipfw and TSO enabled on bce(4) card and 8.1-PRERELEASE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 02 Jul 2010 14:32:29 -0000 On Thu, Jul 1, 2010 at 10:32 PM, Garrett Cooper wrote: > On Thu, Jul 1, 2010 at 7:20 PM, Aleksandr A Babaylov <.@babolo.ru> wrote: >> On Thu, Jul 01, 2010 at 04:54:47PM -0700, Pyun YongHyeon wrote: >>> On Wed, Jun 30, 2010 at 07:00:53PM -0700, Garrett Cooper wrote: >>> > Hi, >>> > =A0 =A0 Just an observation I made while transferring a file: >>> > >>> > # time scp floppy.img somehost: >>> > Password: >>> > floppy.img =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0100% 1440KB =A013.7KB/s =A0 01:45 >>> > >>> > real =A0 =A0 =A0 =A01m59.400s >>> > user =A0 =A0 =A0 =A00m0.031s >>> > sys 0m0.028s >>> > # sysctl net.inet.tcp.tso=3D0 >>> > net.inet.tcp.tso: 1 -> 0 >>> > # time scp floppy.img somehost: >>> > floppy.img =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0 =A0100% 1440KB =A0 1.4MB/s =A0 00:00 >>> > >>> > real =A0 =A0 =A0 =A00m0.712s >>> > user =A0 =A0 =A0 =A00m0.018s >>> > sys 0m0.018s >>> > >>> > =A0 =A0 Going ISDN speeds transferring a 1.44MB file is sad when you = have >>> > a gigabit uplink :(... natd seems to be doing a LOT of spinning when >>> > TSO is enabled (it's going up to 73% CPU on a dual-proc quad-core >>> > machine). >>> I would use pf(4) if I have to handle lots of NAT rules. >> Or ipfw nat. >> man ipfw | grep nat > > =A0 =A0That uses the kernel module though, and that's horribly broken on > my machine with 8-STABLE/9-CURRENT (see: > http://www.mail-archive.com/freebsd-net@freebsd.org/msg33518.html ). I > wonder if that's related to the TSO issue. As Vitezslav pointed out to me off-list, this is a known issue with libalias and IPFW: Due to the architecture of libalias(3), ipfw nat is not compatible wit= h the TCP segmentation offloading (TSO). Thus, to reliably nat your net= - work traffic, please disable TSO on your NICs using ifconfig(8). Both the ipfw kernel based nat'ing and natd use libalias, so both will be affected by this issue. This needs to be fixed (I'll see what I can do to help expedite this). Thanks, -Garrett