Date: Thu, 1 Jul 2010 18:19:13 -0700 From: Garrett Cooper <yanefbsd@gmail.com> To: pyunyh@gmail.com Cc: net@freebsd.org Subject: Re: Poor performance with natd/ipfw and TSO enabled on bce(4) card and 8.1-PRERELEASE Message-ID: <AANLkTikmhWNPRKHJrbMNASnu1Ggtuda6w1UZfA2JqN4d@mail.gmail.com> In-Reply-To: <20100701235447.GJ7090@michelle.cdnetworks.com> References: <AANLkTinuucNtKqWscZ9IDjws-NiNcHpKrMElLgd7Sb8r@mail.gmail.com> <20100701235447.GJ7090@michelle.cdnetworks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Jul 1, 2010 at 4:54 PM, Pyun YongHyeon <pyunyh@gmail.com> wrote: > On Wed, Jun 30, 2010 at 07:00:53PM -0700, Garrett Cooper wrote: >> Hi, >> =A0 =A0 Just an observation I made while transferring a file: >> >> # time scp floppy.img somehost: >> Password: >> floppy.img =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0100% 1440KB =A013.7KB/s =A0 01:45 >> >> real =A01m59.400s >> user =A00m0.031s >> sys =A0 0m0.028s >> # sysctl net.inet.tcp.tso=3D0 >> net.inet.tcp.tso: 1 -> 0 >> # time scp floppy.img somehost: >> floppy.img =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0100% 1440KB =A0 1.4MB/s =A0 00:00 >> >> real =A00m0.712s >> user =A00m0.018s >> sys =A0 0m0.018s >> >> =A0 =A0 Going ISDN speeds transferring a 1.44MB file is sad when you hav= e >> a gigabit uplink :(... natd seems to be doing a LOT of spinning when >> TSO is enabled (it's going up to 73% CPU on a dual-proc quad-core >> machine). > > I would use pf(4) if I have to handle lots of NAT rules. > >> =A0 =A0 Here are some other details: >> >> # ipfw list >> 00050 divert 8668 ip4 from any to any via bce1 >> 00100 allow ip from any to any via lo0 >> 00200 deny ip from any to 127.0.0.0/8 >> 00300 deny ip from 127.0.0.0/8 to any >> 00400 deny ip from any to ::1 >> 00500 deny ip from ::1 to any >> 00600 allow ipv6-icmp from :: to ff02::/16 >> 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 >> 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 >> 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 >> 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 >> 65000 allow ip from any to any >> 65535 deny ip from any to any >> # ls /etc/natd* >> ls: /etc/natd*: No such file or directory >> # uname -a >> FreeBSD tameshi.cisco.com 8.1-PRERELEASE FreeBSD 8.1-PRERELEASE #0 >> r209169: Mon Jun 14 12:41:49 PDT 2010 >> root@:/usr/obj/data/scratch/src/stable/8/sys/TAMESHI_STABLE =A0amd64 >> # pciconf -lv | grep -A 4 bce >> bce1@pci0:7:0:0: =A0 =A0 =A0class=3D0x020000 card=3D0x01b21028 chip=3D0x= 164c14e4 >> rev=3D0x12 hdr=3D0x00 >> =A0 =A0 vendor =A0 =A0 =3D 'Broadcom Corporation' >> =A0 =A0 device =A0 =A0 =3D 'Broadcom NetXtreme II Gigabit Ethernet Adapt= er (BCM5708)' >> =A0 =A0 class =A0 =A0 =A0=3D network >> =A0 =A0 subclass =A0 =3D ethernet >> -- >> bce0@pci0:3:0:0: =A0 =A0 =A0class=3D0x020000 card=3D0x01b21028 chip=3D0x= 164c14e4 >> rev=3D0x12 hdr=3D0x00 >> =A0 =A0 vendor =A0 =A0 =3D 'Broadcom Corporation' >> =A0 =A0 device =A0 =A0 =3D 'Broadcom NetXtreme II Gigabit Ethernet Adapt= er (BCM5708)' >> =A0 =A0 class =A0 =A0 =A0=3D network >> =A0 =A0 subclass =A0 =3D ethernet >> >> =A0 =A0 Let me know what other info is required. > > Can you reproduce this issue on other TSO capable drivers? > I'm not aware of any TSO issues on bce(4). Hi Pyun! I'll have to pop in a Copper Intel card that we have laying around in the lab. I think it's em(4) compatible.. I forget... I have a few things to test network wise this weekend, so I'll try and repro a few things this weekend (say, Sunday?). I also have my msk(4) enabled machine in the lab I can test with, but I'll have to install the machine to spec with the Poweredge 2950 I have in the lab. I'm using ipfw because it was easy to setup according to the handbook, but in reality if ipfw is this bad dealing with nat rules, then I need to work with someone to improve how it scales. Thanks! -Garrett
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikmhWNPRKHJrbMNASnu1Ggtuda6w1UZfA2JqN4d>