From owner-freebsd-net@FreeBSD.ORG Thu Jul 1 23:55:29 2010 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C1B04106566B for ; Thu, 1 Jul 2010 23:55:29 +0000 (UTC) (envelope-from pyunyh@gmail.com) Received: from mail-pw0-f54.google.com (mail-pw0-f54.google.com [209.85.160.54]) by mx1.freebsd.org (Postfix) with ESMTP id 904308FC0C for ; Thu, 1 Jul 2010 23:55:29 +0000 (UTC) Received: by pwj9 with SMTP id 9so415367pwj.13 for ; Thu, 01 Jul 2010 16:55:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:received:from:date:to:cc :subject:message-id:reply-to:references:mime-version:content-type :content-disposition:in-reply-to:user-agent; bh=MyuTj023ESn5FvjjTd+lb0jgYUUFwC3jmw3za+E6fOk=; b=gP2uo4i9xTGuXXJ8WRwMySDgI1tkMly0A39XEWcUB0mng+xBSjQXvnCOMkKujEi5tr hOtgh1rHKrbpmtiC/rb3WUJhnvAYdOWThMdAQRudhXwmqlOddFCJLcgwPG9abN+hvTJZ cvlgq0y+FBXui6IWh+0rqqhcBushqio0ybAYw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:date:to:cc:subject:message-id:reply-to:references:mime-version :content-type:content-disposition:in-reply-to:user-agent; b=D2zGyoCfjbj0A/2dPNtD6Zw6j61PRNl9TlzKyX3unnZRzWOGQ+5j7Lx4bQjb8sEbvx SA4oVIyg7r17zgm+h1//MHpyI6fT6Ykvwgq0UBPDx/s/StizoBcY1x8SSFBRo8T0RrPQ 2uho8diLvM3sCmZPgFG+iDswSm29Hnr9VCHwc= Received: by 10.115.59.1 with SMTP id m1mr300534wak.99.1278028513929; Thu, 01 Jul 2010 16:55:13 -0700 (PDT) Received: from pyunyh@gmail.com ([174.35.1.224]) by mx.google.com with ESMTPS id q6sm380028waj.22.2010.07.01.16.55.12 (version=TLSv1/SSLv3 cipher=RC4-MD5); Thu, 01 Jul 2010 16:55:13 -0700 (PDT) Received: by pyunyh@gmail.com (sSMTP sendmail emulation); Thu, 1 Jul 2010 16:54:47 -0700 From: Pyun YongHyeon Date: Thu, 1 Jul 2010 16:54:47 -0700 To: Garrett Cooper Message-ID: <20100701235447.GJ7090@michelle.cdnetworks.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.2.3i Cc: net@freebsd.org Subject: Re: Poor performance with natd/ipfw and TSO enabled on bce(4) card and 8.1-PRERELEASE X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: pyunyh@gmail.com List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Jul 2010 23:55:29 -0000 On Wed, Jun 30, 2010 at 07:00:53PM -0700, Garrett Cooper wrote: > Hi, > Just an observation I made while transferring a file: > > # time scp floppy.img somehost: > Password: > floppy.img 100% 1440KB 13.7KB/s 01:45 > > real 1m59.400s > user 0m0.031s > sys 0m0.028s > # sysctl net.inet.tcp.tso=0 > net.inet.tcp.tso: 1 -> 0 > # time scp floppy.img somehost: > floppy.img 100% 1440KB 1.4MB/s 00:00 > > real 0m0.712s > user 0m0.018s > sys 0m0.018s > > Going ISDN speeds transferring a 1.44MB file is sad when you have > a gigabit uplink :(... natd seems to be doing a LOT of spinning when > TSO is enabled (it's going up to 73% CPU on a dual-proc quad-core > machine). I would use pf(4) if I have to handle lots of NAT rules. > Here are some other details: > > # ipfw list > 00050 divert 8668 ip4 from any to any via bce1 > 00100 allow ip from any to any via lo0 > 00200 deny ip from any to 127.0.0.0/8 > 00300 deny ip from 127.0.0.0/8 to any > 00400 deny ip from any to ::1 > 00500 deny ip from ::1 to any > 00600 allow ipv6-icmp from :: to ff02::/16 > 00700 allow ipv6-icmp from fe80::/10 to fe80::/10 > 00800 allow ipv6-icmp from fe80::/10 to ff02::/16 > 00900 allow ipv6-icmp from any to any ip6 icmp6types 1 > 01000 allow ipv6-icmp from any to any ip6 icmp6types 2,135,136 > 65000 allow ip from any to any > 65535 deny ip from any to any > # ls /etc/natd* > ls: /etc/natd*: No such file or directory > # uname -a > FreeBSD tameshi.cisco.com 8.1-PRERELEASE FreeBSD 8.1-PRERELEASE #0 > r209169: Mon Jun 14 12:41:49 PDT 2010 > root@:/usr/obj/data/scratch/src/stable/8/sys/TAMESHI_STABLE amd64 > # pciconf -lv | grep -A 4 bce > bce1@pci0:7:0:0: class=0x020000 card=0x01b21028 chip=0x164c14e4 > rev=0x12 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'Broadcom NetXtreme II Gigabit Ethernet Adapter (BCM5708)' > class = network > subclass = ethernet > -- > bce0@pci0:3:0:0: class=0x020000 card=0x01b21028 chip=0x164c14e4 > rev=0x12 hdr=0x00 > vendor = 'Broadcom Corporation' > device = 'Broadcom NetXtreme II Gigabit Ethernet Adapter (BCM5708)' > class = network > subclass = ethernet > > Let me know what other info is required. Can you reproduce this issue on other TSO capable drivers? I'm not aware of any TSO issues on bce(4).