Date: Sat, 16 Aug 2008 15:15:35 -0700 From: Jeremy Chadwick <koitsu@FreeBSD.org> To: "Rick C. Petty" <rick-freebsd2008@kiwi-computer.com> Cc: freebsd-hackers@freebsd.org Subject: Re: local network throughput issues Message-ID: <20080816221535.GA82290@eos.sc1.parodius.com> In-Reply-To: <20080816194311.GA67723@keira.kiwi-computer.com> References: <20080816194311.GA67723@keira.kiwi-computer.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Aug 16, 2008 at 02:43:11PM -0500, Rick C. Petty wrote: > Hello. I've been having some serious network throughput issues recently. > Prior to these issues, I was running 7.0-STABLE from a few months back. I > recently build a world/kernel from csup(1) on 2008-Jul-25 and started > noticing the issues. I'm currently on a RELENG_7 from 2008-Aug-02 and am > building another one today. > > The server is running samba, dnsmasq, an NFS server, and is the gateway for > a couple of freebsd machines. I first noticed the issue when I was unable > to play mp3s served over samba. I was getting a throughput of about 3 > KBytes/sec over my gigabit switched network. After a few weeks of trying > different versions of samba and watching tcpdump/trafshow, I decided to try > another test: > > % ssh gateway cat /dev/zero | dd of=/dev/null > load: 0.00 cmd: dd 68286 [runnable] 0.00u 0.00s 0% 844k > 3680+0 records in > 3680+0 records out > 1884160 bytes transferred in 512.856684 secs (3674 bytes/sec) > ^C3680+0 records in > 3680+0 records out > 1884160 bytes transferred in 513.392858 secs (3670 bytes/sec) > Killed by signal 2. > > Again, not even 4 KB/s throughput over ssh. What's weird is that not all > network traffic is slow. I'm able to download at almost my full DSL speed > (7168 Kbps) and here's the strange bit: NFS is fast! From the same > machine as above, I have an NFS mountpoint: > > % dd if=/nfs/some_large_file of=/dev/null > 55980+0 records in > 55980+0 records out > 28661760 bytes transferred in 0.147159 secs (194767476 bytes/sec) > > Which is what I would expect for a gigabit network. Even if I perform the > same test over ssh: > > % ssh workstation cat /nfs/some_large_file | dd of=/dev/null > 55980+0 records in > 55980+0 records out > 28661760 bytes transferred in 2.791392 secs (10267909 bytes/sec) > > This is reasonable considering ssh overhead. The kernel config for the > server contains: > > include GENERIC > ident DDB > options KDB > options KDB_TRACE > options DDB > options DDB_NUMSYM > > My internal network is on 172.23.20.x and the DSL modem is connected to the > same NIC on 192.168.0.1: > > # ifconfig nfe0 > nfe0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=10b<RXCSUM,TXCSUM,VLAN_MTU,TSO4> > ether 00:15:f2:17:0c:20 > inet 172.23.20.1 netmask 0xffffff00 broadcast 172.23.20.255 > inet 192.168.0.3 netmask 0xffffff00 broadcast 192.168.0.255 > media: Ethernet autoselect (1000baseTX <full-duplex,flag0,flag1>) > status: active > > # ipfw show > 00100 62968 8170100 allow ip from any to any via lo0 > 00110 0 0 deny ip from any to 127.0.0.0/8 > 00120 0 0 deny ip from 127.0.0.0/8 to any > 00130 25444031 23419621364 divert 8668 ip from any to any via 192.168.0.3 > 00140 0 0 deny ip from not 172.23.20.0/24 to 172.23.20.0/24 dst-port 137-149,445 > 00150 20639141 15067620538 allow tcp from any to any established > 00160 21 10683 allow ip from any to any frag > 00170 79791 4696696 allow tcp from any to any setup > 65530 5713367 8864760207 allow ip from any to any > 65535 0 0 deny ip from any to any > > Other than that, I'm not doing anything out of the ordinary. Why is NFS > behaving correctly and why are ssh/smbd connections so slow? I've pasted > my dmesg output below. I've used this configuration for years and it > wasn't until a recent RELENG_7 upgrade that I've had any problems. The box > was 99-100% idle during those tests and I don't see an interrupt storm or > anything funny like that. Any ideas? 1) Please provide netstat -in output. 2) NFS (unless you're explicitly disabling it) is UDP-based, while SSH and Samba are TCP-based. Your nfe0 device has TSO4 enabled on it, so I'm left wondering if the TCP offloading support for your nfe(4) device is broken. Can you try disabling it by adding -tso to your ifconfig_nfe0 line in /etc/rc.conf? If you're using DHCP on that interface, that may pose somewhat of a problem. 3) Can you disable the firewall (disable ipfw entirely) and see if the problem continues? -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080816221535.GA82290>