From owner-freebsd-stable@FreeBSD.ORG Tue Dec 30 20:06:12 2008 Return-Path: Delivered-To: stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 295EF1065672 for ; Tue, 30 Dec 2008 20:06:12 +0000 (UTC) (envelope-from ericlin@tamama.org) Received: from rn-out-0910.google.com (rn-out-0910.google.com [64.233.170.187]) by mx1.freebsd.org (Postfix) with ESMTP id E66828FC0C for ; Tue, 30 Dec 2008 20:06:11 +0000 (UTC) (envelope-from ericlin@tamama.org) Received: by rn-out-0910.google.com with SMTP id j71so3572749rne.12 for ; Tue, 30 Dec 2008 12:06:11 -0800 (PST) Received: by 10.150.149.19 with SMTP id w19mr28874815ybd.114.1230667571166; Tue, 30 Dec 2008 12:06:11 -0800 (PST) Received: by 10.151.74.12 with HTTP; Tue, 30 Dec 2008 12:06:11 -0800 (PST) Message-ID: <47713ee10812301206j12b35264o715976c154080a1b@mail.gmail.com> Date: Wed, 31 Dec 2008 04:06:11 +0800 From: "Lin Jui-Nan Eric" To: stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: Subject: TCP packet out-of-order problem X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Dec 2008 20:06:12 -0000 Dear listers, We recently found our new FreeBSD server (located in some foreign region) has poor network performance. After doing some tcpdump and iperf testing, we found that out-of-order TCP packets are not inserted into queue. This is an 100Mbps line, and TSO is disabled. % uname -a FreeBSD bsd 7.1-RC2 FreeBSD 7.1-RC2 #2: Wed Dec 31 03:12:39 CST 2008 root@bsd:/usr/obj/usr/src/sys/KERNEL amd64 % iperf -c 10.1.1.250 ------------------------------------------------------------ Client connecting to office, TCP port 5001 TCP window size: 3.07 MByte (default) ------------------------------------------------------------ [ 4] local 10.1.1.210 port 61488 connected with 10.1.1.250 port 5001 [ ID] Interval Transfer Bandwidth [ 4] 0.0-10.2 sec 5.74 MBytes 4.74 Mbits/sec 03:47:21.146397 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 159305:160753(1448) ack 1 win 1040 03:47:21.146409 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 160753 win 12568 03:47:21.146473 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 160753:162201(1448) ack 1 win 1040 03:47:21.146485 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 162201 win 12568 03:47:21.146972 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 163649:165097(1448) ack 1 win 1040 03:47:21.146983 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 162201 win 12573 03:47:21.146985 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 162201:163649(1448) ack 1 win 1040 03:47:21.146996 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 163649 win 12568 03:47:21.146998 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 165097:166545(1448) ack 1 win 1040 03:47:21.147006 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 163649 win 12573 03:47:21.147009 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 166545:167993(1448) ack 1 win 1040 03:47:21.147017 IP 10.1.1.250.5001 > 10.1.1.210.54919: . ack 163649 win 12573 03:47:21.147019 IP 10.1.1.210.54919 > 10.1.1.250.5001: . 167993:169441(1448) ack 1 win 1040 * You can see "ack 163649" repeating, but the packet is transmitted before 163649:165097. % cat /etc/sysctl.conf # $FreeBSD: src/etc/sysctl.conf,v 1.8 2003/03/13 18:43:50 mux Exp $ # # This file is read when going to multi-user and its contents piped thru # ``sysctl'' to adjust kernel values. ``man 5 sysctl.conf'' for details. # # Uncomment this to prevent users from seeing information about processes that # are being run under another UID. #security.bsd.see_other_uids=0 debug.bootverbose=1 kern.ipc.somaxconn=8192 kern.maxfiles=65536 kern.maxfilesperproc=32768 kern.maxprocperuid=65536 net.inet.ip.fastforwarding=1 net.inet.tcp.delayed_ack=0 vm.pmap.shpgperproc=2000 kern.ipc.maxsockbuf=8388608 net.inet.tcp.sendspace=3217968 net.inet.tcp.recvspace=3217968 Is our configuration wrong? Or it is an known bug? I have searched stable & net list, but found no similar discussion. Thank you all in advance!