From owner-freebsd-net@FreeBSD.ORG Thu Feb 1 23:53:31 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 49ECC16A401 for ; Thu, 1 Feb 2007 23:53:31 +0000 (UTC) (envelope-from david.baukus@us.fujitsu.com) Received: from fujitsu0.fujitsu.com (fujitsu0.fujitsu.com [192.240.0.5]) by mx1.freebsd.org (Postfix) with ESMTP id 2BE9213C494 for ; Thu, 1 Feb 2007 23:53:31 +0000 (UTC) (envelope-from david.baukus@us.fujitsu.com) Received: from fujitsu0.fujitsu.com (localhost [127.0.0.1]) by fujitsu0.fujitsu.com (8.13.7/8.13.7) with ESMTP id l11NOlF5005214 for ; Thu, 1 Feb 2007 15:24:47 -0800 (PST) Received: from fujitsu7i.fnanic.fujitsu.com ([133.164.253.7]) by fujitsu0.fujitsu.com (8.13.7/8.13.7) with ESMTP id l11NOkec005206 for ; Thu, 1 Feb 2007 15:24:46 -0800 (PST) Received: from fncnmp01.fnc.fujitsu.com (localhost [127.0.0.1]) by fujitsu7i.fnanic.fujitsu.com (8.13.7/8.13.7) with ESMTP id l11NOaoC026372 for ; Thu, 1 Feb 2007 15:24:45 -0800 (PST) Received: from unknown (HELO equinox.tx.fnc.fujitsu.com) ([167.254.252.122]) by fncnmp01.fnc.fujitsu.com with ESMTP; 01 Feb 2007 17:23:13 -0600 X-IronPort-AV: i="4.13,269,1167631200"; d="scan'208"; a="88965383:sNHT121823284" Received: from nomad.tx.fnc.fujitsu.com (localhost [127.0.0.1]) by equinox.tx.fnc.fujitsu.com (8.13.7+Sun/8.13.7) with ESMTP id l11NNDGU001231; Thu, 1 Feb 2007 17:23:13 -0600 (CST) Received: from tdd2172.tddeng00.fnts.com (tdd2172.tddeng00.fnts.com [167.254.250.128]) by nomad.tx.fnc.fujitsu.com (8.13.7+Sun/8.13.7) with ESMTP id l11NN8NT017423; Thu, 1 Feb 2007 17:23:08 -0600 (CST) Received: from [127.0.0.1] (localhost [127.0.0.1]) by tdd2172.tddeng00.fnts.com (8.11.7p3+Sun/8.11.7) with ESMTP id l11NN8G09935; Thu, 1 Feb 2007 17:23:08 -0600 (CST) Message-ID: <45C2765C.7010708@us.fujitsu.com> Date: Thu, 01 Feb 2007 17:23:08 -0600 From: Dave Baukus User-Agent: Mozilla Thunderbird 1.0 (X11/20041208) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "Baukus, David" Subject: ETIMEDOUT bug X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Feb 2007 23:53:31 -0000 There is a bug tcp_output() for at least freeBSD6.1 that causes a perfectly good TCP to be dropped by its retransmit timer; the application receives ETIMEDOUT. Consider a TCP that never transmits (the receive end of the ttcp utility is an example), while the TCP is established snd_max == snd_una == snd_nxt == (isr + 1) and the retransmit timer should never be started. If the retransmit timer is started then it is never stopped by tcp_input/tcp_out because snd_max == snd_una == snd_nxt (always). Once started the timer continues its count up till tp->t_rxtshift == 12 and the connection that never transmitted gets falsely killed. The bug is to blindly rely on the return value of ip_output(). If ip_output() returns ENOBUFS then the retransmit timer is activated: From the end of tcp_output(): out: SOCKBUF_UNLOCK_ASSERT(&so->so_snd); /* Check gotos. */ if (error == ENOBUFS) { if (!callout_active(tp->tt_rexmt) && !callout_active(tp->tt_persist)) callout_reset(tp->tt_rexmt, tp->t_rxtcur, tcp_timer_rexmt, tp); tp->snd_cwnd = tp->t_maxseg; return (0); } My simple minded fix would be not to start the retransmit timer; if tcp_output() wanted to time this transmit it would have started the timer up above. This ETIMEDOUT problem is easily recreated on any old machine using a single slow ethernet device and the ttcp test utility. First, fire up a couple ttcp receivers. Second, flood the same interface with enough ttcp transmitters to cause the driver's transmit ring and interface queue to back up. Eventually, one of the ttcp receives will get ENOBUFS from ip_output() and the retransmit timer will be wrongly activated for a pure ACK segment. I was able to do it w/ the following on freeBSD6.1: box1: ttcp -s -l 16384 -p 9444 -v -b 128000 -r ttcp -s -l 16384 -p 9445 -v -b 128000 -r ttcp -s -n 6553600 -l 4096 -p 9446 -v -b 128000 -t 192.168.222.13 ttcp -s -n 9999999 -l 333 -p 9447 -v -b 128000 -t 192.168.222.13 ttcp -s -n 9999999 -l 8192 -p 9448 -v -b 128000 -t 192.168.222.13 ttcp -s -n 9999999 -l 333 -p 9449 -v -b 128000 -t 192.168.222.13 ttcp -s -n 9999999 -l 8192 -p 9450 -v -b 128000 -t 192.168.222.13 box2: ttcp -s -n 6553600 -l 8192 -p 9444 -v -b 128000 -t 192.168.222.222 ttcp -s -n 9999999 -l 128 -p 9445 -v -b 128000 -t 192.168.222.222 ttcp -s -l 16384 -p 9446 -v -b 128000 -r ttcp -s -l 16384 -p 9447 -v -b 128000 -r ttcp -s -l 16384 -p 9448 -v -b 128000 -r ttcp -s -l 16384 -p 9449 -v -b 128000 -r ttcp -s -l 16384 -p 9450 -v -b 128000 -r -- Dave Baukus david.baukus@us.fujitsu.com Fujitsu Network Communications Richardson, Texas USA