From owner-freebsd-net@FreeBSD.ORG  Thu Apr 24 00:04:01 2008
Return-Path: <owner-freebsd-net@FreeBSD.ORG>
Delivered-To: freebsd-net@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id F34C5106567A;
	Thu, 24 Apr 2008 00:04:00 +0000 (UTC)
	(envelope-from mark@pogo.org.uk)
Received: from metheny.ijneb.com (unknown
	[IPv6:2001:ba8:0:1ba:214:22ff:feb1:2693])
	by mx1.freebsd.org (Postfix) with ESMTP id 9980B8FC15;
	Thu, 24 Apr 2008 00:04:00 +0000 (UTC)
	(envelope-from mark@pogo.org.uk)
Received: from localhost ([127.0.0.1] ident=mark)
	by metheny.ijneb.com with esmtp (Exim 4.69)
	(envelope-from <mark@pogo.org.uk>)
	id 1JoowP-0007MP-GA; Thu, 24 Apr 2008 01:03:57 +0100
Date: Thu, 24 Apr 2008 01:03:57 +0100 (BST)
From: Mark Hills <mark@pogo.org.uk>
To: Andre Oppermann <andre@freebsd.org>
In-Reply-To: <480E7901.5000804@freebsd.org>
Message-ID: <alpine.BSO.1.10.0804240030500.6125@zrgural.vwaro.pbz>
References: <alpine.BSO.1.10.0804191437400.21362@zrgural.vwaro.pbz>
	<20080420025010.GJ73016@server.vk2pj.dyndns.org>
	<alpine.BSO.1.10.0804201238480.31900@zrgural.vwaro.pbz>
	<480BBD7E.8010700@freebsd.org>
	<alpine.BSO.1.10.0804210740100.1745@zrgural.vwaro.pbz>
	<480C9AC6.8090802@freebsd.org> <480E7901.5000804@freebsd.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
X-SA-Exim-Connect-IP: 127.0.0.1
X-SA-Exim-Mail-From: mark@pogo.org.uk
Cc: Peter Jeremy <peterjeremy@optushome.com.au>, freebsd-net@freebsd.org
Subject: Re: read() returns ETIMEDOUT on steady TCP connection
X-BeenThere: freebsd-net@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Networking and TCP/IP with FreeBSD <freebsd-net.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-net>
List-Post: <mailto:freebsd-net@freebsd.org>
List-Help: <mailto:freebsd-net-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-net>,
	<mailto:freebsd-net-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 24 Apr 2008 00:04:01 -0000

On Wed, 23 Apr 2008, Andre Oppermann wrote:

> http://people.freebsd.org/~andre/tcp_output-error-log.diff
>
> Please apply this patch and enable the sysctl net.inet.tcp.log_debug=1
> and report any output.  You likely get some (normal) noise from syncache.
> What we are looking for is reports from tcp_output.

Hi Andre, I've applied the patch and tested.

Aside from syncache noise, I get a constant stream of 'error 55' 
(ENOBUFS?), once the number of connection gets to around 150 at 192kbps.

TCP: [192.168.5.43]:52153 to [192.168.5.40]:8080; tcp_output: error 55 while sending

192.168.5.40 is the IP address of this host, running the server.

I tried to correlate the point of the application receiving ETIMEDOUT with 
these messages, but that is tricky as it seems to be outputting a lot of 
messages, and multiple messages over eachother (see below).

Because of the mention of no buffer space available, I checked the values 
of net.inet.tcp.sendbuf* and recvbuf*, and increased the max values with 
no effect.

When I get time I will modify the kernel to print errors which aren't 
ENOBUFS to see if there are any others. But in the meantime, this sounds 
like a problem to me. Is that correct?

Mark


:8080; tcp_output: error 55 while sending
TCP: [192.168.5.42]:57384T CtPo:  [[119922..116688..55..4402]]::85048400;1  ttoc p[_1o9u2t.p1u6t8:. 5e.r4r0o]r: 8080;5 5t cwp_hoiultep uste:n deirnrgor 55 while sending
TCP: [192.168.5.42]:57382 to [192.168.5.40]:8080; tcp_output: error 55 while sending
TCP: [192.168.5.42]:57381 to [192.168.5.40]:8080; tcp_output: error 55 while sending
TCP: [192.168.5.42]:57380 to [192.168.5.40]:8080; tcp_output: error 55 while sending