From owner-freebsd-net@FreeBSD.ORG Thu May 8 04:06:55 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A1C7C106566B for ; Thu, 8 May 2008 04:06:55 +0000 (UTC) (envelope-from dengxf@gmail.com) Received: from wa-out-1112.google.com (wa-out-1112.google.com [209.85.146.178]) by mx1.freebsd.org (Postfix) with ESMTP id 72BF98FC14 for ; Thu, 8 May 2008 04:06:55 +0000 (UTC) (envelope-from dengxf@gmail.com) Received: by wa-out-1112.google.com with SMTP id j4so755441wah.3 for ; Wed, 07 May 2008 21:06:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:date:from:to:subject:cc:in-reply-to:references:message-id:mime-version:content-type:content-transfer-encoding:x-mailer; bh=1N+cdIPhchPj54XsjCdqbWbC6yC4tfjtNQP1hsa+WQA=; b=VRXCpr5v3TIFscNQeKNO6YRuOYZ52/pZYbU+853o7apCac63E4KguvFgJ64nLBRtZXeLsoxZdOoCMkhNF1YpDMzIEK5Nwnxi07e9A/na4q9bLlfQDOH6AGIUrlcgpvSFTe6Lmy/KwwW6rOSVEbDxCBURaedJ4eN4gt1VdKWWuA4= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=date:from:to:subject:cc:in-reply-to:references:message-id:mime-version:content-type:content-transfer-encoding:x-mailer; b=j5HXqJQJJwS0or58TmhgNZsSNMSAQrjYwI2Dq6R+fUDtMUzwWaq87Kci6/MwkWeZj90dqDUpNV4uZkvW9dKieI0cNjP8/0QlPm/KfW9JNmpDUlmqRWxdj5GQ6HrSLo/PWCj2+DpXpPcoMnx7fxI2EOy2wWsBQ088p/ITtcXreMo= Received: by 10.114.150.1 with SMTP id x1mr2645604wad.109.1210219614705; Wed, 07 May 2008 21:06:54 -0700 (PDT) Received: from ?192.168.0.160? ( [218.94.128.114]) by mx.google.com with ESMTPS id l27sm4577372waf.27.2008.05.07.21.06.50 (version=SSLv3 cipher=RC4-MD5); Wed, 07 May 2008 21:06:53 -0700 (PDT) Date: Thu, 08 May 2008 12:07:04 +0800 From: Deng XueFeng To: Andre Oppermann In-Reply-To: <48223324.6070203@freebsd.org> References: <20080506133208.C2EC.B627C632@gmail.com> <48223324.6070203@freebsd.org> Message-Id: <20080508120410.70E4.B627C632@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit X-Mailer: Becky! ver. 2.45.02 [CN] Cc: freebsd-net@freebsd.org, Mark Hills Subject: Re: read() returns ETIMEDOUT on steady TCP connection X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 08 May 2008 04:06:55 -0000 hi, the patch can not apply to 6.2, cound do a new patch for 6.2 or 6.3 ? > I've looked at the code paths again. There are two possibilities: > > a) the mbuf allocator has some anomaly where it rejects memory requests > but doesn't update the statistics (the code is there however). > > b) the error doesn't come from the mbuf allocation but from ip_output() > and further down the chain. > > To differentiate please try this updated patch and report the log output > again (don't forget to set net.inet.tcp.log_debug=1): > > http://people.freebsd.org/~andre/tcp_output-error-log.diff > > -- > Andre > > Deng XueFeng wrote: > > hi > > I'am also meet this problem in my mss server(missey streaming server). > > one encoder push stream to mss, then run 100 client player playing the > > sream, as the client number increase, mss will occur this error sooner or later > > like this: > > I'am using kqueue, and will got a event with EV_EOF and fflags = > > ETIMEDOUT, > > if i ignore EV_EOF flag, then ETIMEDOUT will be return by read(2), > > > > and the tcpdump also show that server will send RST packet to encoder. > > > > > >> Hello, > >> > >> I'm are having a trouble with TCP connections being dropped with "read: > >> Operation timed out". What is unusual is that this is happening right in > >> the middle of sending a steady stream of data with no network congestion. > >> > >> The system is FreeBSD 7 and a bespoke streaming server with 1Gbit > >> connection. The server receives a 192kbps inbound stream over TCP, and > >> broadcasts it over a large number of TCP streams. > >> > >> With no visible or obvious pattern, the inbound read() fails with > >> ETIMEDOUT. The likelihood of this happening seems to increase as the > >> number of audience connections increases. It's happens every few minutes > >> even with a small audience (eg. 300 outbound connections and about > >> 60mbit). > >> > >> It doesn't cough and splutter -- steady data is coming in, then it just > >> drops the connection. > >> > >> systat doesn't show problems inbound; all packets received are delivered > >> to the upper layer. But on outbound, there is consistent 'output drops': > >> > >> IP Output > >> 7028 total packets sent > >> 7028 - generated locally > >> 314 - output drops > >> > >> As the number of outbound connections increases, the 'output drops' > >> increases to around 10% of the total packets sent and maintains that > >> ratio. There's no problems with network capacity. > >> > >> I've tried different servers, different network interfaces (bge, em), > >> different kernel (7-RELEASE, 7-STABLE). Have also checked dev.bge.0.stats > >> and dev.em.0.stats for CRC errors etc. which show no problems. 'netstat > >> -m' doesn't show any reaching of mbuf and sbuf limits. The problem is seen > >> in a dedicated, uncontended test environment. > >> > >> Can anyone explain why the packets are being dropped outbound, and how > >> this could affect inbound TCP data in such an abrupt way? What can I do to > >> solve this? > >> > >> Thanks, > >> > >> Mark > >> _______________________________________________ > >> freebsd-net@freebsd.org mailing list > >> http://lists.freebsd.org/mailman/listinfo/freebsd-net > >> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" > > -- Deng XueFeng