From owner-freebsd-net@FreeBSD.ORG Tue Dec 6 19:35:08 2005 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56DD116A41F for ; Tue, 6 Dec 2005 19:35:08 +0000 (GMT) (envelope-from is@rambler-co.ru) Received: from yam.park.rambler.ru (yam.park.rambler.ru [81.19.64.116]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2702443D8F for ; Tue, 6 Dec 2005 19:34:50 +0000 (GMT) (envelope-from is@rambler-co.ru) Received: from is.park.rambler.ru (is.park.rambler.ru [81.19.64.102]) by yam.park.rambler.ru (8.13.3/8.13.3) with ESMTP id jB6JYhxw090868; Tue, 6 Dec 2005 22:34:43 +0300 (MSK) (envelope-from is@rambler-co.ru) Date: Tue, 6 Dec 2005 22:34:43 +0300 (MSK) From: Igor Sysoev X-X-Sender: is@is.park.rambler.ru To: John-Mark Gurney In-Reply-To: <20051206183648.GG55657@funkthat.com> Message-ID: <20051206222847.Y73245@is.park.rambler.ru> References: <20050901140051.G11484@is.park.rambler.ru> <20050901182115.F11484@is.park.rambler.ru> <20051206183648.GG55657@funkthat.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: freebsd-net@freebsd.org Subject: Re: strange timeout error returned by kevent() in 6.0 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Dec 2005 19:35:08 -0000 On Tue, 6 Dec 2005, John-Mark Gurney wrote: > Igor Sysoev wrote this message on Thu, Sep 01, 2005 at 18:26 +0400: >> On Thu, 1 Sep 2005, Igor Sysoev wrote: >> >>> I found strange timeout errors returned by kevent() in 6.0 using >>> my http server named nginx. The nginx's run on three machines: >>> two 4.10-RELEASE and one 6.0-BETA3. All machines serve the same >>> content (simple cluster) and each handles about 200 requests/second. >>> >>> On 6.0 sometimes (2 or 3 times per hour) in the daytime kevent() >>> returns EV_EOF in flags and ETIMEDOUT in fflags, nevertheless: >>> >>> 1) nginx does not set any kernel timeout for sockets; >>> 2) the total request time for such failed requests is small, 30 and so >>> seconds. >> >> I have changed code to ignore the ETIMEDOUT error returned by kevent() >> and found that subsequent sendfile() returned the ENOTCONN. >> >> By the way, why sendfile() may return ENOTCONN ? >> I saw this error code on 4.x too. > > The reason that you are seeing ETIMEDOUT/ENOTCONN is that the connection > probably ETIMEDOUT (aka timed out)... and so is ENOTCONN (no longer > connected).. can you also do a read or a write to the socket successfully? At least recv() returns ETIMEDOUT. I could not test write() right now. > and sendfile(3) says: > ERRORS > [...] > > [ENOTCONN] The s argument points to an unconnected socket. > > and a glance at tcp(4) says: > ERRORS > [...] > > [ETIMEDOUT] when a connection was dropped due to excessive > retransmissions; > > There's the answers... Yes, it seems that ETIMEDOUT is retransmission failure. I've seen it in experiment. The strangeness is that I did not see this error on 4.10. Only on 6.0 and recenty on 4.11. May be I will upgrade cluster machine from 4.10 to 4.11 to see changes. Igor Sysoev http://sysoev.ru/en/