From owner-freebsd-net@FreeBSD.ORG Sat Jan 24 22:45:55 2009 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 07E27106567C for ; Sat, 24 Jan 2009 22:45:55 +0000 (UTC) (envelope-from mg@fork.pl) Received: from mail.obligo.pl (srv1.obligo.pl [193.28.230.34]) by mx1.freebsd.org (Postfix) with ESMTP id B36868FC1A for ; Sat, 24 Jan 2009 22:45:53 +0000 (UTC) (envelope-from mg@fork.pl) Received: from localhost (localhost [127.0.0.1]) by mail.obligo.pl (Postfix) with ESMTP id 67BDF1144C for ; Sat, 24 Jan 2009 23:26:24 +0100 (CET) X-Virus-Scanned: amavisd-new at obligo.pl Received: from mail.obligo.pl ([127.0.0.1]) by localhost (smb.obligo.pl [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id tvnZmZTuOj7a; Sat, 24 Jan 2009 23:26:22 +0100 (CET) Received: from hq.fork.pl (hq.fork.pl [217.113.238.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.obligo.pl (Postfix) with ESMTP id D97E61144B for ; Sat, 24 Jan 2009 23:26:22 +0100 (CET) From: Marcin Gryszkalis To: freebsd-net@freebsd.org Date: Sat, 24 Jan 2009 23:26:20 +0100 User-Agent: KMail/1.9.10 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-2" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200901242326.20595.mg@fork.pl> Subject: FreeBSD 7.1 - syncache, tcp queue X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 24 Jan 2009 22:45:55 -0000 I upgraded 2 machines to 7.1 (i386 and amd64) - both are running nginx web server and both are experiencing problem with resetting connections. c -> s TCP 56473 > 80 [SYN] Seq=0 Win=5840 Len=0 MSS=1460 TSV=173160194 TSER=0 WS=6 s -> c TCP 80 > 56473 [SYN, ACK] Seq=0 Ack=1 Win=65535 [TCP CHECKSUM INCORRECT] Len=0 MSS=1460 WS=3 TSV=2680138265 TSER=173160194 c -> s TCP 56473 > 80 [ACK] Seq=1 Ack=1 Win=5888 Len=0 TSV=173160239 TSER=2680138265 s -> c TCP 80 > 56473 [RST] Seq=1 Win=0 [TCP CHECKSUM INCORRECT] Len=0 s -> c HTTP GET / HTTP/1.0 ... on the i386 I can see in debug log: kernel: TCP: [83.6.208.139]:1909 to [62.233.226.70]:80; syncache_socket: Socket create failed due to limits or memory shortage kernel: TCP: [83.6.208.139]:1909 to [62.233.226.70]:80 tcpflags 0x10; tcp_input: Listen socket: Socket allocation failed due to limits or memory shortage, sending RST syncache.count is 0: net.inet.tcp.syncache.rst_on_sock_fail: 1 net.inet.tcp.syncache.rexmtlimit: 3 net.inet.tcp.syncache.hashsize: 512 net.inet.tcp.syncache.count: 0 net.inet.tcp.syncache.cachelimit: 15360 net.inet.tcp.syncache.bucketlimit: 30 and it seems that reason for the problem is that listen queue is overflown: Current listen queue sizes (qlen/incqlen/maxqlen) Proto Listen Local Address tcp4 193/0/128 10.2.3.1.80 I increased kern.ipc.somaxconn as temporary solution. netstat shows many (~193 - like the queue length) connections, mostly in CLOSED state (why aren't they purged?) I found some discussions about syncache changes in freebsd-net and other places but I'm not sure if my problem is known. Is there and reasoning and solution? regards -- Marcin Gryszkalis, PGP 0x9F183FA3 jabber jid:mg@fork.pl, gg:2532994 http://the.fork.pl