From owner-freebsd-performance@FreeBSD.ORG Thu Jul 17 21:04:01 2008 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B38981065675 for ; Thu, 17 Jul 2008 21:04:01 +0000 (UTC) (envelope-from leccine@gmail.com) Received: from mail.nezz.be (nezz.be [195.228.74.74]) by mx1.freebsd.org (Postfix) with ESMTP id 2CE618FC12 for ; Thu, 17 Jul 2008 21:04:01 +0000 (UTC) (envelope-from leccine@gmail.com) Received: from mail.nezz.be (db.nezz.be [127.1.0.2]) by mail.nezz.be (Postfix) with ESMTP id ABD988B167B; Thu, 17 Jul 2008 22:41:20 +0200 (CEST) Received: by mail.nezz.be (Postfix, from userid 65534) id 91E728B1672; Thu, 17 Jul 2008 22:41:20 +0200 (CEST) X-Spam-Checker-Version: SpamAssassin 3.2.1 (2007-05-02) on db.nezz.be X-Spam-Level: X-Spam-Status: No, score=-0.4 required=5.0 tests=AWL,BAYES_20,RCVD_IN_PBL, RDNS_NONE autolearn=no version=3.2.1 Received: from [10.0.0.20] (unknown [78.16.27.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.nezz.be (Postfix) with ESMTP id A5A808B1659; Thu, 17 Jul 2008 22:41:15 +0200 (CEST) Message-ID: <487FAF68.6040200@gmail.com> Date: Thu, 17 Jul 2008 21:45:28 +0100 From: Istvan Szukacs User-Agent: Thunderbird 2.0.0.14 (Windows/20080421) MIME-Version: 1.0 To: freebsd-performance@freebsd.org References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: ClamAV using ClamSMTP Cc: Alexander Strange Subject: Re: Large number of http connections immediately dropped X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 17 Jul 2008 21:04:01 -0000 Hi! Something to read: http://people.freebsd.org/~hmp/utilities/satbl/sysctl-net.html I have these in the sysctl.conf kern.ipc.somaxconn=4096 net.inet.tcp.recvspace=78840 net.inet.tcp.sendspace=78840 kern.ipc.shmmax=67108864 kern.ipc.shmmni=200 kern.ipc.shmseg=128 kern.ipc.semmni=70 net.local.stream.sendspace=82320 net.local.stream.recvspace=82320 net.inet.tcp.local_slowstart_flightsize=10 net.inet.tcp.nolocaltimewait=1 net.inet.tcp.hostcache.expire=3900 and the loader.conf kern.maxusers=512 kern.ipc.nmbclusters=32768 kern.ipc.maxsockets=81920 kern.ipc.maxsockbuf=1048576 net.inet.tcp.tcbhashsize=4096 net.inet.tcp.hostcache.hashsize=1024 Regards, Istvan Alexander Strange wrote: > We're running a rather high-load webserver using FreeBSD > 7-RELEASE/amd64/nginx on an Intel em gigabit connection. > Performance is good for our current bandwidth use (about 20Mbit and > ~2000 connections/sec at the moment), but a large number of HTTP > requests are being immediately dropped before getting to nginx. I see > complaints about this with earlier versions of FreeBSD - > http://forum.lighttpd.net/topic/171 - but no solutions. Does anyone > know what could be the problem, or anything we could do about it? > > There are several other servers running earlier FreeBSDs on i386 which > don't seem to have this problem, but I still haven't ruled out > upstream hardware problems or Sandvine yet. > > On the server: > -nginx's error log is full of "accept() failed (53: Software caused > connection abort)", sometimes printing three or four at the same time. > > -messages is full of: > Limiting open port RST response from 441 to 200 packets/sec > Limiting open port RST response from 488 to 200 packets/sec > Limiting open port RST response from 399 to 200 packets/sec > Limiting open port RST response from 434 to 200 packets/sec > Limiting open port RST response from 308 to 200 packets/sec > I'm not sure if that's related or not. > > -sysctl.conf: > > net.inet.tcp.tso=1 > kern.ipc.somaxconn=10240 > kern.ipc.nmbclusters=65536 > net.inet.tcp.sendspace=65536 > net.inet.tcp.recvspace=65536 > net.inet.tcp.rfc1323=1 > kern.ipc.maxsockbuf=262144 > net.inet.tcp.blackhole=2 > net.inet.udp.blackhole=1 > net.inet.tcp.msl=7500 > net.inet.icmp.icmplim=400 > net.inet.tcp.drop_synfin=1 > net.inet.tcp.icmp_may_rst=0 > net.inet.tcp.fast_finwait2_recycle=1 > > -netstat -m: > 4677/6603/11280 mbufs in use (current/cache/total) > 1017/2643/3660/65536 mbuf clusters in use (current/cache/total/max) > 1017/1961 mbuf+clusters out of packet secondary zone in use > (current/cache) > 9/514/523/12800 4k (page size) jumbo clusters in use > (current/cache/total/max) > 0/0/0/6400 9k jumbo clusters in use (current/cache/total/max) > 0/0/0/3200 16k jumbo clusters in use (current/cache/total/max) > 3239K/8992K/12232K bytes allocated to network (current/cache/total) > 0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters) > 0/0/0 requests for jumbo clusters denied (4k/9k/16k) > 0/0/0 sfbufs in use (current/peak/max) > 0 requests for sfbufs denied > 0 requests for sfbufs delayed > 9204 requests for I/O initiated by sendfile > 0 calls to protocol drain routines > > nginx is not running any accept filters. > > Locally, after sending an HTTP request, I get a normal connection > close, then one RST with sequence 1, then another (possibly more than > one) RST with sequence 2. I can post a tcpdump sequence if necessary, > after I sanitize some cookies away. > _______________________________________________ > freebsd-performance@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to > "freebsd-performance-unsubscribe@freebsd.org"