From owner-freebsd-net@FreeBSD.ORG Sun Jun 17 18:22:25 2007 Return-Path: X-Original-To: net@freebsd.org Delivered-To: freebsd-net@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0D93A16A41F for ; Sun, 17 Jun 2007 18:22:25 +0000 (UTC) (envelope-from joe@joeholden.co.uk) Received: from scarlett.lon.rewt.org.uk (scarlett.lon.rewt.org.uk [62.84.188.34]) by mx1.freebsd.org (Postfix) with ESMTP id CD30113C44C for ; Sun, 17 Jun 2007 18:22:24 +0000 (UTC) (envelope-from joe@joeholden.co.uk) Received: from [172.16.10.59] (87-194-76-190.bethere.co.uk [87.194.76.190]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by scarlett.lon.rewt.org.uk (Postfix) with ESMTP id 2ED2639848; Sun, 17 Jun 2007 19:07:22 +0100 (BST) Message-ID: <46757818.5030005@joeholden.co.uk> Date: Sun, 17 Jun 2007 19:06:16 +0100 From: Joe Holden User-Agent: Thunderbird 2.0.0.4 (Windows/20070604) MIME-Version: 1.0 To: "M. Warner Losh" References: <20070617.114133.778151882.imp@bsdimp.com> In-Reply-To: <20070617.114133.778151882.imp@bsdimp.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: net@freebsd.org Subject: Re: Issue with huge numbers of connections X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 17 Jun 2007 18:22:25 -0000 M. Warner Losh wrote: > Greetings, > > I have a friend who is having problems with a service he's running. > He gets billions and billions of connections to this service a day. > Somewhere between 10^8 and 10^9 connections, he notices that his > servers lose the ability to accept new connections. These are TCP > connections. > > This is with FreeBSD 6.1R. My first question is: does anybody know if > the fixes to -current/7.0 have fixed this? Is there a fix that can be > back ported? He's currently working around the problem by having a > number of different machines that reboot in a round robin fashion, but > would like a better solution. > > Warner > _______________________________________________ Warner, if he hasn't done so already, have you suggested tweaking the sysctl variables, such as: kern.maxfilesperproc kern.ipc.nmbclusters kern.maxprocperuid kern.maxfiles kern.ipc.somaxconn kern.maxvnodes Tweaking those may help, or he may just be exhausting available resources, IIRC its limited to 65k connections per interface, someone correct me if I am wrong. -- Joe Holden T: (UK) 02071009593 (AU) 282442321 E: joe@joeholden.co.uk