From owner-freebsd-hackers@FreeBSD.ORG Mon May 5 20:12:03 2003 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 16B3837B401 for ; Mon, 5 May 2003 20:12:03 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8AFCA43F85 for ; Mon, 5 May 2003 20:12:02 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0018.cvx21-bradley.dialup.earthlink.net ([209.179.192.18] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19Css2-0001oZ-00; Mon, 05 May 2003 20:11:59 -0700 Message-ID: <3EB727B6.FFC89521@mindspring.com> Date: Mon, 05 May 2003 20:10:46 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: sjlai@synatech.com.au References: <20030505232029.A46013@pobox.com.> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a41e42fb83694a87910254fd4f25708cca350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: freebsd-hackers@freebsd.org Subject: Re: 180,000 kevents - out of memory errors X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 May 2003 03:12:03 -0000 Simon Lai wrote: > I want to watch approx 200,000 sockets > using kevent, but I get an ENOMEM error > when adding kevent structures to the kqueue. > The machine in question has 1.5GB RAM > and can happily accept 270,000 TCP connections, > but I need kevent to manage that many sockets, > select() being a little too slow. > > I want to add one kevent struct to the queue > for each connection. > > Currently I can only queue around 80,000 - 90,000 > kevents, with this config - > > maxfiles=250000 > nmbclusters=35000 > nmbufs=210000 > > with this config I can make 180,000 connections and > queue 180,000 kevents - > > maxfiles=200000 > nmbclusters=40000 > nmbufs=240000 > > what should I do to improve the number of kevents > I can queue? 1) Add 2.5G more RAM. 2) Reconfigure the KVA space to use 1G for UVA and 3G for KVA, instead of 2G/2G, which is the default, so you won't run out of page mappings. 3) Disable IPSEC. 4) Separate the allocation of tcpcb's and udpcb's; there is no reason to support 200,000 for both, when you can support 200,000 for one, and 1,000 or less for the other, and save all that memory. -- Terry