From owner-freebsd-net@FreeBSD.ORG Wed Jul 16 06:47:41 2003 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6172D37B401 for ; Wed, 16 Jul 2003 06:47:41 -0700 (PDT) Received: from godel.mtl.distributel.net (nat.MTL.distributel.NET [66.38.181.24]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9C75E43F75 for ; Wed, 16 Jul 2003 06:47:40 -0700 (PDT) (envelope-from bmilekic@technokratis.com) Received: from godel.mtl.distributel.net (localhost [127.0.0.1]) h6G9pkEH013577; Wed, 16 Jul 2003 09:51:46 GMT (envelope-from bmilekic@technokratis.com) Received: (from bmilekic@localhost) by godel.mtl.distributel.net (8.12.9/8.12.9/Submit) id h6G9pkCM013576; Wed, 16 Jul 2003 09:51:46 GMT X-Authentication-Warning: godel.mtl.distributel.net: bmilekic set sender to bmilekic@technokratis.com using -f Date: Wed, 16 Jul 2003 09:51:46 +0000 From: Bosko Milekic To: Scot Loach Message-ID: <20030716095146.GB13330@technokratis.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i cc: "'freebsd-net@freebsd.org'" Subject: Re: Kernel tuning for large maxsockets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Jul 2003 13:47:41 -0000 On Tue, Jul 15, 2003 at 05:22:55PM -0400, Scot Loach wrote: > Currently, whenever maxsockets is increased, this causes kernel memory to be > preallocated for each type of pcb (tcp, udp, raw, divert). The number of > pcbs preallocated for each of these is always the same as maxsockets. > > This is probably a waste of memory for raw sockets and divert sockets, since > they would not normally be used in large numbers. A large server could > save kvm by reducing the number of divert and raw pcbs preallocated. For > example, on a machine configured for 200,000 maxsockets we would save 75MB > of kvm out of a total of 262MB that would have been preallocated. This kvm > savings can be used to increase maxsockets even more. What version of FreeBSD are you talking about here? In -current, the pcbs come off of zones which are capped at a maximum w.r.t. maxsockets. The kva space comes out of kmem_map and the objects are kept cached in their respective zones. One thing to note is that the zones are setup with UMA_ZONE_NOFREE, so the pages wired down for pcb use are probably never unwired. I don't know why, exactly, UMA_ZONE_NOFREE is required for all of the pcb zones in the netinet code. > Is there any reason I should not modify the kernel code to only let a small, > fixed number of raw and divert pcbs be preallocated instead of having them > scale with maxsockets? > > Next, does this seem like a generally useful thing that could be rolled back > into the source tree? I could make this a kernel option or a tunable sysctl > variable. > > thanks > > Scot Loach -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/