From owner-freebsd-net@FreeBSD.ORG Thu Jul 17 17:45:38 2003 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A630837B401 for ; Thu, 17 Jul 2003 17:45:37 -0700 (PDT) Received: from mail.sandvine.com (sandvine.com [199.243.201.138]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0DAFC43F3F for ; Thu, 17 Jul 2003 17:45:37 -0700 (PDT) (envelope-from sloach@sandvine.com) Received: by mail.sandvine.com with Internet Mail Service (5.5.2653.19) id <305LF8B2>; Thu, 17 Jul 2003 20:45:36 -0400 Message-ID: From: Scot Loach To: 'Bosko Milekic' , Scot Loach Date: Thu, 17 Jul 2003 20:45:29 -0400 MIME-Version: 1.0 X-Mailer: Internet Mail Service (5.5.2653.19) Content-Type: text/plain; charset="iso-8859-1" cc: "'freebsd-net@freebsd.org'" Subject: RE: Kernel tuning for large maxsockets X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Jul 2003 00:45:39 -0000 Bosko: The problem I have is that each of the four pcb types has its own zone that is preallocated to hold maxsockets pcbs. This is a waste of kva that could be better used. Since I'm not using divert sockets or raw sockets, I would rather cap these zones to a small constant number, and increase the others as much as possible. I should have a patch for this next week sometime. -----Original Message----- From: Bosko Milekic [mailto:bmilekic@technokratis.com] Sent: Wednesday, July 16, 2003 5:52 AM To: Scot Loach Cc: 'freebsd-net@freebsd.org' Subject: Re: Kernel tuning for large maxsockets On Tue, Jul 15, 2003 at 05:22:55PM -0400, Scot Loach wrote: > Currently, whenever maxsockets is increased, this causes kernel memory to be > preallocated for each type of pcb (tcp, udp, raw, divert). The number of > pcbs preallocated for each of these is always the same as maxsockets. > > This is probably a waste of memory for raw sockets and divert sockets, since > they would not normally be used in large numbers. A large server could > save kvm by reducing the number of divert and raw pcbs preallocated. For > example, on a machine configured for 200,000 maxsockets we would save 75MB > of kvm out of a total of 262MB that would have been preallocated. This kvm > savings can be used to increase maxsockets even more. What version of FreeBSD are you talking about here? In -current, the pcbs come off of zones which are capped at a maximum w.r.t. maxsockets. The kva space comes out of kmem_map and the objects are kept cached in their respective zones. One thing to note is that the zones are setup with UMA_ZONE_NOFREE, so the pages wired down for pcb use are probably never unwired. I don't know why, exactly, UMA_ZONE_NOFREE is required for all of the pcb zones in the netinet code. > Is there any reason I should not modify the kernel code to only let a small, > fixed number of raw and divert pcbs be preallocated instead of having them > scale with maxsockets? > > Next, does this seem like a generally useful thing that could be rolled back > into the source tree? I could make this a kernel option or a tunable sysctl > variable. > > thanks > > Scot Loach -- Bosko Milekic * bmilekic@technokratis.com * bmilekic@FreeBSD.org TECHNOkRATIS Consulting Services * http://www.technokratis.com/