From owner-freebsd-net Mon Dec 10 3:15:31 2001 Delivered-To: freebsd-net@freebsd.org Received: from mailtoaster1.pipeline.ch (mailtoaster1.pipeline.ch [62.48.0.70]) by hub.freebsd.org (Postfix) with SMTP id 8FC8C37B419 for ; Mon, 10 Dec 2001 03:15:25 -0800 (PST) Received: (qmail 79601 invoked from network); 10 Dec 2001 11:15:13 -0000 Received: from unknown (HELO pipeline.ch) ([62.48.21.179]) (envelope-sender ) by mailtoaster1.pipeline.ch (qmail-ldap-1.03) with SMTP for ; 10 Dec 2001 11:15:13 -0000 Message-ID: <3C1498AD.E9B53A10@pipeline.ch> Date: Mon, 10 Dec 2001 12:12:45 +0100 From: Andre Oppermann X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: Mike Silbersack Cc: net@freebsd.org, David Xu , Mike Barcroft , Leo Bicknell Subject: Re: mbuf / maxfiles / maxsockets / etc autoscaling patch References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org Cool stuff and overdue for a loooong time! A few comments on the code: - IMHO it would be helpful to print the chosen values into the boot dmesg (maybe larger parts of it only with verbose). This aids explaining strange symtoms if someone adds/removes RAM and is not aware of autoscaling. "I took out 128MB of RAM and my webserver starts to fail..." Autoscaling enabled, setting values based on 64MB of usable RAM Autoscaling mproc=1024, mfiles=8192, msocket=4096, callout=9232, nmbcl=2048, nsfbuf=1024, tcphashsize=512 - I would suggest to append "MIN|MAX" to the variable instead of prepending it +#define TCBHASHPERMB 8 +#define TCBHASHAUTOMIN 512 +#define TCBHASHAUTOMAX 8192 - An update to the tuning man page describing this autoscaling is missing from this patch. -- Andre Mike Silbersack wrote: > > Here's the autoscaling patch I was mumbling about earlier this week. > With this patch applied, the necessity of tuning maxusers when one > upgrades to a machine with more ram should be removed in most cases. > (This patch is only to -current, the mbuf changes will make it not apply > cleanly to -stable patch if there is sufficient demand right now.) > > Here's a quick look at the size of various memory allocations with various > maxusers sizes and with the autoscaling patch: > > With maxusers: > > musers mproc mfiles msocket callout nmbcl nsfbuf tcp hash size > 32 532 1064 1064 1612 1024 1024 512 > 64 1044 2088 2088 3148 1536 1536 512 > 128 2068 4136 4136 6220 2560 2560 512 > 256 4116 8232 8232 12364 4608 4608 512 > > With autoscaling: > > MB ram mproc mfiles msocket callout nmbcl nsfbuf tcp hash size > 32 512 4096 2048 4624 1024 1024 512 > 64 1024 8192 4096 9232 2048 1024 512 > 128 2048 16384 8192 18448 4096 2048 1024 > 256 4096 32768 16384 36880 8192 4096 2048 > 384 6144 49152 24576 55312 12288 6144 3072 > 512 8192 65536 32767 73744 16384 8192 4096 > (Values above this start to flatten out due to #defined maximums) > > Note that in general calculations are of the following form: > > value = max(maxusers-derived value, autoscale-derived value); > value = loader tuned value if present > > As such, under no circumstances will people suddenly see a decrease in > various parameters when they upgrade to an autoscaling kernel; only > increases. > > I'm sure that there will be much commotion about what scaling factors are > correct. To make changes to these easy, I have grouped all the mins, > scaling factors, and maxes in param.h - tweaking them is quite simple. > > I included mins and maxes to make sure that autoscaling doesn't cause > problems by creating low values on small memory machines and also so that > it does not specify really high values on 2GB+ machines. The high case is > what worries me; I have not heard much about how well maxsockets / > nmbclusters > 32767 really works. If people running high volume systems > that actively use that many simultaneous sockets + clusters + files, I'd > be glad to bump up the maxes. > > Oh, there's one more kicker thrown in; I changed maxfilesperproc to equal > 9/10ths of maxfiles, and maxprocperuid to equal 9/10 maxproc; this'll help > to prevent a single process or user from forkbombing the system or running > it out of file handles with a default configuration. > > Please review. > > Thanks, > > Mike "Silby" Silbersack > > ------------------------------------------------------------------------ > Name: autoscale.patch > autoscale.patch Type: Plain Text (TEXT/PLAIN) > Encoding: BASE64 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message