Date: Thu, 08 Nov 2012 12:16:58 -0800 From: Alfred Perlstein <bright@mu.org> To: Peter Wemm <peter@wemm.org> Cc: src-committers@freebsd.org, John Baldwin <jhb@freebsd.org>, svn-src-all@freebsd.org, Alfred Perlstein <alfred@freebsd.org>, svn-src-head@freebsd.org, Konstantin Belousov <kostikbel@gmail.com> Subject: Re: svn commit: r242029 - head/sys/kern Message-ID: <509C133A.1080601@mu.org> In-Reply-To: <CAGE5yCobr4ZU0DEWZSez1kp4jo_V4gSby0kGqFF06Dev_Cc_jA@mail.gmail.com> References: <201210250146.q9P1kLi8043704@svn.freebsd.org> <20121025080551.GG35915@deviant.kiev.zoral.com.ua> <201210250950.57161.jhb@freebsd.org> <509B501F.5050109@mu.org> <CAGE5yCobr4ZU0DEWZSez1kp4jo_V4gSby0kGqFF06Dev_Cc_jA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------000807030900030106000406 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Peter, can you let me know what you think? -Alfred On 11/7/12 11:46 PM, Peter Wemm wrote: > On Wed, Nov 7, 2012 at 10:24 PM, Alfred Perlstein <bright@mu.org> wrote: >> [[ + peter ]] >> >> Folks, I spent quite a bit of time trying to figure out how to resolve >> maxusers scaling in a happy way for all. >> >> I think I came up with a solution. >> >> This solution should work for i386, and other 32 bit platforms, as well as >> scaling well for 64 bit (or higher) platforms that have virtually unlimited >> AND 64bit with limited kernel address space. >> >> Here is how it works: >> >> We calculate the maxusers value based on physical memory, and then clamp it >> down if physical memory exceeds kernel addressable memory. >> >> The algorithm actually remains the same for all architectures, with the >> exception that machines with large kernel address space it is no longer >> clamped at 384. >> >> I've attached a test program that lets you play with various values for >> VM_MIN_KERNEL_ADDRESS, VM_MAX_KERNEL_ADDRESS and physpages. (argv[1, 2, 3] >> respectively.) >> >> Please give me your feedback. > This is still bogus. VM_MIN_KERNEL_ADDRESS and VM_MAX_KERNEL_ADDRESS > have no bearing on how much space should be allocated for mbuf > clusters on amd64. If anything, you want dmapbase / dmapend if you > want a practical cap for amd64. Even then, jumbo clusters are >4K so > they come out of kva rather than direct map. > > maxusers is the wrong thing for this. maxusers should, if anything, > be used to set things like kern.maxproc. Preferably it should be > deleted entirely and sysctl.conf should be used to change > kern.maxproc. > > Setting limits for the mbuf / cluster pool should be a MD parameter. > > Trying to scale maxusers based on physical ram in order to get mbuf > cluster limits set as a side effect is just plain wrong. > > It makes no more sense than trying to set nmbclusters based on > PRINTF_BUFR_SIZE, and then trying to scale PRINTF_BUFR_SIZE in order > to get desirable second and third order side effects. > > Scale nmbclusters based on physical ram, with a MD method for capping > it for when there are MD limits (eg: disproportionately small kva on > an i386 PAE machine). Don't use maxusers. --------------000807030900030106000406 Content-Type: message/rfc822; name="Attached Message" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="Attached Message" Return-Path: <owner-src-committers@FreeBSD.org> X-Original-To: bright@mu.org Delivered-To: bright@mu.org Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by elvis.mu.org (Postfix) with ESMTP id 67D781A3C55 for <bright@mu.org>; Thu, 8 Nov 2012 12:15:19 -0800 (PST) Received: from hub.freebsd.org (hub.FreeBSD.org [8.8.178.136]) by mx2.freebsd.org (Postfix) with ESMTP id 301193B5242 for <bright@mu.org>; Thu, 8 Nov 2012 20:15:19 +0000 (UTC) Received: by hub.freebsd.org (Postfix) id 17ECAC2C; Thu, 8 Nov 2012 20:15:19 +0000 (UTC) Delivered-To: alfred@freebsd.org Received: by hub.freebsd.org (Postfix, from userid 538) id 0FF21C26; Thu, 8 Nov 2012 20:15:19 +0000 (UTC) Delivered-To: src-committers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 9366CC21; Thu, 8 Nov 2012 20:15:13 +0000 (UTC) (envelope-from alfred@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) by mx1.freebsd.org (Postfix) with ESMTP id 751B08FC17; Thu, 8 Nov 2012 20:15:13 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.5/8.14.5) with ESMTP id qA8KFDO6015306; Thu, 8 Nov 2012 20:15:13 GMT (envelope-from alfred@svn.freebsd.org) Received: (from alfred@localhost) by svn.freebsd.org (8.14.5/8.14.5/Submit) id qA8KFD1O015304; Thu, 8 Nov 2012 20:15:13 GMT (envelope-from alfred@svn.freebsd.org) Message-Id: <201211082015.qA8KFD1O015304@svn.freebsd.org> From: Alfred Perlstein <alfred@FreeBSD.org> Date: Thu, 8 Nov 2012 20:15:13 +0000 (UTC) To: src-committers@freebsd.org, svn-src-user@freebsd.org Subject: svn commit: r242783 - in user/alfred/9-alfred/sys: i386/include kern X-SVN-Group: user MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: owner-src-committers@FreeBSD.org Precedence: bulk X-Loop: FreeBSD.ORG Author: alfred Date: Thu Nov 8 20:15:12 2012 New Revision: 242783 URL: http://svnweb.freebsd.org/changeset/base/242783 Log: Divorce autotune nmbclusters from maxusers. Provide arch specific override maximum. Suggested by: peter Modified: user/alfred/9-alfred/sys/i386/include/vmparam.h user/alfred/9-alfred/sys/kern/kern_mbuf.c Modified: user/alfred/9-alfred/sys/i386/include/vmparam.h ============================================================================== --- user/alfred/9-alfred/sys/i386/include/vmparam.h Thu Nov 8 18:11:31 2012 (r242782) +++ user/alfred/9-alfred/sys/i386/include/vmparam.h Thu Nov 8 20:15:12 2012 (r242783) @@ -202,4 +202,9 @@ #define ZERO_REGION_SIZE (64 * 1024) /* 64KB */ +#ifndef MAX_AUTOTUNE_NMBCLUSTERS +/* old maxusers max value. */ +#define MAX_AUTOTUNE_NMBCLUSTERS (1024 + 384 * 64) +#endif + #endif /* _MACHINE_VMPARAM_H_ */ Modified: user/alfred/9-alfred/sys/kern/kern_mbuf.c ============================================================================== --- user/alfred/9-alfred/sys/kern/kern_mbuf.c Thu Nov 8 18:11:31 2012 (r242782) +++ user/alfred/9-alfred/sys/kern/kern_mbuf.c Thu Nov 8 20:15:12 2012 (r242783) @@ -102,6 +102,30 @@ int nmbjumbo9; /* limits number of 9k int nmbjumbo16; /* limits number of 16k jumbo clusters */ struct mbstat mbstat; +static int +nmbclusters_from_physpages(void) +{ + long factor; + long rv; + + factor = physmem / (2 * 1024 * 1024 / PAGE_SIZE); + if (factor < 32) + factor = 32; + /* after 384, switch scale to 1/4 */ + if (factor > 384) + factor = 384 + (factor - 384) / 4; + rv = 1024 + factor * 64; + /* + * allow a platform specific override to prevent exhausting + * kernel memory on large memory + small address space machines. + */ +#ifdef MAX_AUTOTUNE_NMBCLUSTERS + if (rv > MAX_AUTOTUNE_NMBCLUSTERS) + rv = MAX_AUTOTUNE_NMBCLUSTERS +#endif + return (rv); +} + /* * tunable_mbinit() has to be run before init_maxsockets() thus * the SYSINIT order below is SI_ORDER_MIDDLE while init_maxsockets() @@ -114,7 +138,7 @@ tunable_mbinit(void *dummy) /* This has to be done before VM init. */ TUNABLE_INT_FETCH("kern.ipc.nmbclusters", &nmbclusters); if (nmbclusters == 0) - nmbclusters = 1024 + maxusers * 64; + nmbclusters = nmbclusters_from_physpages(); TUNABLE_INT_FETCH("kern.ipc.nmbjumbop", &nmbjumbop); if (nmbjumbop == 0) --------------000807030900030106000406--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?509C133A.1080601>