From owner-svn-src-all@FreeBSD.ORG Sun Nov 11 06:22:55 2012 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 0750619F for ; Sun, 11 Nov 2012 06:22:55 +0000 (UTC) (envelope-from peter@wemm.org) Received: from mail-la0-f54.google.com (mail-la0-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id 589E08FC0C for ; Sun, 11 Nov 2012 06:22:53 +0000 (UTC) Received: by mail-la0-f54.google.com with SMTP id e12so4918222lag.13 for ; Sat, 10 Nov 2012 22:22:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=wemm.org; s=google; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bQyCu1ZXMLmd0sYcFgwUzssX568q584UEmm/4ecMmpQ=; b=ZLNi0Fxkl3ibv+IFI4TsafKBGN/SsojldjDUvoVHGcK3HF9hWoDAzzu7iY2DMRRWfR cTtMbf/aLZpXgaxl06umz3fT76yVJQRSlWFroG0xSEaNd5i4SWKbnK+FQNguLfz9NWtR 3NWCGXYZAHkT5n3QOeK/gk8jRXOc2N+96Fkbk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=bQyCu1ZXMLmd0sYcFgwUzssX568q584UEmm/4ecMmpQ=; b=AivoCOC2DGIdlorco3gg+TICr/9/Y3ayP8RrZgStAqjey+3jadkTqh3EbbwUFCBPmJ xhq/IoL+rVaDbdHWzFeE+ODKrZPom51Ib0VlKhvfQ0ALsxQDO6+OBioWIWETCgTONGZA rZlhjjl0Tcr4cHjlB4e+vQdYdTPtwp5gMDWkgR/7WQM9dDukchnV4NUUL7OOD/hlSg2z XhR2x6MGnf9bAasmZ0gopcE1NYKQBVvPDtYZ24ufVnuDEhVxU0hw0+CnRHixWYOqidPl PLFGllKRTr+qZW3hToBOWoPWSNMQIyvXaG8+zhc9cfDwL/720JyeFdMwyZ+E73Za29W4 7jXA== MIME-Version: 1.0 Received: by 10.152.148.8 with SMTP id to8mr15049899lab.2.1352614972951; Sat, 10 Nov 2012 22:22:52 -0800 (PST) Received: by 10.112.100.230 with HTTP; Sat, 10 Nov 2012 22:22:52 -0800 (PST) In-Reply-To: <20121111061517.H1208@besplex.bde.org> References: <201211100208.qAA28e0v004842@svn.freebsd.org> <509DC25E.5030306@mu.org> <509E3162.5020702@FreeBSD.org> <509E7E7C.9000104@mu.org> <509E830D.5080006@mu.org> <1352568275.17290.85.camel@revolution.hippie.lan> <20121111061517.H1208@besplex.bde.org> Date: Sat, 10 Nov 2012 22:22:52 -0800 Message-ID: Subject: Re: svn commit: r242847 - in head/sys: i386/include kern From: Peter Wemm To: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Content-Type: text/plain; charset=ISO-8859-1 X-Gm-Message-State: ALoCoQlMqm+g2hDIOJ7igc/76yX54lk85KpGXPDIN1mSh9kjGwpJN6mINup4kpLWGRSlAoFoZ1op X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Nov 2012 06:22:55 -0000 On Sat, Nov 10, 2012 at 11:32 AM, Bruce Evans wrote: > On Sat, 10 Nov 2012, Peter Wemm wrote: > >> On Sat, Nov 10, 2012 at 9:24 AM, Ian Lepore >> wrote: >>> >>> On Sat, 2012-11-10 at 08:38 -0800, Alfred Perlstein wrote: >>>> >>>> It probably could be added, but then a bunch of other people would >>>> complain about the comment being too wordy or "not in English". >>> >>> >>> The fact that such a thing could happen explains much about the current >>> state of the code. An outsider could easily come to the conclusion that >>> the FreeBSD motto is something along the lines of "It should be as hard >>> to read as it was to write." >> >> >> Don't forget to explain that you get 1 maxusers per 2MB of physical >> memory which turns into 64 x 2k clusters and a whole series of side >> effects. >> >> Wouldn't it be nice if we could write "By default, mbuf clusters are >> capped at 6% of physical ram or 6% of kernel address space, whichever >> is smaller" ? >> >> That's a little easier than trying to explain >> maxusers = physpages / (2 * 1024 * 1024 / PAGE_SIZE) >> if (maxusers > 384) >> maxusers = 384 + ((maxusers - 384) / 8); >> nmbclusters = 1024 + maxusers * 64; >> >> I'd sure prefer to explain: >> >> /* pick smaller of kva pages or physical pages */ >> if ((physpages / 16) < (kvapages / 16)) >> nmbclusters = physpages / 16; >> else >> nmbclusters = kvapages / 16; >> >> Leave maxusers for calculating maxproc. > > > I prefer to write clear code and not echo it in comments: > > nmbclusters = min(physpages, kvapages) / 16; > > The most interesting point (the reason why 6% was chosen) is not > documented by either. The divide by 16 was based on observation of existing behavior, notwithstanding my confusion over the point that the slope change kicks in. nmbclusters = 1024 + maxusers * 64; Aside from the elevated starting point, that is 64 x 2K clusters per maxuser bogo-unit. Before the slope change, 2MB ram gives 1 maxuser bogo-unit, which gives 128K of clusters. 2M / 128K = 16, or 6.25%. After the slope change, 16MB ram gives 1 maxuser bogo-unit. 16M / 128k = 128.. or 0.78%. ie: current autoconfiguration is roughly 6% of ram below 768MB, and less than 1% of ram above 768MB. Given that the allocator has changed significantly over the last few years and is a cap now, not a fixed reservation, we should be setting these limits to a comfortable safe limit. Back when it was a fixed reservation we had far more in the way of constraints. Telling people "increase maxusers 10x" is silly. "maxusers" sets two things directly. maxproc and nmbclusers. maxproc = maxusers * 16. Having people increase maxproc as a side effect of trying to increase network buffers is not helpful. Having people do post-defaults tuning in understandable units is much more likely to give predictable results. Which is easier? Q: "How do I get an extra 2MB of clusters?" A1: "increase kern.ipc.nmbclusters by 1000" A2: "well, 2MB is 1000 clusters, and since you have mode than 768M of ram, you divide that 1000 clusters by 16 to get 62, then multiply that by 8 to reverse the maxusers slope factor above 768M, so you need to find the maxusers value you have and add 496 to it. That will probably increase your clusters by 2MB. Maybe." We've had kern.ipc.nmbclusters for years. It is simple to understand, easy to predict the outcome of a change, is runtime adjustable, is a *cap* and not a reservation (like it used to be) and does not require a reboot like maxusers tweaking would, and doesn't change maxproc as a side effect. Tune the caps sensibly by default and move on. maxusers must die. -- Peter Wemm - peter@wemm.org; peter@FreeBSD.org; peter@yahoo-inc.com; KI6FJV "All of this is for nothing if we don't go to the stars" - JMS/B5 "If Java had true garbage collection, most programs would delete themselves upon execution." -- Robert Sewell