From owner-svn-src-head@freebsd.org Mon Feb 18 21:37:15 2019 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C4E7114ECA95; Mon, 18 Feb 2019 21:37:14 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (glebi.us [198.45.61.253]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "cell.glebi.us", Issuer "cell.glebi.us" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 32447873D9; Mon, 18 Feb 2019 21:37:13 +0000 (UTC) (envelope-from glebius@freebsd.org) Received: from cell.glebi.us (localhost [127.0.0.1]) by cell.glebi.us (8.15.2/8.15.2) with ESMTPS id x1ILbC14077058 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Mon, 18 Feb 2019 13:37:12 -0800 (PST) (envelope-from glebius@freebsd.org) Received: (from glebius@localhost) by cell.glebi.us (8.15.2/8.15.2/Submit) id x1ILbBeX077057; Mon, 18 Feb 2019 13:37:11 -0800 (PST) (envelope-from glebius@freebsd.org) X-Authentication-Warning: cell.glebi.us: glebius set sender to glebius@freebsd.org using -f Date: Mon, 18 Feb 2019 13:37:11 -0800 From: Gleb Smirnoff To: Bruce Evans Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r344188 - in head: lib/libc/sys sys/vm Message-ID: <20190218213711.GR83215@FreeBSD.org> References: <201902152336.x1FNaNUo039321@repo.freebsd.org> <20190217011341.S833@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190217011341.S833@besplex.bde.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Rspamd-Queue-Id: 32447873D9 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.99 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.99)[-0.990,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; ASN(0.00)[asn:2906, ipnet:198.45.48.0/20, country:US] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Feb 2019 21:37:15 -0000 Bruce, On Sun, Feb 17, 2019 at 02:58:43AM +1100, Bruce Evans wrote: B> This is only a rollback for the vnode pager pbufs sub-pool. Total B> resource usage (preallocated kva and maximum on RAM that can be mapped B> into this kva) is still about 5/2 times higher than before in my B> configuration. It would be 7/2 times higher if I configured fuse and B> smbfs. r343030 changed the allocation methods in all subsystems except B> out-of-tree modules, and broke at least the KBI for these modules (*), B> so it is easy to find the full expansion except for these modules by B> looking at the diffs (I found the use in fuse and smbfs by grepping B> some subtrees). Also, the user's and vfs_bio's resource limit is still B> broken by expanding it by this factor of 5/2 or more. B> B> In the old allocation method, there was a single pool of pbufs of size B> nswbuf which normally has its normal limiting value of 256, where this B> magic 256 is hard-coded in vfs_bio.c, but if the user somehow knows B> about this and the tunable kern.nswbuf, then it can be overridden. B> The limit of 256 was documented in pbuf(9), but the tunable was never B> documented AFAIK. The variable nswbuf for this was documented in B> pbuf(9). The 256 entries are shared between any number of subsystems. B> Most subsystems limited themselves to nswbuf/2 entries, and the man B> page recommended this. This gave overcommit by a factor of about 5/2 B> in my configuration (there are 7 subsystems, but some of these have a B> smaller limit). B> B> Now there each subsystem has a separate pool. The size of the sub-pool B> is still usually nswbuf / 2. This gives overallocation by a factor of B> about 5/2 in my configuration. The overcommit only causes minor B> performance problems. 2 subsystems might use all of the buffers, and B> then all all the other subsystems have to wait, but it is rare for even B> 2 subsystems to be under load at the same time. It is more of a problem B> that the limit is too small for a single subsystem. The overallocation B> gives worse problems such as crashing at boot time or a little later when B> the user or auto-tuning has maxed out nswbuf. This is all true and this is intentional. Before any two subsystems could screw up the system by allocating all bufs. Now they are more separated from each other and yes, the price is extra memory. B> Now the user has to know even more arcane details to limit the kva, and B> it is impossible to recover the old behaviour. To get the old limit, B> kern.nswbuf must be set to (256 * 2 / 5) in my configuration, but B> that significantly decreases the number of buffers for each subsystem. B> Users might already have set kern.nswbuf to a large value. Since most B> subsystems used to use half of that many buffers, the wastage from setting B> it large for the benefit of 1 subsystem was at most a factor of 2. Now B> the limit can't be increased as much without running out of kva, and the B> safe increase is more arcane and machine-dependent (starting with the B> undocumented default being 8 times higher for 64-bit systems, but only B> for 1 of the subsystems). Yes, this sucks. If you can propose auto-tuning algorithm, I'm all for it. For now I'm just shifting defaults from something that was reasonable in 2000 to something that is going to be reasonable in 2020. -- Gleb Smirnoff