Date: Wed, 13 Feb 2019 19:24:50 -0600 From: Justin Hibbits <chmeeedalf@gmail.com> To: Gleb Smirnoff <glebius@FreeBSD.org> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r343030 - in head/sys: cam conf dev/md dev/nvme fs/fuse fs/nfsclient fs/smbfs kern sys ufs/ffs vm Message-ID: <20190213192450.32343d6a@ralga.knownspace> In-Reply-To: <201901150102.x0F12Hlt025856@repo.freebsd.org> References: <201901150102.x0F12Hlt025856@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, 15 Jan 2019 01:02:17 +0000 (UTC) Gleb Smirnoff <glebius@FreeBSD.org> wrote: > Author: glebius > Date: Tue Jan 15 01:02:16 2019 > New Revision: 343030 > URL: https://svnweb.freebsd.org/changeset/base/343030 > > Log: > Allocate pager bufs from UMA instead of 80-ish mutex protected > linked list. > o In vm_pager_bufferinit() create pbuf_zone and start accounting on > how many pbufs are we going to have set. > In various subsystems that are going to utilize pbufs create > private zones via call to pbuf_zsecond_create(). The latter calls > uma_zsecond_create(), and sets a limit on created zone. After startup > preallocate pbufs according to requirements of all pbuf zones. > > Subsystems that used to have a private limit with old allocator > now have private pbuf zones: md(4), fusefs, NFS client, smbfs, VFS > cluster, FFS, swap, vnode pager. > > The following subsystems use shared pbuf zone: cam(4), nvme(4), > physio(9), aio(4). They should have their private limits, but > changing that is out of scope of this commit. > > o Fetch tunable value of kern.nswbuf from init_param2() and while > here move NSWBUF_MIN to opt_param.h and eliminate opt_swap.h, that > was holding only this option. > Default values aren't touched by this commit, but they probably > should be reviewed wrt to modern hardware. > > This change removes a tight bottleneck from sendfile(2) operation, > that uses pbufs in vnode pager. Other pagers also would benefit from > faster allocation. > > Together with: gallatin > Tested by: pho > > Modified: > head/sys/cam/cam_periph.c > head/sys/conf/options > head/sys/dev/md/md.c > head/sys/dev/nvme/nvme_ctrlr.c > head/sys/fs/fuse/fuse_main.c > head/sys/fs/fuse/fuse_vnops.c > head/sys/fs/nfsclient/nfs_clbio.c > head/sys/fs/nfsclient/nfs_clport.c > head/sys/fs/smbfs/smbfs_io.c > head/sys/fs/smbfs/smbfs_vfsops.c > head/sys/kern/kern_physio.c > head/sys/kern/subr_param.c > head/sys/kern/vfs_aio.c > head/sys/kern/vfs_bio.c > head/sys/kern/vfs_cluster.c > head/sys/sys/buf.h > head/sys/ufs/ffs/ffs_rawread.c > head/sys/vm/swap_pager.c > head/sys/vm/vm_pager.c > head/sys/vm/vnode_pager.c > Hi Gleb, This seems to break 32-bit platforms, or at least 32-bit book-e powerpc, which has a limited KVA space (~500MB). It preallocates I've seen over 2500 pbufs, at 128kB each, eating up over 300MB KVA, leaving very little left for the rest of runtime. I spent a couple hours earlier today debugging with Mark Johnston, and his consensus is that the vnode_pbuf_zone is too big on 32-bit platforms. Unfortunately I know very little about this area, so can't provide much extra insight, but can readily reproduce the issues I see triggered by this change, so am willing to help where I can. - Justin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20190213192450.32343d6a>