From owner-svn-src-all@freebsd.org Thu Feb 14 01:24:59 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E278614DF814; Thu, 14 Feb 2019 01:24:58 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: from mail-it1-x133.google.com (mail-it1-x133.google.com [IPv6:2607:f8b0:4864:20::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B894B868CB; Thu, 14 Feb 2019 01:24:57 +0000 (UTC) (envelope-from chmeeedalf@gmail.com) Received: by mail-it1-x133.google.com with SMTP id y184so10957162itc.1; Wed, 13 Feb 2019 17:24:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WOjbXo39TpT/vI4ShRySZaUd1wXtI+HsVXjQZ9S9W4s=; b=KirxsFDWQRaGhyyM8eBhOwsoxPV6v8boHKXQtkC7PsMC18GVlJWaQT6XRjJQd6Bcxw sZQ0rgzy9g3+awLNzgLz+HwB1a02YWJb9jThV7fHewwtlzL6YKcx6LrMp87sMcs3dd76 tKHicVZUI9Y26GxjJpbi27yx6B6U8unv9GYUBUmVUrK8P+EZW/2XRvyaAR0Xy/UXCwhP +v+WbnhtZdBoFae51Y3ngC3NDke6IOcb4wdGk1W+1lxboT+cotFNaKsuWtkFcS6KkGf4 gXKP+MMRVWZJKsmVMvBKMgH7jWm9h0cNZVGOGiaWzsowlCl+6FX4d7rR9J0uuf56QcqY fkiQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WOjbXo39TpT/vI4ShRySZaUd1wXtI+HsVXjQZ9S9W4s=; b=a8L4nll9iFbLdsYfhjhNnFeGywIdLQ3xDhDU5Ey13q5mE35XPso0WE5W65mWCEOW/Q fAOcbE2XJbjm3LsdcO5d6g9WuYBvUCVgqStbqpOJusV5d3E3muz+kYNDj6tjvJXZUSHB 3bcvRDSdaQ73+JG5xbbxf9z2JyX0qZ9SsaBrQPFhK3U+jY2B9w3oKlGU3bAqEBTnaTGG 2HSrwSfXyvSGkmgkMu7SeCu/4qfyDFhLKvWs2KF0UtOVpAzQVHk0p8R6qzz4lLBmvcX8 Fp/pp91ccqDtrZO021+cbaACScQLMd44/GSBHrFUaAANlf6RfxMnOOpEf2WywNfvbtDv yFww== X-Gm-Message-State: AHQUAubtHjF2syppL5XoWQffylWh5RVv3rg8xRjdAb0PTQnLCNyfcylJ 2ounadHZScf28mlvyNJUR90wFuH5 X-Google-Smtp-Source: AHgI3IaVTeOXsqiLrBXLkBrS8xMT9jvhhL8Lmd3+2A60eUOgVz7U0TsjcrT0C2i/PnlKF6Gz7ZNMUQ== X-Received: by 2002:a02:9f86:: with SMTP id a6mr655957jam.87.1550107496538; Wed, 13 Feb 2019 17:24:56 -0800 (PST) Received: from ralga.knownspace (173-25-245-129.client.mchsi.com. [173.25.245.129]) by smtp.gmail.com with ESMTPSA id u18sm424974ioc.64.2019.02.13.17.24.55 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 13 Feb 2019 17:24:56 -0800 (PST) Date: Wed, 13 Feb 2019 19:24:50 -0600 From: Justin Hibbits To: Gleb Smirnoff Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r343030 - in head/sys: cam conf dev/md dev/nvme fs/fuse fs/nfsclient fs/smbfs kern sys ufs/ffs vm Message-ID: <20190213192450.32343d6a@ralga.knownspace> In-Reply-To: <201901150102.x0F12Hlt025856@repo.freebsd.org> References: <201901150102.x0F12Hlt025856@repo.freebsd.org> X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; powerpc64-portbld-freebsd13.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: B894B868CB X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=KirxsFDW; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of chmeeedalf@gmail.com designates 2607:f8b0:4864:20::133 as permitted sender) smtp.mailfrom=chmeeedalf@gmail.com X-Spamd-Result: default: False [-6.67 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; TO_DN_SOME(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; IP_SCORE(-2.70)[ip: (-8.90), ipnet: 2607:f8b0::/32(-2.54), asn: 15169(-1.98), country: US(-0.07)]; DKIM_TRACE(0.00)[gmail.com:+]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[3.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MX_GOOD(-0.01)[cached: alt3.gmail-smtp-in.l.google.com]; NEURAL_HAM_SHORT(-0.97)[-0.966,0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_LAST(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com.dwl.dnswl.org : 127.0.5.0] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Feb 2019 01:24:59 -0000 On Tue, 15 Jan 2019 01:02:17 +0000 (UTC) Gleb Smirnoff wrote: > Author: glebius > Date: Tue Jan 15 01:02:16 2019 > New Revision: 343030 > URL: https://svnweb.freebsd.org/changeset/base/343030 > > Log: > Allocate pager bufs from UMA instead of 80-ish mutex protected > linked list. > o In vm_pager_bufferinit() create pbuf_zone and start accounting on > how many pbufs are we going to have set. > In various subsystems that are going to utilize pbufs create > private zones via call to pbuf_zsecond_create(). The latter calls > uma_zsecond_create(), and sets a limit on created zone. After startup > preallocate pbufs according to requirements of all pbuf zones. > > Subsystems that used to have a private limit with old allocator > now have private pbuf zones: md(4), fusefs, NFS client, smbfs, VFS > cluster, FFS, swap, vnode pager. > > The following subsystems use shared pbuf zone: cam(4), nvme(4), > physio(9), aio(4). They should have their private limits, but > changing that is out of scope of this commit. > > o Fetch tunable value of kern.nswbuf from init_param2() and while > here move NSWBUF_MIN to opt_param.h and eliminate opt_swap.h, that > was holding only this option. > Default values aren't touched by this commit, but they probably > should be reviewed wrt to modern hardware. > > This change removes a tight bottleneck from sendfile(2) operation, > that uses pbufs in vnode pager. Other pagers also would benefit from > faster allocation. > > Together with: gallatin > Tested by: pho > > Modified: > head/sys/cam/cam_periph.c > head/sys/conf/options > head/sys/dev/md/md.c > head/sys/dev/nvme/nvme_ctrlr.c > head/sys/fs/fuse/fuse_main.c > head/sys/fs/fuse/fuse_vnops.c > head/sys/fs/nfsclient/nfs_clbio.c > head/sys/fs/nfsclient/nfs_clport.c > head/sys/fs/smbfs/smbfs_io.c > head/sys/fs/smbfs/smbfs_vfsops.c > head/sys/kern/kern_physio.c > head/sys/kern/subr_param.c > head/sys/kern/vfs_aio.c > head/sys/kern/vfs_bio.c > head/sys/kern/vfs_cluster.c > head/sys/sys/buf.h > head/sys/ufs/ffs/ffs_rawread.c > head/sys/vm/swap_pager.c > head/sys/vm/vm_pager.c > head/sys/vm/vnode_pager.c > Hi Gleb, This seems to break 32-bit platforms, or at least 32-bit book-e powerpc, which has a limited KVA space (~500MB). It preallocates I've seen over 2500 pbufs, at 128kB each, eating up over 300MB KVA, leaving very little left for the rest of runtime. I spent a couple hours earlier today debugging with Mark Johnston, and his consensus is that the vnode_pbuf_zone is too big on 32-bit platforms. Unfortunately I know very little about this area, so can't provide much extra insight, but can readily reproduce the issues I see triggered by this change, so am willing to help where I can. - Justin