From owner-svn-src-head@freebsd.org Sat Sep 7 14:32:48 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 257C9D4062; Sat, 7 Sep 2019 14:32:48 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-qt1-x82e.google.com (mail-qt1-x82e.google.com [IPv6:2607:f8b0:4864:20::82e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46QcMy2ls1z4cvb; Sat, 7 Sep 2019 14:32:46 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-qt1-x82e.google.com with SMTP id g13so10296628qtj.4; Sat, 07 Sep 2019 07:32:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=fMzRLsjOOdIcQoN+jt7UHDAlW7+rbJAck4jNhuU51PA=; b=ZQF/3hLYH/lwHOpw+nXPtZ2eCc4X0RmBvNHyAxbsxfBVYyCldxpJ+Y1LK78TzW886r 5SukNFKvUI1ye3WGzUiugRFi/IaE5VWwKfUqMSZeGfZaUkA313QivDIbTuGaBQMxLs/V qkJd+kMj3WMwO/gGQPq8rQG9K1rE1SMO6STRH8NoUokHJnYYfKt7BBZn0pLE9VJsyzV6 mh7Fv0GPjlaBQuKPdxSoxhObPqcsAxGxzpQMFU/uvLINfclpMi3veUunnlwmCA7cvIR6 0AVAOgu+Dq89bmtBW+OGoF8J639IlI5r3MsFAco8/X7y898motLcBlDBWy6ou+dqQg49 nCUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to:user-agent; bh=fMzRLsjOOdIcQoN+jt7UHDAlW7+rbJAck4jNhuU51PA=; b=dNYRREa1WxOHeLFrVTPLAJgWVpwFoRjtA8t4KotHuM7SdKvXHB7zhQw4rGkRHhrO1R nO/pXMG0HBVHLmIip+y61JU7TvHzl/fLfWywJ121VQeTu2yMXVGduXI0ATaUdftkLqTf FVMG+IKJaOiCDDKxyPAGh6oNhyP6b0YwlsfODIDd+SkTkFc+GfMG3B07Jxl4XnKGelFb 4ndfgYkM3i/B4E84VP129jg4B7PPwW48IerocWkbCW8s0XjDiSA30V5l8PbbyMqktcTL +Qt8wgyVwIn4j/MeiR43psKmw2Fqm/6ulyQjW4IPn2RyHNYKInkL75fxlD8DIqNbJDzd oX4w== X-Gm-Message-State: APjAAAUYrQl+oKMoxAXING31cNZKHtuBU4us6MmJnA/toj8rv30iAXDT 8JdF+r7EDKMa4ARCyAAgKMf72/rD4Kg= X-Google-Smtp-Source: APXvYqxzccaqsiOwRrRZsWniBgR2/brQo6cWjobGKwIbrPmsaz6fbMWvxmfn3CIb2N/6+a3cswvqYg== X-Received: by 2002:ac8:1793:: with SMTP id o19mr14504269qtj.64.1567866764553; Sat, 07 Sep 2019 07:32:44 -0700 (PDT) Received: from spy ([50.235.236.73]) by smtp.gmail.com with ESMTPSA id w11sm4752575qtj.10.2019.09.07.07.32.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 07 Sep 2019 07:32:43 -0700 (PDT) Sender: Mark Johnston Date: Sat, 7 Sep 2019 10:32:39 -0400 From: Mark Johnston To: Andriy Gapon Cc: Slawa Olhovchenkov , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r351673 - in head: lib/libmemstat share/man/man9 sys/cddl/compat/opensolaris/kern sys/kern sys/vm Message-ID: <20190907143239.GA3971@spy> References: <201909012222.x81MMh0F022462@repo.freebsd.org> <79c74018-1329-ee69-3480-e2f99821fa93@FreeBSD.org> <20190903161427.GA38096@zxy.spb.ru> <20190903220106.GB26733@raichu> <2b4562be-46d9-7781-ac35-2fa3bfb4551f@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2b4562be-46d9-7781-ac35-2fa3bfb4551f@FreeBSD.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Rspamd-Queue-Id: 46QcMy2ls1z4cvb X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=ZQF/3hLY; dmarc=none; spf=pass (mx1.freebsd.org: domain of markjdb@gmail.com designates 2607:f8b0:4864:20::82e as permitted sender) smtp.mailfrom=markjdb@gmail.com X-Spamd-Result: default: False [-1.05 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(0.00)[+ip6:2607:f8b0:4000::/36]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[gmail.com:+]; NEURAL_HAM_SHORT(-0.98)[-0.977,0]; FORGED_SENDER(0.30)[markj@freebsd.org,markjdb@gmail.com]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-2.88)[ip: (-9.32), ipnet: 2607:f8b0::/32(-2.75), asn: 15169(-2.27), country: US(-0.05)]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[markj@freebsd.org,markjdb@gmail.com]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(0.00)[gmail.com:s=20161025]; RECEIVED_SPAMHAUS_CSS(4.00)[73.236.235.50.khpj7ygk5idzvmvt5x4ziurxhy.zen.dq.spamhaus.net : 127.0.0.3]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[freebsd.org]; BAD_REP_POLICIES(0.10)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[e.2.8.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; MID_RHS_NOT_FQDN(0.50)[]; RCVD_TLS_ALL(0.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Sep 2019 14:32:48 -0000 On Wed, Sep 04, 2019 at 09:00:03AM +0300, Andriy Gapon wrote: > On 04/09/2019 01:01, Mark Johnston wrote: > > Slawa and I talked about this in the past. His complaint is that a > > large cache can take a significant amount of time to trim, and it > > manifests as a spike of CPU usage and contention on the zone lock. In > > particular, keg_drain() iterates over the list of free slabs with the > > keg lock held, and if the many items were freed to the keg while > > trimming/draining, the list can be quite long. This can have effects > > outside the zone, for example if we are reclaiming items from zones used > > by other UMA zones, like the bucket or slab zones. > > My concern is different, though. > I feel that having oversized caches for long periods of time produces a skewed > picture of memory usage. Particularly, some ZFS caches are sometimes extremely > oversized. I don't care much about details of consequences of such oversized > caches. I just think that that is not right on a more general level. > > > Reclaiming cached items when there is no demand for free pages seems > > wrong to me. > > It certainly was wrong before. > Now that we have a capability to trim a cache size to a workset size it doesn't > feel as wrong to me. One partial problem is that some UMA items are expensive to allocate and free. On amd64 slabs larger than the page size must be mapped into KVA, and this is not a scalable operation: pages must be inserted into and removed from kernel_object, and when removed we must issue a TLB shootdown to all CPUs. Proactively freeing such items from the cache might also exacerbate fragmentation over time. For direct-mapped items I think the tradeoff makes more sense and we could indeed start regularly freeing items based on the current WSS estimation. There has been a lot of work in the past year or two to make the page allocator cheaper and more scalable. > > We historically had similar problems with the page daemon, > > which last year was changed to perform smaller reclamations at a greater > > frequency. I suspect a better approach for UMA would be to similarly > > increase reclaim frequency and reduce the number of items freed in one > > go.