From owner-dev-commits-src-main@freebsd.org Wed Apr 14 17:03:44 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B27635E19A0; Wed, 14 Apr 2021 17:03:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FL8284hCfz3lZH; Wed, 14 Apr 2021 17:03:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 936F221642; Wed, 14 Apr 2021 17:03:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 13EH3idi038615; Wed, 14 Apr 2021 17:03:44 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 13EH3ikX038614; Wed, 14 Apr 2021 17:03:44 GMT (envelope-from git) Date: Wed, 14 Apr 2021 17:03:44 GMT Message-Id: <202104141703.13EH3ikX038614@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: 29bb6c19f044 - main - domainset: Define additional global policies MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 29bb6c19f044274cc6693f0c2463ca22106cd3ab Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Apr 2021 17:03:44 -0000 The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=29bb6c19f044274cc6693f0c2463ca22106cd3ab commit 29bb6c19f044274cc6693f0c2463ca22106cd3ab Author: Mark Johnston AuthorDate: 2021-04-14 16:56:39 +0000 Commit: Mark Johnston CommitDate: 2021-04-14 17:03:33 +0000 domainset: Define additional global policies Add global definitions for first-touch and interleave policies. The former may be useful for UMA, which implements a similar policy without using domainset iterators. No functional change intended. Reviewed by: mav MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D29104 --- share/man/man9/domainset.9 | 20 ++++++++++++++++++-- sys/kern/kern_cpuset.c | 38 ++++++++++++++++++++++++-------------- sys/sys/domainset.h | 4 ++++ 3 files changed, 46 insertions(+), 16 deletions(-) diff --git a/share/man/man9/domainset.9 b/share/man/man9/domainset.9 index 5660f13afa88..969d9dd3b554 100644 --- a/share/man/man9/domainset.9 +++ b/share/man/man9/domainset.9 @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 30, 2018 +.Dd April 14, 2021 .Dt DOMAINSET 9 .Os .Sh NAME @@ -46,6 +46,10 @@ struct domainset { .Ft struct domainset * .Fn DOMAINSET_FIXED domain .Ft struct domainset * +.Fn DOMAINSET_FT +.Ft struct domainset * +.Fn DOMAINSET_IL +.Ft struct domainset * .Fn DOMAINSET_RR .Ft struct domainset * .Fn DOMAINSET_PREF domain @@ -104,6 +108,8 @@ efficiency higher and is preferential to round-robin for general use. .Pp The .Fn DOMAINSET_FIXED , +.Fn DOMAINSET_FT , +.Fn DOMAINSET_IL , .Fn DOMAINSET_RR and .Fn DOMAINSET_PREF @@ -111,8 +117,18 @@ macros provide pointers to global pre-defined policies for use when the desired policy is known at compile time. .Fn DOMAINSET_FIXED is a policy which only permits allocations from the specified domain. +.Fn DOMAINSET_FT +is a policy which attempts to allocate memory local to the current CPU, +falling back to a round-robin policy if the initial allocation fails. +.Fn DOMAINSET_IL +and .Fn DOMAINSET_RR -provides round-robin selection among all domains in the system. +provide round-robin selection among all domains in the system, corresponding +to the +.Dv DOMAINSET_POLICY_INTERLEAVE +and +.Dv DOMAINSET_POLICY_ROUNDROBIN +policies, respectively. The .Fn DOMAINSET_PREF policies attempt allocation from the specified domain, but unlike diff --git a/sys/kern/kern_cpuset.c b/sys/kern/kern_cpuset.c index 19ad3fd20955..119df57c7b02 100644 --- a/sys/kern/kern_cpuset.c +++ b/sys/kern/kern_cpuset.c @@ -119,7 +119,9 @@ __FBSDID("$FreeBSD$"); */ LIST_HEAD(domainlist, domainset); +struct domainset __read_mostly domainset_firsttouch; struct domainset __read_mostly domainset_fixed[MAXMEMDOM]; +struct domainset __read_mostly domainset_interleave; struct domainset __read_mostly domainset_prefer[MAXMEMDOM]; struct domainset __read_mostly domainset_roundrobin; @@ -130,7 +132,7 @@ static struct setlist cpuset_ids; static struct domainlist cpuset_domains; static struct unrhdr *cpuset_unr; static struct cpuset *cpuset_zero, *cpuset_default, *cpuset_kernel; -static struct domainset domainset0, domainset2; +static struct domainset *domainset0, *domainset2; /* Return the size of cpuset_t at the kernel level */ SYSCTL_INT(_kern_sched, OID_AUTO, cpusetsize, CTLFLAG_RD | CTLFLAG_CAPRD, @@ -568,7 +570,7 @@ domainset_create(const struct domainset *domain) if (domain->ds_policy == DOMAINSET_POLICY_PREFER && !DOMAINSET_ISSET(domain->ds_prefer, &domain->ds_mask)) return (NULL); - if (!DOMAINSET_SUBSET(&domainset0.ds_mask, &domain->ds_mask)) + if (!DOMAINSET_SUBSET(&domainset0->ds_mask, &domain->ds_mask)) return (NULL); ndomain = uma_zalloc(domainset_zone, M_WAITOK | M_ZERO); domainset_copy(domain, ndomain); @@ -1532,6 +1534,18 @@ domainset_init(void) struct domainset *dset; int i; + dset = &domainset_firsttouch; + DOMAINSET_COPY(&all_domains, &dset->ds_mask); + dset->ds_policy = DOMAINSET_POLICY_FIRSTTOUCH; + dset->ds_prefer = -1; + _domainset_create(dset, NULL); + + dset = &domainset_interleave; + DOMAINSET_COPY(&all_domains, &dset->ds_mask); + dset->ds_policy = DOMAINSET_POLICY_INTERLEAVE; + dset->ds_prefer = -1; + _domainset_create(dset, NULL); + dset = &domainset_roundrobin; DOMAINSET_COPY(&all_domains, &dset->ds_mask); dset->ds_policy = DOMAINSET_POLICY_ROUNDROBIN; @@ -1554,7 +1568,7 @@ domainset_init(void) } /* - * Create the domainset for cpuset 0, 1 and cpuset 2. + * Define the domainsets for cpuset 0, 1 and cpuset 2. */ void domainset_zero(void) @@ -1563,15 +1577,11 @@ domainset_zero(void) mtx_init(&cpuset_lock, "cpuset", NULL, MTX_SPIN | MTX_RECURSE); - dset = &domainset0; - DOMAINSET_COPY(&all_domains, &dset->ds_mask); - dset->ds_policy = DOMAINSET_POLICY_FIRSTTOUCH; - dset->ds_prefer = -1; - curthread->td_domain.dr_policy = _domainset_create(dset, NULL); + domainset0 = &domainset_firsttouch; + curthread->td_domain.dr_policy = domainset0; - domainset_copy(dset, &domainset2); - domainset2.ds_policy = DOMAINSET_POLICY_INTERLEAVE; - kernel_object->domain.dr_policy = _domainset_create(&domainset2, NULL); + domainset2 = &domainset_interleave; + kernel_object->domain.dr_policy = domainset2; /* Remove empty domains from the global policies. */ LIST_FOREACH_SAFE(dset, &cpuset_domains, ds_link, tmp) @@ -1613,7 +1623,7 @@ cpuset_thread0(void) LIST_INSERT_HEAD(&cpuset_ids, set, cs_link); refcount_init(&set->cs_ref, 1); set->cs_flags = CPU_SET_ROOT | CPU_SET_RDONLY; - set->cs_domain = &domainset0; + set->cs_domain = domainset0; cpuset_zero = set; cpuset_root = &set->cs_mask; @@ -1630,7 +1640,7 @@ cpuset_thread0(void) set = uma_zalloc(cpuset_zone, M_WAITOK | M_ZERO); error = cpuset_init(set, cpuset_zero, NULL, NULL, 2); KASSERT(error == 0, ("Error creating kernel set: %d\n", error)); - set->cs_domain = &domainset2; + set->cs_domain = domainset2; cpuset_kernel = set; /* @@ -2315,7 +2325,7 @@ kern_cpuset_setdomain(struct thread *td, cpulevel_t level, cpuwhich_t which, * across all domains. */ if (domainset_empty_vm(&domain)) - domainset_copy(&domainset2, &domain); + domainset_copy(domainset2, &domain); switch (level) { case CPU_LEVEL_ROOT: diff --git a/sys/sys/domainset.h b/sys/sys/domainset.h index 7b1180012f4c..2113196d8d26 100644 --- a/sys/sys/domainset.h +++ b/sys/sys/domainset.h @@ -95,6 +95,10 @@ struct domainset { domainid_t ds_order[MAXMEMDOM]; /* nth domain table. */ }; +extern struct domainset domainset_firsttouch; +#define DOMAINSET_FT() (&domainset_firsttouch) +extern struct domainset domainset_interleave; +#define DOMAINSET_IL() (&domainset_interleave) extern struct domainset domainset_fixed[MAXMEMDOM], domainset_prefer[MAXMEMDOM]; #define DOMAINSET_FIXED(domain) (&domainset_fixed[(domain)]) #define DOMAINSET_PREF(domain) (&domainset_prefer[(domain)])