From owner-svn-src-stable-12@freebsd.org Wed Oct 31 02:02:14 2018 Return-Path: Delivered-To: svn-src-stable-12@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B0FEA1072837; Wed, 31 Oct 2018 02:02:14 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 65BBD8539C; Wed, 31 Oct 2018 02:02:14 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 478A325CC0; Wed, 31 Oct 2018 02:02:14 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w9V22EKv045995; Wed, 31 Oct 2018 02:02:14 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w9V22C20045988; Wed, 31 Oct 2018 02:02:12 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201810310202.w9V22C20045988@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Wed, 31 Oct 2018 02:02:12 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r339939 - in stable/12: share/man/man9 sys/kern sys/sys sys/vm sys/x86/acpica X-SVN-Group: stable-12 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in stable/12: share/man/man9 sys/kern sys/sys sys/vm sys/x86/acpica X-SVN-Commit-Revision: 339939 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-stable-12@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for only the 12-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 31 Oct 2018 02:02:15 -0000 Author: markj Date: Wed Oct 31 02:02:12 2018 New Revision: 339939 URL: https://svnweb.freebsd.org/changeset/base/339939 Log: MFC r339452, r339664: Create some global domainsets and refactor NUMA registration. Approved by: re (kib) Modified: stable/12/share/man/man9/domainset.9 stable/12/sys/kern/kern_cpuset.c stable/12/sys/sys/domainset.h stable/12/sys/vm/vm_init.c stable/12/sys/vm/vm_phys.c stable/12/sys/vm/vm_phys.h stable/12/sys/x86/acpica/srat.c Directory Properties: stable/12/ (props changed) Modified: stable/12/share/man/man9/domainset.9 ============================================================================== --- stable/12/share/man/man9/domainset.9 Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/share/man/man9/domainset.9 Wed Oct 31 02:02:12 2018 (r339939) @@ -24,14 +24,11 @@ .\" .\" $FreeBSD$ .\" -.Dd March 24, 2018 +.Dd October 20, 2018 .Dt DOMAINSET 9 .Os .Sh NAME .Nm domainset(9) -\(em -.Nm domainset_create , -.Nm sysctl_handle_domainset . .Nd domainset functions and operation .Sh SYNOPSIS .In sys/_domainset.h @@ -46,6 +43,8 @@ struct domainset { }; .Ed .Pp +.Fn DOMAINSET_RR +.Fn DOMAINSET_PREF domain .Ft struct domainset * .Fn domainset_create "const struct domainset *key" .Ft int @@ -98,6 +97,12 @@ This gives good distribution among memory domains whil efficiency higher and is preferential to round-robin for general use. .El .Pp +The +.Fn DOMAINSET_RR +and +.Fn DOMAINSET_PREF +provide pointers to global pre-defined policies for use when the +desired policy is known at compile time. The .Fn domainset_create function takes a partially filled in domainset as a key and returns a Modified: stable/12/sys/kern/kern_cpuset.c ============================================================================== --- stable/12/sys/kern/kern_cpuset.c Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/sys/kern/kern_cpuset.c Wed Oct 31 02:02:12 2018 (r339939) @@ -119,6 +119,8 @@ __FBSDID("$FreeBSD$"); */ LIST_HEAD(domainlist, domainset); +struct domainset __read_mostly domainset_prefer[MAXMEMDOM]; +struct domainset __read_mostly domainset_roundrobin; static uma_zone_t cpuset_zone; static uma_zone_t domainset_zone; @@ -1369,28 +1371,53 @@ cpuset_setithread(lwpid_t id, int cpu) } /* + * Initialize static domainsets after NUMA information is available. This is + * called very early during boot. + */ +void +domainset_init(void) +{ + struct domainset *dset; + int i; + + dset = &domainset_roundrobin; + DOMAINSET_COPY(&all_domains, &dset->ds_mask); + dset->ds_policy = DOMAINSET_POLICY_ROUNDROBIN; + dset->ds_prefer = -1; + _domainset_create(dset, NULL); + + for (i = 0; i < vm_ndomains; i++) { + dset = &domainset_prefer[i]; + DOMAINSET_COPY(&all_domains, &dset->ds_mask); + dset->ds_policy = DOMAINSET_POLICY_PREFER; + dset->ds_prefer = i; + _domainset_create(dset, NULL); + } +} + +/* * Create the domainset for cpuset 0, 1 and cpuset 2. */ void domainset_zero(void) { struct domainset *dset; - int i; mtx_init(&cpuset_lock, "cpuset", NULL, MTX_SPIN | MTX_RECURSE); dset = &domainset0; - DOMAINSET_ZERO(&dset->ds_mask); - for (i = 0; i < vm_ndomains; i++) - DOMAINSET_SET(i, &dset->ds_mask); + DOMAINSET_COPY(&all_domains, &dset->ds_mask); dset->ds_policy = DOMAINSET_POLICY_FIRSTTOUCH; dset->ds_prefer = -1; - (void)domainset_empty_vm(dset); curthread->td_domain.dr_policy = _domainset_create(dset, NULL); domainset_copy(dset, &domainset2); domainset2.ds_policy = DOMAINSET_POLICY_INTERLEAVE; kernel_object->domain.dr_policy = _domainset_create(&domainset2, NULL); + + /* Remove empty domains from the global policies. */ + LIST_FOREACH(dset, &cpuset_domains, ds_link) + (void)domainset_empty_vm(dset); } /* Modified: stable/12/sys/sys/domainset.h ============================================================================== --- stable/12/sys/sys/domainset.h Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/sys/sys/domainset.h Wed Oct 31 02:02:12 2018 (r339939) @@ -32,8 +32,8 @@ #define _SYS_DOMAINSET_H_ #include - #include +#include #define _NDOMAINSETBITS _BITSET_BITS #define _NDOMAINSETWORDS __bitset_words(DOMAINSET_SETSIZE) @@ -96,6 +96,12 @@ struct domainset { domainid_t ds_order[MAXMEMDOM]; /* nth domain table. */ }; +extern struct domainset domainset_prefer[MAXMEMDOM]; +#define DOMAINSET_PREF(domain) (&domainset_prefer[(domain)]) +extern struct domainset domainset_roundrobin; +#define DOMAINSET_RR() (&domainset_roundrobin) + +void domainset_init(void); void domainset_zero(void); /* Modified: stable/12/sys/vm/vm_init.c ============================================================================== --- stable/12/sys/vm/vm_init.c Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/sys/vm/vm_init.c Wed Oct 31 02:02:12 2018 (r339939) @@ -68,6 +68,7 @@ __FBSDID("$FreeBSD$"); #include +#include #include #include #include @@ -115,6 +116,11 @@ SYSINIT(vm_mem, SI_SUB_VM, SI_ORDER_FIRST, vm_mem_init static void vm_mem_init(void *dummy) { + + /* + * Initialize static domainsets, used by various allocators. + */ + domainset_init(); /* * Initializes resident memory structures. From here on, all physical Modified: stable/12/sys/vm/vm_phys.c ============================================================================== --- stable/12/sys/vm/vm_phys.c Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/sys/vm/vm_phys.c Wed Oct 31 02:02:12 2018 (r339939) @@ -46,6 +46,7 @@ __FBSDID("$FreeBSD$"); #include #include +#include #include #include #include @@ -584,6 +585,31 @@ vm_phys_init(void) } rw_init(&vm_phys_fictitious_reg_lock, "vmfctr"); +} + +/* + * Register info about the NUMA topology of the system. + * + * Invoked by platform-dependent code prior to vm_phys_init(). + */ +void +vm_phys_register_domains(int ndomains, struct mem_affinity *affinity, + int *locality) +{ +#ifdef NUMA + int i; + + vm_ndomains = ndomains; + mem_affinity = affinity; + mem_locality = locality; + + for (i = 0; i < vm_ndomains; i++) + DOMAINSET_SET(i, &all_domains); +#else + (void)ndomains; + (void)affinity; + (void)locality; +#endif } /* Modified: stable/12/sys/vm/vm_phys.h ============================================================================== --- stable/12/sys/vm/vm_phys.h Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/sys/vm/vm_phys.h Wed Oct 31 02:02:12 2018 (r339939) @@ -88,6 +88,8 @@ void vm_phys_free_contig(vm_page_t m, u_long npages); void vm_phys_free_pages(vm_page_t m, int order); void vm_phys_init(void); vm_page_t vm_phys_paddr_to_vm_page(vm_paddr_t pa); +void vm_phys_register_domains(int ndomains, struct mem_affinity *affinity, + int *locality); vm_page_t vm_phys_scan_contig(int domain, u_long npages, vm_paddr_t low, vm_paddr_t high, u_long alignment, vm_paddr_t boundary, int options); void vm_phys_set_pool(int pool, vm_page_t m, int order); Modified: stable/12/sys/x86/acpica/srat.c ============================================================================== --- stable/12/sys/x86/acpica/srat.c Wed Oct 31 02:01:28 2018 (r339938) +++ stable/12/sys/x86/acpica/srat.c Wed Oct 31 02:02:12 2018 (r339939) @@ -153,10 +153,6 @@ parse_slit(void) acpi_unmap_table(slit); slit = NULL; -#ifdef NUMA - /* Tell the VM about it! */ - mem_locality = vm_locality_table; -#endif return (0); } @@ -481,13 +477,6 @@ parse_srat(void) return (-1); } -#ifdef NUMA - vm_ndomains = ndomain; - for (int i = 0; i < vm_ndomains; i++) - DOMAINSET_SET(i, &all_domains); - mem_affinity = mem_info; -#endif - return (0); } @@ -511,7 +500,8 @@ parse_acpi_tables(void *dummy) if (parse_srat() < 0) return; init_mem_locality(); - (void) parse_slit(); + (void)parse_slit(); + vm_phys_register_domains(ndomain, mem_info, vm_locality_table); } SYSINIT(parse_acpi_tables, SI_SUB_VM - 1, SI_ORDER_FIRST, parse_acpi_tables, NULL);