From owner-svn-src-all@freebsd.org Tue Nov 13 16:51:33 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1C30E112A7D9; Tue, 13 Nov 2018 16:51:33 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 980946D6DA; Tue, 13 Nov 2018 16:51:32 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 5D6A112F8F; Tue, 13 Nov 2018 16:51:32 +0000 (UTC) (envelope-from markj@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id wADGpWbu076059; Tue, 13 Nov 2018 16:51:32 GMT (envelope-from markj@FreeBSD.org) Received: (from markj@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id wADGpVFL076053; Tue, 13 Nov 2018 16:51:31 GMT (envelope-from markj@FreeBSD.org) Message-Id: <201811131651.wADGpVFL076053@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: markj set sender to markj@FreeBSD.org using -f From: Mark Johnston Date: Tue, 13 Nov 2018 16:51:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r340396 - in stable/12/sys: kern vm X-SVN-Group: stable-12 X-SVN-Commit-Author: markj X-SVN-Commit-Paths: in stable/12/sys: kern vm X-SVN-Commit-Revision: 340396 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 980946D6DA X-Spamd-Result: default: False [-106.88 / 200.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; ALLOW_DOMAIN_WHITELIST(-100.00)[FreeBSD.org]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; DMARC_NA(0.00)[FreeBSD.org]; RCVD_COUNT_THREE(0.00)[4]; MX_GOOD(-0.01)[cached: mx1.FreeBSD.org]; NEURAL_HAM_SHORT(-1.00)[-0.999,0]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; IP_SCORE(-3.77)[ip: (-9.91), ipnet: 2610:1c1:1::/48(-4.93), asn: 11403(-3.91), country: US(-0.09)] X-Rspamd-Server: mx1.freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Nov 2018 16:51:33 -0000 Author: markj Date: Tue Nov 13 16:51:30 2018 New Revision: 340396 URL: https://svnweb.freebsd.org/changeset/base/340396 Log: MFC r339925: Fix some problems that manifest when NUMA domain 0 is empty. Approved by: re (gjb) Modified: stable/12/sys/kern/kern_cpuset.c stable/12/sys/vm/uma_core.c stable/12/sys/vm/vm_init.c stable/12/sys/vm/vm_kern.c stable/12/sys/vm/vm_page.c stable/12/sys/vm/vm_pageout.c Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/kern/kern_cpuset.c ============================================================================== --- stable/12/sys/kern/kern_cpuset.c Tue Nov 13 15:28:27 2018 (r340395) +++ stable/12/sys/kern/kern_cpuset.c Tue Nov 13 16:51:30 2018 (r340396) @@ -492,20 +492,29 @@ _domainset_create(struct domainset *domain, struct dom } /* - * Are any of the domains in the mask empty? If so, silently - * remove them. If only empty domains are present, we must - * return failure. + * Are any of the domains in the mask empty? If so, silently + * remove them and update the domainset accordingly. If only empty + * domains are present, we must return failure. */ static bool domainset_empty_vm(struct domainset *domain) { - int i, max; + int i, j, max; max = DOMAINSET_FLS(&domain->ds_mask) + 1; - for (i = 0; i < max; i++) { - if (DOMAINSET_ISSET(i, &domain->ds_mask) && - VM_DOMAIN_EMPTY(i)) + for (i = 0; i < max; i++) + if (DOMAINSET_ISSET(i, &domain->ds_mask) && VM_DOMAIN_EMPTY(i)) DOMAINSET_CLR(i, &domain->ds_mask); + domain->ds_cnt = DOMAINSET_COUNT(&domain->ds_mask); + max = DOMAINSET_FLS(&domain->ds_mask) + 1; + for (i = j = 0; i < max; i++) { + if (DOMAINSET_ISSET(i, &domain->ds_mask)) + domain->ds_order[j++] = i; + else if (domain->ds_policy == DOMAINSET_POLICY_PREFER && + domain->ds_prefer == i && domain->ds_cnt > 1) { + domain->ds_policy = DOMAINSET_POLICY_ROUNDROBIN; + domain->ds_prefer = -1; + } } return (DOMAINSET_EMPTY(&domain->ds_mask)); @@ -1378,7 +1387,7 @@ cpuset_setithread(lwpid_t id, int cpu) /* * Initialize static domainsets after NUMA information is available. This is - * called very early during boot. + * called before memory allocators are initialized. */ void domainset_init(void) @@ -1407,7 +1416,7 @@ domainset_init(void) void domainset_zero(void) { - struct domainset *dset; + struct domainset *dset, *tmp; mtx_init(&cpuset_lock, "cpuset", NULL, MTX_SPIN | MTX_RECURSE); @@ -1422,8 +1431,9 @@ domainset_zero(void) kernel_object->domain.dr_policy = _domainset_create(&domainset2, NULL); /* Remove empty domains from the global policies. */ - LIST_FOREACH(dset, &cpuset_domains, ds_link) - (void)domainset_empty_vm(dset); + LIST_FOREACH_SAFE(dset, &cpuset_domains, ds_link, tmp) + if (domainset_empty_vm(dset)) + LIST_REMOVE(dset, ds_link); } /* Modified: stable/12/sys/vm/uma_core.c ============================================================================== --- stable/12/sys/vm/uma_core.c Tue Nov 13 15:28:27 2018 (r340395) +++ stable/12/sys/vm/uma_core.c Tue Nov 13 16:51:30 2018 (r340396) @@ -3608,29 +3608,30 @@ uma_zone_reserve_kva(uma_zone_t zone, int count) void uma_prealloc(uma_zone_t zone, int items) { + struct vm_domainset_iter di; uma_domain_t dom; uma_slab_t slab; uma_keg_t keg; - int domain, slabs; + int domain, flags, slabs; keg = zone_first_keg(zone); if (keg == NULL) return; KEG_LOCK(keg); slabs = items / keg->uk_ipers; - domain = 0; if (slabs * keg->uk_ipers < items) slabs++; + flags = M_WAITOK; + vm_domainset_iter_policy_ref_init(&di, &keg->uk_dr, &domain, &flags); while (slabs-- > 0) { - slab = keg_alloc_slab(keg, zone, domain, M_WAITOK); + slab = keg_alloc_slab(keg, zone, domain, flags); if (slab == NULL) return; MPASS(slab->us_keg == keg); dom = &keg->uk_domain[slab->us_domain]; LIST_INSERT_HEAD(&dom->ud_free_slab, slab, us_link); - do { - domain = (domain + 1) % vm_ndomains; - } while (VM_DOMAIN_EMPTY(domain)); + if (vm_domainset_iter_policy(&di, &domain) != 0) + break; } KEG_UNLOCK(keg); } Modified: stable/12/sys/vm/vm_init.c ============================================================================== --- stable/12/sys/vm/vm_init.c Tue Nov 13 15:28:27 2018 (r340395) +++ stable/12/sys/vm/vm_init.c Tue Nov 13 16:51:30 2018 (r340396) @@ -123,11 +123,17 @@ vm_mem_init(void *dummy) domainset_init(); /* - * Initializes resident memory structures. From here on, all physical + * Initialize resident memory structures. From here on, all physical * memory is accounted for, and we use only virtual addresses. */ vm_set_page_size(); virtual_avail = vm_page_startup(virtual_avail); + + /* + * Set an initial domain policy for thread0 so that allocations + * can work. + */ + domainset_zero(); #ifdef UMA_MD_SMALL_ALLOC /* Announce page availability to UMA. */ Modified: stable/12/sys/vm/vm_kern.c ============================================================================== --- stable/12/sys/vm/vm_kern.c Tue Nov 13 15:28:27 2018 (r340395) +++ stable/12/sys/vm/vm_kern.c Tue Nov 13 16:51:30 2018 (r340396) @@ -800,7 +800,6 @@ kmem_bootstrap_free(vm_offset_t start, vm_size_t size) vmd = vm_pagequeue_domain(m); vm_domain_free_lock(vmd); vm_phys_free_pages(m, 0); - vmd->vmd_page_count++; vm_domain_free_unlock(vmd); vm_domain_freecnt_inc(vmd, 1); Modified: stable/12/sys/vm/vm_page.c ============================================================================== --- stable/12/sys/vm/vm_page.c Tue Nov 13 15:28:27 2018 (r340395) +++ stable/12/sys/vm/vm_page.c Tue Nov 13 16:51:30 2018 (r340396) @@ -855,11 +855,6 @@ vm_page_startup(vm_offset_t vaddr) */ vm_reserv_init(); #endif - /* - * Set an initial domain policy for thread0 so that allocations - * can work. - */ - domainset_zero(); return (vaddr); } Modified: stable/12/sys/vm/vm_pageout.c ============================================================================== --- stable/12/sys/vm/vm_pageout.c Tue Nov 13 15:28:27 2018 (r340395) +++ stable/12/sys/vm/vm_pageout.c Tue Nov 13 16:51:30 2018 (r340396) @@ -2072,41 +2072,41 @@ vm_pageout_init(void) static void vm_pageout(void) { - int error; - int i; + struct proc *p; + struct thread *td; + int error, first, i; + p = curproc; + td = curthread; + swap_pager_swap_init(); - snprintf(curthread->td_name, sizeof(curthread->td_name), "dom0"); - error = kthread_add(vm_pageout_laundry_worker, NULL, curproc, NULL, - 0, 0, "laundry: dom0"); - if (error != 0) - panic("starting laundry for domain 0, error %d", error); - for (i = 1; i < vm_ndomains; i++) { + for (first = -1, i = 0; i < vm_ndomains; i++) { if (VM_DOMAIN_EMPTY(i)) { if (bootverbose) printf("domain %d empty; skipping pageout\n", i); continue; } - - error = kthread_add(vm_pageout_worker, (void *)(uintptr_t)i, - curproc, NULL, 0, 0, "dom%d", i); - if (error != 0) { - panic("starting pageout for domain %d, error %d\n", - i, error); + if (first == -1) + first = i; + else { + error = kthread_add(vm_pageout_worker, + (void *)(uintptr_t)i, p, NULL, 0, 0, "dom%d", i); + if (error != 0) + panic("starting pageout for domain %d: %d\n", + i, error); } error = kthread_add(vm_pageout_laundry_worker, - (void *)(uintptr_t)i, curproc, NULL, 0, 0, - "laundry: dom%d", i); + (void *)(uintptr_t)i, p, NULL, 0, 0, "laundry: dom%d", i); if (error != 0) - panic("starting laundry for domain %d, error %d", - i, error); + panic("starting laundry for domain %d: %d", i, error); } - error = kthread_add(uma_reclaim_worker, NULL, curproc, NULL, - 0, 0, "uma"); + error = kthread_add(uma_reclaim_worker, NULL, p, NULL, 0, 0, "uma"); if (error != 0) panic("starting uma_reclaim helper, error %d\n", error); - vm_pageout_worker((void *)(uintptr_t)0); + + snprintf(td->td_name, sizeof(td->td_name), "dom%d", first); + vm_pageout_worker((void *)(uintptr_t)first); } /*