From nobody Sun May 18 18:15:01 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4b0prQ2MVjz5wbh3; Sun, 18 May 2025 18:15:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4b0prQ0mBDz40bP; Sun, 18 May 2025 18:15:02 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747592102; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zm3dD752bp/aYyHaMfiC9LFYVBBkXravsfYJLqoIaLk=; b=HeV8ddh/sK66dN0s+oxQtTU8B/TZ9ucMuC8qsE4WzlZB4FtLjaZhNV9/y/dfg9ywTLVnIu gv1Qydn5UHNvbt8BEppiKhGR8UuxwtbJ1CPmJxI2UpTDoGoI+s/0c6AKmSy5t8PipWTbNN kQWG2i+fjLNKYS1QamVEaOfoQE2sFiYOQKq2PsosblmbjY6DvXcOiqDLm1vnk0xTZc4fdd AMwkGdFi/ResvyxXFhzpgtoprygCXFRj9A+vkSKaQMi3LwFFmGIj4HDgmuRNyFxXy+vDst jzmh4ie6xe+kbfg6O9w/TUya1eWYefi8MpFQ3MOZ7aDRdzi6jnYHaHTCn4E3dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1747592102; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zm3dD752bp/aYyHaMfiC9LFYVBBkXravsfYJLqoIaLk=; b=H/kYMlxlV05dyeZ48eWWBPFPSgvJatzKy13TWqw7WPAdGzYRjV4Cw9ridIt4m5PC+JZKyE XbAuP8hjMo+CGu6vLsdzrafJ5YUDORfmk2ynPgFq8uW8HkI0mhbJJMGKCtTagwcFGFychB /c4D13YVuRZgdbEQ8nNxelxcRWbul3zSzeIHAcTsIUI0zNGPx77KhrgoJHqJkzlaLgZn9S 79Ac2rNhD7rQTB8jnNpH+cohmaZZZxUjM5aZoPdsdUAInLv133ydW5du59E6w+FPQnxazM F8Zg8fwoUT4/m2sQHdSiIgu3OKr5Izyc3Y5AUirGeiDQ1f5Hh7FIPSy9oigDZA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1747592102; a=rsa-sha256; cv=none; b=ZWX+mW3X72WvinxSPvk2oNAt30R/bJjYzZBXhDXBQH8raKg8y2F7dApAN1+anN/rwrHWTk 3IPV0CYCI1u5vyKqUx+sS0A1xHldMW4xMr8Nf8eWPgV2o82zARMC70xm1lsNbs+mO3b0f7 ex0vTPFTOSOt5PN9y4nZiI49UyVZgPXfdlDG1DZ7KrIH71EdTU7xo1pVJSFgD7WwB+P9jf 7EWzIV4VFJZe6TjgaNrFhi2LojxdUqt3q++Xo61Ng3jBJeiNUSMUz890TtvYGfbv3YTkcO 2AOg9W5iw/BTYPh7mqvKpYmiZZTsQSOhZIX902OHS9Q+GEG8Cfp8iYvkLBXoSg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4b0prQ022NzDS1; Sun, 18 May 2025 18:15:02 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 54IIF1iP005712; Sun, 18 May 2025 18:15:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 54IIF1jU005710; Sun, 18 May 2025 18:15:01 GMT (envelope-from git) Date: Sun, 18 May 2025 18:15:01 GMT Message-Id: <202505181815.54IIF1jU005710@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Doug Moore Subject: git: 7fa19ee28c90 - main - vm_page: reset iterator after domainset drops lock List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: dougm X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7fa19ee28c90c159fdb98d2339455a5201dc5993 Auto-Submitted: auto-generated The branch main has been updated by dougm: URL: https://cgit.FreeBSD.org/src/commit/?id=7fa19ee28c90c159fdb98d2339455a5201dc5993 commit 7fa19ee28c90c159fdb98d2339455a5201dc5993 Author: Doug Moore AuthorDate: 2025-05-18 18:09:00 +0000 Commit: Doug Moore CommitDate: 2025-05-18 18:09:00 +0000 vm_page: reset iterator after domainset drops lock Restore the WAITFAIL check on resetting the iterator in vm_page_alloc_domain_iter. It was dropped in 5239a9fb8662 ("vm_page: alloc_domain_iter") and blamed for a stress test failure. In fact, the failure was elsewhere, and removing the WAITFAIL check just made it less likely to be discovered, without fixing it. Modify vm_domainset_iter_page_init and vm_domainset_iter_page so that they can take an iterator argument and reset the iterator if vm_domainset_iter_releases and reacquires the object lock. Failure to reset the iterator in this case was the real problem. Reviewed by: alc, markj, kib Tested by: pho Differential Revision: https://reviews.freebsd.org/D50392 --- sys/vm/vm_domainset.c | 16 ++++++++++------ sys/vm/vm_domainset.h | 7 +++++-- sys/vm/vm_glue.c | 4 ++-- sys/vm/vm_page.c | 24 +++++++++++++----------- 4 files changed, 30 insertions(+), 21 deletions(-) diff --git a/sys/vm/vm_domainset.c b/sys/vm/vm_domainset.c index f6ac3c984cbf..7b8bf4c77663 100644 --- a/sys/vm/vm_domainset.c +++ b/sys/vm/vm_domainset.c @@ -39,6 +39,7 @@ #include #include #include +#include #include #include @@ -199,7 +200,7 @@ vm_domainset_iter_first(struct vm_domainset_iter *di, int *domain) void vm_domainset_iter_page_init(struct vm_domainset_iter *di, struct vm_object *obj, - vm_pindex_t pindex, int *domain, int *req) + vm_pindex_t pindex, int *domain, int *req, struct pctrie_iter *pages) { struct domainset_ref *dr; @@ -218,12 +219,12 @@ vm_domainset_iter_page_init(struct vm_domainset_iter *di, struct vm_object *obj, VM_ALLOC_NOWAIT; vm_domainset_iter_first(di, domain); if (vm_page_count_min_domain(*domain)) - vm_domainset_iter_page(di, obj, domain); + vm_domainset_iter_page(di, obj, domain, pages); } int vm_domainset_iter_page(struct vm_domainset_iter *di, struct vm_object *obj, - int *domain) + int *domain, struct pctrie_iter *pages) { if (__predict_false(DOMAINSET_EMPTY(&di->di_valid_mask))) return (ENOMEM); @@ -248,8 +249,11 @@ vm_domainset_iter_page(struct vm_domainset_iter *di, struct vm_object *obj, return (ENOMEM); /* Wait for one of the domains to accumulate some free pages. */ - if (obj != NULL) + if (obj != NULL) { VM_OBJECT_WUNLOCK(obj); + if (pages != NULL) + pctrie_iter_reset(pages); + } vm_wait_doms(&di->di_valid_mask, 0); if (obj != NULL) VM_OBJECT_WLOCK(obj); @@ -339,7 +343,7 @@ vm_domainset_iter_ignore(struct vm_domainset_iter *di, int domain) int vm_domainset_iter_page(struct vm_domainset_iter *di, struct vm_object *obj, - int *domain) + int *domain, struct pctrie_iter *pages) { return (EJUSTRETURN); @@ -347,7 +351,7 @@ vm_domainset_iter_page(struct vm_domainset_iter *di, struct vm_object *obj, void vm_domainset_iter_page_init(struct vm_domainset_iter *di, struct vm_object *obj, - vm_pindex_t pindex, int *domain, int *flags) + vm_pindex_t pindex, int *domain, int *flags, struct pctrie_iter *pages) { *domain = 0; diff --git a/sys/vm/vm_domainset.h b/sys/vm/vm_domainset.h index d2cfe362ae78..0d325a642f40 100644 --- a/sys/vm/vm_domainset.h +++ b/sys/vm/vm_domainset.h @@ -28,6 +28,8 @@ #ifndef __VM_DOMAINSET_H__ #define __VM_DOMAINSET_H__ +struct pctrie_iter; + struct vm_domainset_iter { struct domainset *di_domain; unsigned int *di_iter; @@ -40,9 +42,10 @@ struct vm_domainset_iter { }; int vm_domainset_iter_page(struct vm_domainset_iter *, struct vm_object *, - int *); + int *, struct pctrie_iter *); void vm_domainset_iter_page_init(struct vm_domainset_iter *, - struct vm_object *, vm_pindex_t, int *, int *); + struct vm_object *, vm_pindex_t, int *, int *, + struct pctrie_iter *); int vm_domainset_iter_policy(struct vm_domainset_iter *, int *); void vm_domainset_iter_policy_init(struct vm_domainset_iter *, struct domainset *, int *, int *); diff --git a/sys/vm/vm_glue.c b/sys/vm/vm_glue.c index c2a032b24000..94df2c2f9a9e 100644 --- a/sys/vm/vm_glue.c +++ b/sys/vm/vm_glue.c @@ -453,7 +453,7 @@ vm_thread_stack_create(struct domainset *ds, int pages) obj = vm_thread_kstack_size_to_obj(pages); if (vm_ndomains > 1) obj->domain.dr_policy = ds; - vm_domainset_iter_page_init(&di, obj, 0, &domain, &req); + vm_domainset_iter_page_init(&di, obj, 0, &domain, &req, NULL); do { /* * Get a kernel virtual address for this thread's kstack. @@ -480,7 +480,7 @@ vm_thread_stack_create(struct domainset *ds, int pages) vm_page_valid(ma[i]); pmap_qenter(ks, ma, pages); return (ks); - } while (vm_domainset_iter_page(&di, obj, &domain) == 0); + } while (vm_domainset_iter_page(&di, obj, &domain, NULL) == 0); return (0); } diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index 91250060190d..eec85a5eb6f4 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -2007,13 +2007,14 @@ vm_page_alloc_iter(vm_object_t object, vm_pindex_t pindex, int req, vm_page_t m; int domain; - vm_domainset_iter_page_init(&di, object, pindex, &domain, &req); + vm_domainset_iter_page_init(&di, object, pindex, &domain, &req, + pages); do { m = vm_page_alloc_domain_iter(object, pindex, domain, req, pages); if (m != NULL) break; - } while (vm_domainset_iter_page(&di, object, &domain) == 0); + } while (vm_domainset_iter_page(&di, object, &domain, pages) == 0); return (m); } @@ -2142,7 +2143,8 @@ again: * Not allocatable, give up. */ (void)vm_domain_alloc_fail(vmd, object, req); - pctrie_iter_reset(pages); + if ((req & VM_ALLOC_WAITFAIL) != 0) + pctrie_iter_reset(pages); return (NULL); } @@ -2254,7 +2256,7 @@ vm_page_alloc_contig(vm_object_t object, vm_pindex_t pindex, int req, start_segind = -1; - vm_domainset_iter_page_init(&di, object, pindex, &domain, &req); + vm_domainset_iter_page_init(&di, object, pindex, &domain, &req, NULL); do { m = vm_page_alloc_contig_domain(object, pindex, domain, req, npages, low, high, alignment, boundary, memattr); @@ -2266,7 +2268,7 @@ vm_page_alloc_contig(vm_object_t object, vm_pindex_t pindex, int req, npages, low, high) == -1) { vm_domainset_iter_ignore(&di, domain); } - } while (vm_domainset_iter_page(&di, object, &domain) == 0); + } while (vm_domainset_iter_page(&di, object, &domain, NULL) == 0); return (m); } @@ -2576,12 +2578,12 @@ vm_page_alloc_noobj(int req) vm_page_t m; int domain; - vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req); + vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req, NULL); do { m = vm_page_alloc_noobj_domain(domain, req); if (m != NULL) break; - } while (vm_domainset_iter_page(&di, NULL, &domain) == 0); + } while (vm_domainset_iter_page(&di, NULL, &domain, NULL) == 0); return (m); } @@ -2595,13 +2597,13 @@ vm_page_alloc_noobj_contig(int req, u_long npages, vm_paddr_t low, vm_page_t m; int domain; - vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req); + vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req, NULL); do { m = vm_page_alloc_noobj_contig_domain(domain, req, npages, low, high, alignment, boundary, memattr); if (m != NULL) break; - } while (vm_domainset_iter_page(&di, NULL, &domain) == 0); + } while (vm_domainset_iter_page(&di, NULL, &domain, NULL) == 0); return (m); } @@ -3316,7 +3318,7 @@ vm_page_reclaim_contig(int req, u_long npages, vm_paddr_t low, vm_paddr_t high, ret = ERANGE; - vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req); + vm_domainset_iter_page_init(&di, NULL, 0, &domain, &req, NULL); do { status = vm_page_reclaim_contig_domain(domain, req, npages, low, high, alignment, boundary); @@ -3329,7 +3331,7 @@ vm_page_reclaim_contig(int req, u_long npages, vm_paddr_t low, vm_paddr_t high, "from vm_page_reclaim_contig_domain()", status)); ret = ENOMEM; } - } while (vm_domainset_iter_page(&di, NULL, &domain) == 0); + } while (vm_domainset_iter_page(&di, NULL, &domain, NULL) == 0); return (ret); }