Date: Mon, 19 Oct 2020 16:55:03 +0000 (UTC) From: Mark Johnston <markj@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r366839 - head/sys/vm Message-ID: <202010191655.09JGt36s042305@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: markj Date: Mon Oct 19 16:55:03 2020 New Revision: 366839 URL: https://svnweb.freebsd.org/changeset/base/366839 Log: uma: Avoid depleting keg reserves when filling a bucket zone_import() fetches a free or partially free slab from the keg and then uses its items to populate an array, typically filling a bucket. If a single allocation causes the keg to drop below its minimum reserve, the inner loop ends. However, if the bucket is still not full and M_USE_RESERVE is specified, the outer loop will continue to fetch items from the keg. If M_USE_RESERVE is specified and the number of free items is below the reserved limit, we should return only a single item. Otherwise, if the bucket size is larger than the reserve, all of the reserved items may end up in a single per-CPU bucket, invisible to other CPUs. Reviewed by: rlibby MFC after: 2 weeks Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D26771 Modified: head/sys/vm/uma_core.c Modified: head/sys/vm/uma_core.c ============================================================================== --- head/sys/vm/uma_core.c Mon Oct 19 16:54:06 2020 (r366838) +++ head/sys/vm/uma_core.c Mon Oct 19 16:55:03 2020 (r366839) @@ -3734,10 +3734,17 @@ zone_import(void *arg, void **bucket, int max, int dom stripe = howmany(max, vm_ndomains); #endif dom = &keg->uk_domain[slab->us_domain]; - while (slab->us_freecount && i < max) { + do { bucket[i++] = slab_alloc_item(keg, slab); - if (dom->ud_free_items <= keg->uk_reserve) - break; + if (dom->ud_free_items <= keg->uk_reserve) { + /* + * Avoid depleting the reserve after a + * successful item allocation, even if + * M_USE_RESERVE is specified. + */ + KEG_UNLOCK(keg, slab->us_domain); + goto out; + } #ifdef NUMA /* * If the zone is striped we pick a new slab for every @@ -3751,13 +3758,14 @@ zone_import(void *arg, void **bucket, int max, int dom vm_ndomains > 1 && --stripe == 0) break; #endif - } + } while (slab->us_freecount != 0 && i < max); KEG_UNLOCK(keg, slab->us_domain); + /* Don't block if we allocated any successfully. */ flags &= ~M_WAITOK; flags |= M_NOWAIT; } - +out: return i; }
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202010191655.09JGt36s042305>