Date: Thu, 27 Feb 2020 02:58:01 -0800 From: Ryan Libby <rlibby@gmail.com> To: Jeff Roberson <jeff@freebsd.org> Cc: src-committers <src-committers@freebsd.org>, svn-src-all <svn-src-all@freebsd.org>, svn-src-head <svn-src-head@freebsd.org> Subject: Re: svn commit: r358377 - head/sys/vm Message-ID: <CAHgpiFyZZ9U9ZBVM%2BZN%2Bo_%2BWh%2BbrKGAX8=HjoUyNdvBhOFz%2BNw@mail.gmail.com> In-Reply-To: <202002270823.01R8NAAQ032671@repo.freebsd.org> References: <202002270823.01R8NAAQ032671@repo.freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, Feb 27, 2020 at 12:23 AM Jeff Roberson <jeff@freebsd.org> wrote: > > Author: jeff > Date: Thu Feb 27 08:23:10 2020 > New Revision: 358377 > URL: https://svnweb.freebsd.org/changeset/base/358377 > > Log: > A pair of performance improvements. > > Swap buckets on free as well as alloc so that alloc is always the most > cache-hot data. > > When selecting a zone domain for the round-robin bucket cache use the > local domain unless there is a severe imbalance. This does not affinitize > memory, only locks and queues. > > Reviewed by: markj, rlibby > Differential Revision: https://reviews.freebsd.org/D23824 > > Modified: > head/sys/vm/uma_core.c > > Modified: head/sys/vm/uma_core.c > ============================================================================== > --- head/sys/vm/uma_core.c Thu Feb 27 07:02:33 2020 (r358376) > +++ head/sys/vm/uma_core.c Thu Feb 27 08:23:10 2020 (r358377) > @@ -564,26 +564,29 @@ zone_domain_lock(uma_zone_t zone, int domain) > } > > /* > - * Search for the domain with the least cached items and return it, breaking > - * ties with a preferred domain by returning it. > + * Search for the domain with the least cached items and return it if it > + * is out of balance with the preferred domain. > */ > static __noinline int > zone_domain_lowest(uma_zone_t zone, int pref) > { > - long least, nitems; > + long least, nitems, prefitems; > int domain; > int i; > > - least = LONG_MAX; > + prefitems = least = LONG_MAX; > domain = 0; > for (i = 0; i < vm_ndomains; i++) { > nitems = ZDOM_GET(zone, i)->uzd_nitems; > if (nitems < least) { > domain = i; > least = nitems; > - } else if (nitems == least && (i == pref || domain == pref)) > - domain = pref; > + } > + if (domain == pref) > + prefitems = nitems; > } > + if (prefitems < least * 2) > + return (pref); If think this does need to be <=, not <, for the case where prefitems=0, but it wasn't the first domain. So e.g. if all domains nitems are 0, but pref=1, then with < we return domain=0. > > return (domain); > } > @@ -4102,8 +4105,11 @@ uma_zfree_arg(uma_zone_t zone, void *item, void *udata > bucket = &cache->uc_crossbucket; > } else > #endif > - if (bucket->ucb_cnt >= bucket->ucb_entries) > - bucket = &cache->uc_freebucket; > + if (bucket->ucb_cnt == bucket->ucb_entries && > + cache->uc_freebucket.ucb_cnt < > + cache->uc_freebucket.ucb_entries) > + cache_bucket_swap(&cache->uc_freebucket, > + &cache->uc_allocbucket); > if (__predict_true(bucket->ucb_cnt < bucket->ucb_entries)) { > cache_bucket_push(cache, bucket, item); > critical_exit();
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHgpiFyZZ9U9ZBVM%2BZN%2Bo_%2BWh%2BbrKGAX8=HjoUyNdvBhOFz%2BNw>