Date: Sun, 19 Sep 2010 12:41:16 +0100 From: "Robert N. M. Watson" <rwatson@freebsd.org> To: Andriy Gapon <avg@freebsd.org> Cc: Andre Oppermann <andre@freebsd.org>, Jeff Roberson <jeff@freebsd.org>, Jeff Roberson <jroberson@jroberson.net>, freebsd-hackers@freebsd.org Subject: Re: zfs + uma Message-ID: <8D2A1836-CA85-4F1B-A5A5-9B75A8E2DA51@freebsd.org> In-Reply-To: <4C95C804.1010701@freebsd.org> References: <4C93236B.4050906@freebsd.org> <4C935F56.4030903@freebsd.org> <alpine.BSF.2.00.1009181221560.86826@fledge.watson.org> <alpine.BSF.2.00.1009181135430.23448@desktop> <4C95C804.1010701@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 19 Sep 2010, at 09:21, Andriy Gapon wrote: >> I believe the combination of these approaches would significantly = solve the >> problem and should be relatively little new code. It should also = preserve the >> adaptable nature of the system without penalizing resource heavy = systems. I >> would be happy to review patches from anyone who wishes to undertake = it. >=20 > FWIW, the approach of simply limiting maximum bucket size based on = item size > seems to work rather well too, as my testing with zfs+uma shows. > I will also try to add code to completely bypass the per-cpu cache for = "really > huge" items. This is basically what malloc(9) does already: for small items, it = allocates from a series of fixed-size buckets (which could probably use = tuning), but maintains its own stats with respect to the types it maps = into the buckets. This is why there's double-counting between vmstat -z = and vmstat -m, since the former shows the buckets used to allocate the = latter. For large items, malloc(9) goes through UMA, but it's basically a = pass-through to VM, which directly provides pages. This means that for = small malloc types, you get per-CPU caches, and for large malloc types, = you don't. malloc(9) doesn't require fixed-size allocations, but also can't provide = the ctor/dtor partial tear-down caching, nor different effective working = sets of memory for different types. UMA should really only be used directly for memory types where tight = packing, per-CPU caching, and possibly partial tear-down, have benefits. = mbufs are a great example, because we allocate tons and tons of them = continuously in operation. More stable types allocated in smaller = quantities make very little sense, since we waste lots of memory = overhead in allocating buckets that won't be used, etc. Robert=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8D2A1836-CA85-4F1B-A5A5-9B75A8E2DA51>