From owner-freebsd-arch Thu Feb 28 13:25:38 2002 Delivered-To: freebsd-arch@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id D7D3637B43E for ; Thu, 28 Feb 2002 13:25:09 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.11.6/8.9.1) id g1SLP8738805; Thu, 28 Feb 2002 13:25:08 -0800 (PST) (envelope-from dillon) Date: Thu, 28 Feb 2002 13:25:08 -0800 (PST) From: Matthew Dillon Message-Id: <200202282125.g1SLP8738805@apollo.backplane.com> To: Jeff Roberson Cc: Terry Lambert , arch@FreeBSD.ORG Subject: Re: Slab allocator References: <20020228142424.R529-100000@mail.chesapeake.net> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG :I'm very interested in potential optimizations to my current locking :strategies. I understand why critical sections would be a good thing in :the fast path. I actually started to implement it this way originally. I :haven't been able to come up with a really clean way to implement the :cache flushing that doesn't have some nasty side effect though. For :instance, telling each cpu to flush it's caches would be somewhat :possible, but then you'd need some way to effectively block and restart :each cpu once the entire flush operation was done. This could lead to the :page daemon thread being suspended until each cpu had entered malloc. :That could be well over a time slice given that a processor may want to :continue scheduling user threads and not actually do any allocations. As :it is cpus are only blocked for the duration of the zone draining, so this :would increase the latency here. : :Anyway, I'd appreciate more comments on the subject. Keep in mind the :bucket flushing operations, and potentially statistics gathering as well. : :Thanks, :Jeff In looking over the code only cache_drain() seems to need to lock some 'other' cpu's cache. So the immediate question is: "When does cache_drain() get called and is immediate action necessary?" In looking at your code, the sequence is: vm_pageout_scan() -> uma_reclaim() -> zone_drain() To me this indicates that timing is not absolutely critical, in which case I would recommend that instead of doing this directly in vm_pageout_scan() you instead flag it in the cpu's per-cpu area and do the drain on a per-cpu basis. Where to put the test I am not entirely sure, but it seems worthwhile considering that if you can accomplish this you can get rid of the per-cpu mutex entirely and use a critical section instead. Another alternative is to be a little more pro-active about the draining. For example, when allocating or freeing from a zone the code would also check for 'excessive cached data' and clean it out for the calling cpu. I am not advocating that you do all of this before you commit. On the contrary, I believe it is far more important to first get what you have stabilized and into the tree and then work on tuning it as another stage, similar to the two and three stage commit I intend to do for critical_*(). Stage 1: get the thing working and safe, Stage 2: cleanup and optimize, Stage 3: repeat, etc... -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message