From owner-freebsd-smp  Sun Feb  2 16:47:40 1997
Return-Path: <owner-smp>
Received: (from root@localhost)
          by freefall.freebsd.org (8.8.5/8.8.5) id QAA26275
          for smp-outgoing; Sun, 2 Feb 1997 16:47:40 -0800 (PST)
Received: from parkplace.cet.co.jp (parkplace.cet.co.jp [202.32.64.1])
          by freefall.freebsd.org (8.8.5/8.8.5) with ESMTP id QAA26266
          for <freebsd-smp@freebsd.org>; Sun, 2 Feb 1997 16:47:36 -0800 (PST)
Received: from localhost (michaelh@localhost) by parkplace.cet.co.jp (8.8.5/CET-v2.1) with SMTP id AAA04791; Mon, 3 Feb 1997 00:47:06 GMT
Date: Mon, 3 Feb 1997 09:47:05 +0900 (JST)
From: Michael Hancock <michaelh@cet.co.jp>
To: "David S. Miller" <davem@jenolan.rutgers.edu>
cc: netdev@roxanne.nuclecu.unam.mx, roque@di.fc.ul.pt, freebsd-smp@freebsd.org
Subject: Re: SMP
In-Reply-To: <199702021202.HAA09281@jenolan.caipgeneral>
Message-ID: <Pine.SV4.3.95.970203093914.4624A-100000@parkplace.cet.co.jp>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-smp@freebsd.org
X-Loop: FreeBSD.org
Precedence: bulk

On Sun, 2 Feb 1997, David S. Miller wrote:

>    I think some key subsystems would benefit from per processor pools
>    (less contention) and the code can still be structured for short
>    holds.
> 

[..]
 
> On the other hand, I like the idea of:
> 
> 	1) per cpu pools for object FOO
> 
> 	2) global secondard pool for object class BAR which
> 	   FOO is a member of
> 
> and thus the strategy becomes:
> 
> 	alloc_a_FOO()
> 	{
> 	again:
> 		if(FOO_free_list[smp_processor_id()]) {
> 			/* Go like smoke */
> 		} else {
> 			lock_global_pool();
> 			get_some_for_me();
> 			if(others_running_low)
> 				feed_them_too();
> 			unlock_global_pool();
> 			goto again;
> 		}
> 	}
> 
> 	free_a_FOO()
> 	{
> 		if(my_pool_only_moderately_full) {
> 			give_it_to_me();
> 		} else {
> 			feed_someone_else_who_needs_it();
> 		}
> 	}
> 
> The idea is that when you need to enter a contension situation, you
> make the most of it by working towards keeping others from running
> into the same situation.
> 
> Another modification to the idea on the freeing side is to always free
> this FOO to the local per-cpu queue, and if other cpus could starve
> soon you give some of your oldest FOO's to them, the encourage cache
> locality.
> 
> Just some ideas.  These are some of ideas I was toying about with
> Larry McVoy when we were discussing with each other how N processors
> could go full throttle on an N high speed interface machine with near
> zero contention.

I like it a lot.  You can use a simple policy mechanism such as high and
low watermarks for resource balancing.

Terry has talked about using a global pool for resource balancing on this
list, but it's nice to see in something C like. 

Regards,


Mike Hancock