From owner-freebsd-arch@FreeBSD.ORG Mon Feb 25 23:17:47 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 79E1016A407; Mon, 25 Feb 2008 23:17:47 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5DAE213C4EB; Mon, 25 Feb 2008 23:17:47 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: by elvis.mu.org (Postfix, from userid 1192) id 31F2B1A4D7E; Mon, 25 Feb 2008 15:17:47 -0800 (PST) Date: Mon, 25 Feb 2008 15:17:47 -0800 From: Alfred Perlstein To: Jeff Roberson Message-ID: <20080225231747.GT99258@elvis.mu.org> References: <20080220175532.Q920@desktop> <20080220213253.A920@desktop> <20080221092011.J52922@fledge.watson.org> <20080222121253.N920@desktop> <20080222231245.GA28788@lor.one-eyed-alien.net> <20080222134923.M920@desktop> <20080223194047.GB38485@lor.one-eyed-alien.net> <20080223111659.K920@desktop> <20080223213507.GD39699@lor.one-eyed-alien.net> <20080224001902.J920@desktop> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20080224001902.J920@desktop> User-Agent: Mutt/1.4.2.3i Cc: Brooks Davis , Andrew Gallatin , Daniel Eischen , arch@freebsd.org, Robert Watson , David Xu Subject: Re: cpuset and affinity implementation X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Feb 2008 23:17:47 -0000 * Jeff Roberson [080224 02:39] wrote: > Please see: > http://people.freebsd.org/~jeff/cpuset.diff > > This is unfortunately intertwined with ULE's new CPU selection algorithm > so that code is in the patch as well. Otherwise, this includes a simple, > ugly userland tool called cpuset and all of the kernel support required. > I have tested this by creating sets and subsets and modifying their cpu > masks under load. I'm able to dynamically reprovision without issue. > > This doesn't have support for jails but the infrastructure is there. It > also fails to modify sets if it would leave threads without a valid cpu > to run on. I have not implemented a force option but it will be trivial > to do so. The initial cpu set is also created before we know all_cpus so > it's faked up with all cpus set for now. > > I mostly want people to look at the interface in cpuset.h and make sure > they agree with it before I start polishing to commit. I'm fairly happy > with the way the syscall api looks now. The code itself ended up being > much more complicated than I'd hoped due to locking considerations. Try > not to look at cpuset_setproc() ;). Jeff, this is very cool. I do have one issue though: + * A thread may not be assigned to a a group seperate from other threads in + * the process. This is to remove ambiguity when the setid is queried with + * a pid argument. There is no other technical limitation. Am I understanding things correctly such that within a process there can only be one "set"? If so this restricts some of the benifits you get with sets and binding. An example would be some sort of system with multiple CPUs where some are assigned specifically for pseudo-realtime processing and others are for general control things such as cli, stats, etc. In our case we would like to be able to run some threads on specific cpu sets, and other threads to be run anywhere on the control CPUs. Can this be done with this API? -Alfred