From owner-freebsd-current@FreeBSD.ORG Tue Feb 26 01:10:26 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66B9E16A400; Tue, 26 Feb 2008 01:10:26 +0000 (UTC) (envelope-from jroberson@chesapeake.net) Received: from webaccess-cl.virtdom.com (webaccess-cl.virtdom.com [216.240.101.25]) by mx1.freebsd.org (Postfix) with ESMTP id 1549913C44B; Tue, 26 Feb 2008 01:10:26 +0000 (UTC) (envelope-from jroberson@chesapeake.net) Received: from [192.168.1.107] (cpe-24-94-75-93.hawaii.res.rr.com [24.94.75.93]) (authenticated bits=0) by webaccess-cl.virtdom.com (8.13.6/8.13.6) with ESMTP id m1Q1AMdb094101; Mon, 25 Feb 2008 20:10:25 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Mon, 25 Feb 2008 15:12:00 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Brooks Davis In-Reply-To: <20080225233315.GA59569@lor.one-eyed-alien.net> Message-ID: <20080225150551.O920@desktop> References: <20080224173229.I920@desktop> <20080225233315.GA59569@lor.one-eyed-alien.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: current@freebsd.org Subject: Re: cpuset and affinity implementation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2008 01:10:26 -0000 On Mon, 25 Feb 2008, Brooks Davis wrote: > On Sun, Feb 24, 2008 at 05:38:37PM -1000, Jeff Roberson wrote: >> Hello, >> >> I have implemented a new api similar to processors sets on solaris. This >> allows you to assign processes to sets of cpus and dynamically change those >> sets. This is useful for provisioning purposes to add and remove cpu >> resources for a particular process or group of processes. This new >> facility also supports binding secific threads to specific cpus which some >> applications may want to do. At some point in the future this will be >> integrated with jail so you can restrict the cpus any jail is allowed to >> use. >> >> This api should not be considered final and the 'cpuset' tool is quite >> rough. This also only works with ULE and is unfortunately intertwined with >> a big ULE patch I've been working on. The set management code is generic >> but 4BSD doesn't contain the hooks to actually constrain threads. > > I took a look at the patch this morning. The API looks like it's > capable of doing what I need, at least at a first pass. It looks like I > should be able to implement the semantics currently employed by the Sun > Grid Engine scheduler on Irix systems. > > The one thing I noticed that I found worrying was the recursion in > cpuset_(test)update(). It wasn't immediately clear to me there there > is anything to would prevent an arbitrarily deep hierarchy from being > created and blowing the kernel stack. I'm I missing something? Yes, presently it can never be more than 3 levels deep. Once we have jails the max would be 6 levels, unless you can make jails within jails. There is presently now way for the user to create a cpuset that is a subset of another set. So the three cpu sets are: 1) Root set - immutable, all cpus, may be root of jail in which case root outside of the jail can change the set. 2) cpuset - the set this process is a member of. 3) mask - the anonymous set that is applied to an individual thread. Did you look at the userland tool at all? I think this needs the most improvement. I basically just made something that would allow me to pass every possible parameter to the api. Not exactly engineered for usability. > > -- Brooks >