From owner-freebsd-arch@FreeBSD.ORG Tue Feb 26 02:11:20 2008 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3D15C16A402; Tue, 26 Feb 2008 02:11:20 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.netplex.net (mail.netplex.net [204.213.176.10]) by mx1.freebsd.org (Postfix) with ESMTP id 0678313C468; Tue, 26 Feb 2008 02:11:19 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.netplex.net (8.14.2/8.14.2/NETPLEX) with ESMTP id m1Q2B7Uh023172; Mon, 25 Feb 2008 21:11:07 -0500 (EST) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.netplex.net) X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-4.0 (mail.netplex.net [204.213.176.10]); Mon, 25 Feb 2008 21:11:07 -0500 (EST) Date: Mon, 25 Feb 2008 21:11:07 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Jeff Roberson In-Reply-To: <20080225160433.P920@desktop> Message-ID: References: <20080220175532.Q920@desktop> <20080220213253.A920@desktop> <20080221092011.J52922@fledge.watson.org> <20080222121253.N920@desktop> <20080222231245.GA28788@lor.one-eyed-alien.net> <20080222134923.M920@desktop> <20080223194047.GB38485@lor.one-eyed-alien.net> <20080223111659.K920@desktop> <20080223213507.GD39699@lor.one-eyed-alien.net> <20080224001902.J920@desktop> <20080225231747.GT99258@elvis.mu.org> <20080225143222.B920@desktop> <20080225160433.P920@desktop> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Brooks Davis , Andrew Gallatin , Alfred Perlstein , arch@freebsd.org, Robert Watson , David Xu Subject: Re: cpuset and affinity implementation X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2008 02:11:20 -0000 On Mon, 25 Feb 2008, Jeff Roberson wrote: > On Mon, 25 Feb 2008, Daniel Eischen wrote: > >> On Mon, 25 Feb 2008, Jeff Roberson wrote: >> >>> On Mon, 25 Feb 2008, Alfred Perlstein wrote: >>> >>>> Jeff, this is very cool. I do have one issue though: >>>> >>>> + * A thread may not be assigned to a a group seperate from other threads >>>> in >>>> + * the process. This is to remove ambiguity when the setid is queried >>>> with >>>> + * a pid argument. There is no other technical limitation. >>>> >>>> Am I understanding things correctly such that within a process there >>>> can only be one "set"? >>>> >>>> If so this restricts some of the benifits you get with sets and >>>> binding. >>>> >>>> An example would be some sort of system with multiple CPUs where some >>>> are assigned specifically for pseudo-realtime processing and others are >>>> for >>>> general control things such as cli, stats, etc. >>>> >>>> In our case we would like to be able to run some threads on specific >>>> cpu sets, and other threads to be run anywhere on the control CPUs. >>>> >>>> Can this be done with this API? >>> >>> Individual threads can be bound to any cpu or group of cpus within the >>> set. So if you just make a set that includes all cpus in the system you >>> can then bind your realtime threads to specific cpus and the other threads >>> to the remainder. You will have to specifically bind each thread however. >>> >>> The reason individual threads can't be assigned to groups is because >>> cpuset_getid() for a pid wouldn't make sense then and I expect >>> administrators to be mostly interested in managing groups of processes. >> >> If the administrator sets up a set of CPUs specifically for >> real-time and another set for non-real-time, you may want to >> bind some threads to the real-time set, and leave other threads >> unbound (or even bound to the non-real-time set). In this >> case, I think cpuset_getid() should either return the default >> cpuset of all cpus in the system, or the last cpuset to >> which the process was bound. >> >> But regardless, I think binding a thread to a different >> processor set should be allowed and should override its >> inherent binding of the process' processor set. >> Hmm, I guess in this case, a subsequent binding of the >> process to a processor set should probably override any >> per-thread bindings. > > I think we're getting into complex corner cases here which will only confuse > the api and administrators. I don't expect administrators will want to set > groups to individual threads. How would he even identify the individual > thread? And if he did, he could just as easily set masks on that thread > along with others in the process. > > I'm already a little nervous about how complicated this will be for > programmers. If we allowed each thread in a pid to be in its own set, we'd > have to make cpuset_getid() return an array of ids. I definitely don't want > to do that. Solaris does seem to allow this BTW. -- DE