From owner-freebsd-current@FreeBSD.ORG Tue Feb 26 01:50:35 2008 Return-Path: Delivered-To: current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F11FB16A401; Tue, 26 Feb 2008 01:50:34 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id 833CE13C461; Tue, 26 Feb 2008 01:50:34 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.1/8.13.8) with ESMTP id m1Q1oPW6063951; Mon, 25 Feb 2008 19:50:25 -0600 (CST) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.1/8.13.8/Submit) id m1Q1oPSE063950; Mon, 25 Feb 2008 19:50:25 -0600 (CST) (envelope-from brooks) Date: Mon, 25 Feb 2008 19:50:25 -0600 From: Brooks Davis To: Jeff Roberson Message-ID: <20080226015025.GA63847@lor.one-eyed-alien.net> References: <20080224173229.I920@desktop> <20080225233315.GA59569@lor.one-eyed-alien.net> <20080225150551.O920@desktop> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fdj2RfSjLxBAspz7" Content-Disposition: inline In-Reply-To: <20080225150551.O920@desktop> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Mon, 25 Feb 2008 19:50:25 -0600 (CST) Cc: Brooks Davis , current@freebsd.org Subject: Re: cpuset and affinity implementation X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Feb 2008 01:50:35 -0000 --fdj2RfSjLxBAspz7 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Mon, Feb 25, 2008 at 03:12:00PM -1000, Jeff Roberson wrote: >=20 > On Mon, 25 Feb 2008, Brooks Davis wrote: >=20 >> On Sun, Feb 24, 2008 at 05:38:37PM -1000, Jeff Roberson wrote: >>> Hello, >>>=20 >>> I have implemented a new api similar to processors sets on solaris. Th= is >>> allows you to assign processes to sets of cpus and dynamically change= =20 >>> those >>> sets. This is useful for provisioning purposes to add and remove cpu >>> resources for a particular process or group of processes. This new >>> facility also supports binding secific threads to specific cpus which= =20 >>> some >>> applications may want to do. At some point in the future this will be >>> integrated with jail so you can restrict the cpus any jail is allowed to >>> use. >>>=20 >>> This api should not be considered final and the 'cpuset' tool is quite >>> rough. This also only works with ULE and is unfortunately intertwined= =20 >>> with >>> a big ULE patch I've been working on. The set management code is gener= ic >>> but 4BSD doesn't contain the hooks to actually constrain threads. >>=20 >> I took a look at the patch this morning. The API looks like it's >> capable of doing what I need, at least at a first pass. It looks like I >> should be able to implement the semantics currently employed by the Sun >> Grid Engine scheduler on Irix systems. >>=20 >> The one thing I noticed that I found worrying was the recursion in >> cpuset_(test)update(). It wasn't immediately clear to me there there >> is anything to would prevent an arbitrarily deep hierarchy from being >> created and blowing the kernel stack. I'm I missing something? >=20 > Yes, presently it can never be more than 3 levels deep. Once we have jai= ls=20 > the max would be 6 levels, unless you can make jails within jails. >=20 > There is presently now way for the user to create a cpuset that is a subs= et=20 > of another set. So the three cpu sets are: >=20 > 1) Root set - immutable, all cpus, may be root of jail in which case roo= t=20 > outside of the jail can change the set. > 2) cpuset - the set this process is a member of. > 3) mask - the anonymous set that is applied to an individual thread. OK. That makes sense. It would be useful from my perspective if creating a root set (or an otherwise inescapable set) was not explicitly tied to jails. I could see doing this either by extending the syscalls or by introducing a more fine grained light weight virtualization as was discussed in Milan. This would probably want to be a privileged operation regardless. > Did you look at the userland tool at all? I think this needs the most=20 > improvement. I basically just made something that would allow me to pass= =20 > every possible parameter to the api. Not exactly engineered for usabilit= y. I glanced at it, but haven't really thought about what should/shouldn't be there much. I'll try to do that in the next day or so. -- Brooks --fdj2RfSjLxBAspz7 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (FreeBSD) iD8DBQFHw3BgXY6L6fI4GtQRAok6AKCQclvEFaJKg0kd4t+GSXqnootnBACdH01o AR7AOMrCynBkEviJleiqRlc= =qeHr -----END PGP SIGNATURE----- --fdj2RfSjLxBAspz7--