From owner-freebsd-hackers@FreeBSD.ORG Thu Dec 14 18:55:57 2006 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C36A916A407 for ; Thu, 14 Dec 2006 18:55:57 +0000 (UTC) (envelope-from lidl@rox.fddx.com) Received: from rox.fddx.com (rox.pix.net [71.241.230.59]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D60A43EB8 for ; Thu, 14 Dec 2006 18:50:43 +0000 (GMT) (envelope-from lidl@rox.fddx.com) Received: from rox.fddx.com (localhost [127.0.0.1]) by rox.fddx.com (8.13.6/8.13.5) with ESMTP id kBEIq5M2010146 for ; Thu, 14 Dec 2006 13:52:05 -0500 (EST) Received: (from lidl@localhost) by rox.fddx.com (8.13.6/8.13.5/Submit) id kBEIq5Ou010145 for freebsd-hackers@freebsd.org; Thu, 14 Dec 2006 13:52:05 -0500 (EST) Date: Thu, 14 Dec 2006 13:52:05 -0500 From: "Kurt J. Lidl" To: freebsd-hackers@freebsd.org Message-ID: <20061214185203.GA9701@pix.net> Mail-Followup-To: freebsd-hackers@freebsd.org References: <20061214041952.GA7513@dragon.NUXI.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20061214041952.GA7513@dragon.NUXI.org> User-Agent: Mutt/1.5.12-2006-07-14 X-Scanned-By: milter-spiff/0.6.11 (localhost [127.0.0.1]); Thu, 14 Dec 2006 13:52:05 -0500 Received-SPF: Pass; receiver=localhost; client-ip=127.0.0.1; envelope-from= Received-SPF: Pass; receiver=localhost; client-ip=127.0.0.1; helo=rox.fddx.com X-Virus-Scanned: ClamAV version 0.88, clamav-milter version 0.87 on rox.fddx.com X-Virus-Status: Clean Subject: Re: [PATCH] "automated" make -j value X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Dec 2006 18:55:57 -0000 On Wed, Dec 13, 2006 at 08:19:52PM -0800, David O'Brien wrote: > With multi-socket systems becoming more prevalent, and the continued > increase in cores per processors, I thought it would be nice for > 'make -j' to gain some automation. > > Attached is a patch that makes "-j-" be the same as > "-j `sysctl -n kern.smp.cpus`" and "-j=" be twice that. > > comments? (redirected back to list) I think you can do it better than this, by just support setting the concurrancy level via reading a environmental variable. Like say "NPROC", which is what CrayOS used, and also appeared in the BSD/OS pmake varient several years ago. Then you can just do this: export NPROC=`/sbin/sysctl hw.ncpu | awk '{print $3*2}'` or this: export NPROC=`/sbin/sysctl hw.ncpu | awk '{print $3}'` in your shells .rc files. (Obviously, change the sysctl node as appropriate for your OS.) I found this really useful when compiling a large tree of sources, where some of the Makefiles didn't have their dependencies written correctly, sucht that a parallel make wouldn't work properly. It's easy to turn off, just by unsetting the environmental variable. It's also easy to iterate over a set of values to figure out which one will compile a tree the fastest. (FYI -- setting 3*hw.ncpu was optimal for BSD/OS.) If you hack on make to put in automagic around -j, you should add the environmental variable support too. It's actually more useful in a lot of cases. (Mostly cause you don't have to touch any Makefile to turn it on, it just works...) -Kurt