From owner-freebsd-hackers Fri Jan 19 21:42:30 2001 Delivered-To: freebsd-hackers@freebsd.org Received: from phnxpop5.phnx.uswest.net (phnxpop5.phnx.uswest.net [206.80.192.5]) by hub.freebsd.org (Postfix) with SMTP id BB5AD37B400 for ; Fri, 19 Jan 2001 21:42:12 -0800 (PST) Received: (qmail 94320 invoked by uid 0); 20 Jan 2001 05:42:12 -0000 Received: from ndslppp221.phnx.uswest.net (HELO pinyon.org) (63.224.136.221) by phnxpop5.phnx.uswest.net with SMTP; 20 Jan 2001 05:42:12 -0000 Received: from chomsky.Pinyon.ORG (localhost [127.0.0.1]) by pinyon.org (Postfix) with ESMTP id 1EBBE70; Fri, 19 Jan 2001 22:42:11 -0700 (MST) Date: Fri, 19 Jan 2001 22:42:11 -0700 Message-Id: <20010120054211.1EBBE70@pinyon.org> From: "Russell L. Carter" To: "Wes Peters" Cc: freebsd-hackers@FreeBSD.ORG X-Mailer: exmh version 2.1.1 10/15/1999 Subject: Re: Clustering FreeBSD In-Reply-To: Message from Wes Peters of "Thu, 18 Jan 2001 09:52:42 MST." <3A671F5A.6BB2D2B2@softweyr.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG %> No it would not! Back in '94 I ported dmake to FreeBSD %> and built just about every numerics package out there %> on a 4 CPU cluster. Worked fine, but not much in overall %> speedup, because... tadum! Where do you get the source %> files, and how do you get the objs back :-) Not low %> latency, eh? F-Enet then, G-Enet now :) % %You need a better file server. My previous employer, where the software %staff recompiles 3 million lines of code 20 or 30 times a day, employs %pmake and a farm of Sun Ultra-5 workstations to parallelize their makes. %It allows them to complete a build in an hour that would take a single %Ultra-5 almost 20 hours to complete, even with 3 or 4 builds running in %parallel. The network is 100BaseTX to the workstations and 1000BaseSX %to the (NFS) fileserver. Cool! I'd like to learn more. Then... can you elaborate on the build structure a bit? Is it a single large dir (surely not), or how do the dependencies work? For instance, with ACE/TAO (many hours to build when including orbsvcs) there's only a few large directories that can be parallelized over say 10 cpus by gmake, at least. The rest have ten files or less where each file takes maybe 45s to compile on a 1GHz processor. There are quite a few of these. And directories are compiled sequentially. %> Nowadays, you'd want to "globus ify" things, rather than %> use use PVM. %> %> But critically, speedup would only happen if jobs were %> allocated at a higher level than they are now. %> %> Now for building something like a full version of TAO, %> why that might work. But even then, a factor of 2x is %> unlikely until the dependencies are factored out at %> the directory level. % %See the paper "Recursive Make Considered Harmful." Make is an amazing %tool when used correctly. That's not the problem, unfortunately. I've never had a problem rebuilding dependencies unnecessarily, or any of those other problems described. Well precompiled headers would be really really cool. The problem, again, is that parallelism is limited by the directory structure, and the directory structure is entirely rational. Thanks! Russell To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message