From owner-freebsd-hackers@FreeBSD.ORG  Wed Oct 31 19:58:26 2012
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52])
 by hub.freebsd.org (Postfix) with ESMTP id 594C6498
 for <hackers@freebsd.org>; Wed, 31 Oct 2012 19:58:26 +0000 (UTC)
 (envelope-from bright@mu.org)
Received: from elvis.mu.org (elvis.mu.org [192.203.228.196])
 by mx1.freebsd.org (Postfix) with ESMTP id 3E9998FC0C
 for <hackers@freebsd.org>; Wed, 31 Oct 2012 19:58:25 +0000 (UTC)
Received: from Alfreds-MacBook-Pro-5.local
 (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218])
 by elvis.mu.org (Postfix) with ESMTPSA id 0615B1A3CB6
 for <hackers@freebsd.org>; Wed, 31 Oct 2012 12:58:19 -0700 (PDT)
Message-ID: <509182DA.8070303@mu.org>
Date: Wed, 31 Oct 2012 12:58:18 -0700
From: Alfred Perlstein <bright@mu.org>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7;
 rv:16.0) Gecko/20121026 Thunderbird/16.0.2
MIME-Version: 1.0
To: hackers@freebsd.org
Subject: make -jN buildworld on < 512MB ram
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 31 Oct 2012 19:58:26 -0000

It seems like the new compiler likes to get up to ~200+MB resident when 
building some basic things in our tree.

Unfortunately this causes smaller machines (VMs) to take days because of 
swap thrashing.

Doesn't our make(1) have some stuff to mitigate this?  I would expect it 
to be a bit smarter about detecting the number of swaps/pages/faults of 
its children and taking into account the machine's total ram before 
forking off new processes.  I know gmake has some algorithms, although 
last I checked they were very naive and didn't work well.

Any ideas?  I mean a really simple algorithm could be devised that would 
be better than what we appear to have (which is nothing).

Even if an algorithm can't be come up with, why not something just to 
throttle the max number of c++/g++ processes thrown out.  Maybe I'm 
missing a trick I can pull off with some make.conf knobs?

Idk, summer of code idea?  Anyone mentoring someone they want to have a 
look at this?

-Alfred