From owner-freebsd-hackers@FreeBSD.ORG Thu Nov 1 16:16:15 2012 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 495DDD08 for ; Thu, 1 Nov 2012 16:16:15 +0000 (UTC) (envelope-from kaduk@mit.edu) Received: from dmz-mailsec-scanner-6.mit.edu (DMZ-MAILSEC-SCANNER-6.MIT.EDU [18.7.68.35]) by mx1.freebsd.org (Postfix) with ESMTP id D744C8FC16 for ; Thu, 1 Nov 2012 16:16:14 +0000 (UTC) X-AuditID: 12074423-b7fab6d0000008f9-a1-5092a0484d02 Received: from mailhub-auth-1.mit.edu ( [18.9.21.35]) by dmz-mailsec-scanner-6.mit.edu (Symantec Messaging Gateway) with SMTP id 87.5D.02297.840A2905; Thu, 1 Nov 2012 12:16:08 -0400 (EDT) Received: from outgoing.mit.edu (OUTGOING-AUTH.MIT.EDU [18.7.22.103]) by mailhub-auth-1.mit.edu (8.13.8/8.9.2) with ESMTP id qA1GG7E6008333; Thu, 1 Nov 2012 12:16:08 -0400 Received: from multics.mit.edu (SYSTEM-LOW-SIPB.MIT.EDU [18.187.2.37]) (authenticated bits=56) (User authenticated as kaduk@ATHENA.MIT.EDU) by outgoing.mit.edu (8.13.6/8.12.4) with ESMTP id qA1GG48c013025 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Thu, 1 Nov 2012 12:16:05 -0400 (EDT) Received: (from kaduk@localhost) by multics.mit.edu (8.12.9.20060308) id qA1GG3dC003711; Thu, 1 Nov 2012 12:16:03 -0400 (EDT) Date: Thu, 1 Nov 2012 12:16:03 -0400 (EDT) From: Benjamin Kaduk To: Peter Jeremy Subject: Re: make -jN buildworld on < 512MB ram In-Reply-To: <20121031204152.GK3309@server.rulingia.com> Message-ID: References: <509182DA.8070303@mu.org> <20121031204152.GK3309@server.rulingia.com> User-Agent: Alpine 1.10 (GSO 962 2008-03-14) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrKIsWRmVeSWpSXmKPExsUixCmqrOuxYFKAwaxNYhYbFhRaPL37ndWB yWPGp/ksHotu3mULYIrisklJzcksSy3St0vgynhy5QBrQZNAxYKpnewNjA94uhg5OSQETCTe bZvECmGLSVy4t56ti5GLQ0hgH6PEv7P3mEASQgLrGSW+PsyHSBxnkrgx6x5zFyMHkFMv8avL EKSGRUBL4sTbJ8wgNpuAisTMNxvZQGwRAVWJQ6svgy1gFhCXWHivlwXEFhbQl9iwahYryBhO AQuJR5ucQcK8Ag4Sz2Z1M0OsDZSYtH4jWKuogI7E6v1TWCBqBCVOznzCAjHSUuLcn+tsExgF ZyFJzUKSWsDItIpRNiW3Sjc3MTOnODVZtzg5MS8vtUjXTC83s0QvNaV0EyMoRNldlHcw/jmo dIhRgINRiYfXoHRigBBrYllxZe4hRkkOJiVR3q9TJwUI8SXlp1RmJBZnxBeV5qQWH2KU4GBW EuF93A1UzpuSWFmVWpQPk5LmYFES572WctNfSCA9sSQ1OzW1ILUIJivDwaEkwVs+H2ioYFFq empFWmZOCUKaiYMTZDgP0HAdkBre4oLE3OLMdIj8KUZdjl998x4yCrHk5eelSonz1oMUCYAU ZZTmwc2BpZZXjOJAbwnzuoFU8QDTEtykV0BLmICWqFiAfFBckoiQkmpgnMHLNNFhg1Grf2bF lfBF9suZ2yYFq4TMmXbnVtZZjS9muxJ2pcdw3t28w3uj9ol4gzOPmzlnzeGcP2GCqsYEJvPX dr+4y2IWcgYJMpzas/7ex7cBgUHCr1d+XrAkuZRD7OC5C4GJv7oztkQxze3PrBfdkeX31vfZ 7KuOVRqH/2zLLpor++/sdyWW4oxEQy3mouJEAHAEWFEIAwAA Cc: hackers@freebsd.org X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Nov 2012 16:16:15 -0000 On Thu, 1 Nov 2012, Peter Jeremy wrote: > On 2012-Oct-31 12:58:18 -0700, Alfred Perlstein wrote: >> It seems like the new compiler likes to get up to ~200+MB resident when >> building some basic things in our tree. > > The killer I found was the ctfmerge(1) on the kernel - which exceeds > ~400MB on i386. Under low RAM, that fails _without_ reporting any > errors back to make(1), resulting in a corrupt new kernel (it booted > but had virtually no devices so it couldn't find root). > >> Doesn't our make(1) have some stuff to mitigate this? I would expect it >> to be a bit smarter about detecting the number of swaps/pages/faults of >> its children and taking into account the machine's total ram before >> forking off new processes. > > The difficulty I see is that the make process can't tell anything > about the memory requirements of the pipeline it is about to spawn. > As a rule of thumb, C++ needs more memory than C but that depends > on what is being compiled - I have a machine-generated C program that > makes gcc bloat to ~12GB. > >> Any ideas? I mean a really simple algorithm could be devised that would >> be better than what we appear to have (which is nothing). > > If you can afford to waste CPU, one approach would be for make(1) to > setrlimit(2) child processes and if the child dies, it retries that > child by itself - but that will generate unnecessary retries. > > Another, more involved, approach would be for the scheduler to manage > groups of processes - if a group of processes is causing memory > pressure as a whole then the scheduler just stops scheduling some of > them until the pressure reduces (effectively swap them out). (Yes, > that's vague and lots of hand-waving that might not be realisable). Starts to sound similar to parts of linux cgroups. Which, incidentally, a friend of mine was recently complaining about not being able to run systemd in the linuxolator due to the lack of certain syscalls pertaining to cgroups... -Ben Kaduk