From owner-freebsd-current@freebsd.org Sun Jan 26 17:45:50 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id AFE421F5711 for ; Sun, 26 Jan 2020 17:45:50 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 485Kzd5N9yz4Jhb for ; Sun, 26 Jan 2020 17:45:49 +0000 (UTC) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: from gndrsh.dnsmgr.net (localhost [127.0.0.1]) by gndrsh.dnsmgr.net (8.13.3/8.13.3) with ESMTP id 00QHjlTv044007; Sun, 26 Jan 2020 09:45:47 -0800 (PST) (envelope-from freebsd-rwg@gndrsh.dnsmgr.net) Received: (from freebsd-rwg@localhost) by gndrsh.dnsmgr.net (8.13.3/8.13.3/Submit) id 00QHjkuW044006; Sun, 26 Jan 2020 09:45:46 -0800 (PST) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <202001261745.00QHjkuW044006@gndrsh.dnsmgr.net> Subject: Re: After update to r357104 build of poudriere jail fails with 'out of swap space' In-Reply-To: <202001252359.00PNx6n1007151@slippy.cwsent.com> To: Cy Schubert Date: Sun, 26 Jan 2020 09:45:46 -0800 (PST) CC: sgk@troutmask.apl.washington.edu, freebsd-current@freebsd.org, Mark Millard , yasu@utahime.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-Rspamd-Queue-Id: 485Kzd5N9yz4Jhb X-Spamd-Bar: ++ Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of freebsd-rwg@gndrsh.dnsmgr.net has no SPF policy when checking 69.59.192.140) smtp.mailfrom=freebsd-rwg@gndrsh.dnsmgr.net X-Spamd-Result: default: False [2.56 / 15.00]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; DMARC_NA(0.00)[dnsmgr.net]; AUTH_NA(1.00)[]; RCPT_COUNT_FIVE(0.00)[5]; NEURAL_SPAM_MEDIUM(0.73)[0.731,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.90)[0.899,0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:13868, ipnet:69.59.192.0/19, country:US]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(0.03)[ip: (0.13), ipnet: 69.59.192.0/19(0.07), asn: 13868(0.02), country: US(-0.05)]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 26 Jan 2020 17:45:50 -0000 > In message <20200125233116.GA49916@troutmask.apl.washington.edu>, Steve > Kargl w > rites: > > On Sat, Jan 25, 2020 at 02:09:29PM -0800, Cy Schubert wrote: > > > On January 25, 2020 1:52:03 PM PST, Steve Kargl > on.edu> wrote: > > > >On Sat, Jan 25, 2020 at 01:41:16PM -0800, Cy Schubert wrote: > > > >> > > > >> It's not just poudeiere. Standard port builds of chromium, rust > > > >> and thunderbird also fail on my machines with less than 8 GB. > > > >> > > > > > > > >Interesting. I routinely build chromium, rust, firefox, > > > >llvm and few other resource-hunger ports on a i386-freebsd > > > >laptop with 3.4 GB available memory. This is done with > > > >chrome running with a few tabs swallowing a 1-1.5 GB of > > > >memory. No issues. > > > > > > Number of threads makes a difference too. How many core/threads does your l > > aptop have? > > > > 2 cores. > > This is why. > > > > > > Reducing number of concurrent threads allowed my builds to complete > > > on the 5 GB machine. My build machines have 4 cores, 1 thread per > > > core. Reducing concurrent threads circumvented the issue. > > > > I use portmaster, and AFIACT, it uses 'make -j 2' for the build. > > Laptop isn't doing too much, but an update and browsing. It does > > take a long time especially if building llvm is required. > > I use portmaster as well (for quick incidental builds). It uses > MAKE_JOBS_NUMBER=4 (which is equivalent to make -j 4). I suppose machines > with not enough memory to support their cores with certain builds might > have a better chance of having this problem. > > MAKE_JOBS_NUMBER_LIMIT to limit a 4 core machine with less than 2 GB per > core might be an option. Looking at it this way, instead of an extra 3 GB, > the extra 60% more memory in the other machine makes a big difference. A > rule of thumb would probably be, have ~ 2 GB RAM for every core or thread > when doing large parallel builds. Perhaps we need to redo some boot time calculations, for one the ZFS arch cache, IMHO, is just silly at a fixed percent of total memory. A high percentage at that. One idea based on what you just said might be: percore_memory_reserve = 2G (Your number, I personally would use 1G here) arc_max = MAX(memory size - (Cores * percore_memory_reserve), 512mb) I think that simple change would go a long ways to cutting down the number of OOM reports we see. ALSO IMHO there should be a way for sub systems to easily tell zfs they are memory pigs too and need to share the space. Ie, bhyve is horrible if you do not tune zfs arc based on how much memory your using up for VM's. Another formulation might be percore_memory_reserve = alpha * memory_zire / cores Alpha most likely falling in the 0.25 to 0.5 range, I think this one would have better scalability, would need to run some numbers. Probably needs to become non linear above some core count. > Cy Schubert -- Rod Grimes rgrimes@freebsd.org