From nobody Thu Jul 8 17:54:36 2021 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id CB4F38D2EF4 for ; Thu, 8 Jul 2021 17:54:35 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (www.zefox.net [50.1.20.27]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "www.zefox.com", Issuer "www.zefox.com" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLP7b3rc9z4WKM for ; Thu, 8 Jul 2021 17:54:35 +0000 (UTC) (envelope-from fbsd@www.zefox.net) Received: from www.zefox.net (localhost [127.0.0.1]) by www.zefox.net (8.16.1/8.15.2) with ESMTPS id 168HsbPs071391 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 8 Jul 2021 10:54:37 -0700 (PDT) (envelope-from fbsd@www.zefox.net) Received: (from fbsd@localhost) by www.zefox.net (8.16.1/8.15.2/Submit) id 168HsaRQ071390; Thu, 8 Jul 2021 10:54:36 -0700 (PDT) (envelope-from fbsd) Date: Thu, 8 Jul 2021 10:54:36 -0700 From: bob prohaska To: Mark Millard Cc: freebsd-ports@freebsd.org Subject: Re: Too many pythons in poudriere Message-ID: <20210708175436.GA70414@www.zefox.net> References: <044A7E63-2734-41F4-A1A2-AE5096C6A62C@yahoo.com> <20210708154558.GB60914@www.zefox.net> List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Rspamd-Queue-Id: 4GLP7b3rc9z4WKM X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] X-ThisMailContainsUnwantedMimeParts: N On Thu, Jul 08, 2021 at 09:41:13AM -0700, Mark Millard wrote: > > > On 2021-Jul-8, at 08:45, bob prohaska wrote: > > > Even with -J1 and no ALLOW_MAKE_JOBS I'm still > > seeing five pythons occupying at least 3 GB on > > the loose. > > Actually I just looked and saw: > > Swapinfo 7.36% > > (Unlike the 83% or so I saw somewhat around 3 hours ago.) > > Load Averages (220%) 2.20 2.18 1.76 > > Elapsed 12:54:56 > > I do not see a swaplog in http://www.zefox.org/~bob/swaplogs/ > to look at. So I can not see how much the peak swap space > usage was so far (approximately). Started a new swaplog: http://www.zefox.org/~bob/swaplogs/202107182930.log It came within a whisker of running out of swap, then abruptly the python threads vanished and the build seems to be proceeding. I'm curious if this was blind luck, or some adaptive behavior by poudrirere. One other oddity: occasionally one see in top a PID using more than 100% WCPU. Is one thread occupying two cores? > > > I'm fairly sure this didn't happen > > when using make by itself (IIRC it was -j2). > > I also got rid of the mistaken directive in > > poudriere.d/make.conf. > > When I look at http://www.zefox.org/~bob/poudriere.d/make.conf > now I see: > > ALLOW_MAKE_JOBS=yes > #MAKE_JOBS_NUMBER=2 > #.if ${.CURDIR:M*www/chromium} > #MAKE_JOBS_NUMBER_LIMIT=2 > #.endif > #.if ${.CURDIR:M*databases/sqlite3} > #MAKE_JOBS_NUMBER_LIMIT=2 > #.endif > #.if ${.CURDIR:M*www/firefox} > #MAKE_JOBS_NUMBER_LIMIT=2 > #.endif > > which does not match your wording. > Thank you for catching my error. _now_ it's fixed. [snip] > To see what is getting CPU time that leads to > the load averages being around 2 might take > using something like top sorted by cpu time > and watching for a while. > > > There is a > > #MAX_MEMORY=8 > > in poudriere.conf, presumably GB. > > Documented as GiB: > > # How much memory to limit jail processes to for *each builder* > # in GiB (default: none) > #MAX_MEMORY=8 > > Per builder, not per-make-process. > Within a builder each make-process shares > that size space with the others. > > > That > > looks like a good knob to play with. Would > > setting it to something like 3 or 4 help? > > If the memory use exceeds what you set, the builder > process is likely killed. [snip] I was hopeful it might inhibit starting new PIDs when memory/swap is below some threshold. Guess not. Thanks for writing! bob prohaska