From nobody Thu Jul 8 16:41:13 2021 X-Original-To: freebsd-ports@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0C7A81229162 for ; Thu, 8 Jul 2021 16:41:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-19.consmr.mail.gq1.yahoo.com (sonic314-19.consmr.mail.gq1.yahoo.com [98.137.69.82]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4GLMW65V8Bz3sVy for ; Thu, 8 Jul 2021 16:41:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1625762480; bh=+16efBj8mL3dnF4qTzG6/tL1HHJKgN6ChaAyihkTs88=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=Kstq0jzYiD4rP4Zj51DUGJYbJTkwSyVl9OPRY4OPE93jdQKgOoxBVK5SKhXxNl3pa/VA0+AmfCHD2QmFYCPkNQlhv9V3N5BcZFuwDoVrYsyyF/AnP8y/qx9Ahu5kMDebN5rb9uZ0X/OP0322jc6roTd/fPkCY9Cl8tnJwpQXz7snxA5kGM4kTE9oNoPnIpxd8JNNPjifndu3yDjZy1imuUmz8iYSlGyhOUc+sQC4m/TIupK4d8FuplkIN1jezAAFfOepRJGtB3XZ3yZShXMMnaxgHRVFy2uqDEUBLhUxvnZ8Mchfy2KLWDd+gc2+p8fxCH7fDpIqhWxC42hx/NzXdA== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1625762480; bh=kkdVMEPiW4c6hBcYgpA85jaE+sQpLDKCkPpJ+61Z4r6=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=G0JKsermHDrWpA2hTdCcfpzOby1Xo/FojDRu6LJf+/liNPiDIcHNuHB54l7iBLLgriF/jHI8UcLcQsYhIkE6/9roDVj28dUSjmHpLZSxlD9/WIL48iB3tmTi7x+awkrHynEEjKpiMrbd2dx3eKOHwBAaFenItre9+RR80XIcUHiU/Vkp+bST5hP1SxGaZWdqLnS2f08QVMsAPs6eJD4S0ma4DfRXC9F0LZ7uzjXF3c7O8pHEQ2II0C/vld3SED0RbaFIVPexYjtW/1sN16l4cFq3flKsAkdSY9PZevAk6MTUPK98S54AzjNcqBWnN8/XgiRN7j2gbBFPuvljyt6DHw== X-YMail-OSG: IsLhq_wVM1n9vyQYAoq1jltp.fq11f6bv3Z6Puln.gn.NXbbd54qd4RbUTtQajc nQuNkkJOvHdAehqgiytlxWBhq1r3162Ry0BlfeHOaRwfiGAt_UMjGOIGqMeOWV10mbLdCPMRC5ml aA61lCHFv2XVdBahEfTHdAcptyQOq2KF3v6bke1vkm4rO9BsanGb67mdOJhh73AGaBipfwjgRz3L Ij6MB3ns33MRrqclHkP.jZMgO8eLyuAyEIEwvbA6t7NcU0QwRXOn.0i87ZoJAUhvpB3ItARQ_whp lloum74ey4WOy_SAqqqNK8wgHR9gMn_qXm.ASf.HlWYbgOEsD9MTEMw9L0hFR7P2MMHZBjxBc4m0 btMJdmwoJ3F8w83RNCXkgTiUoUjodFJx.zmYSj1N2KMUvYWI59cXZNt7J8N9IZtvHTj0mU2K4YKi 0ixzVZe3tidSgvYJIYniF.V.RhmrMSinj97aARf91hPOE4DFaHgMj.WDDJBtzR0MlDOBpElEuMfZ m2F1k2hU2eGDhuDfAcb4vouMhJS2E1T4I.yaaELTvYvtvCILtmsRuoXHVjLg8cG1B2Bu70Z3Ge82 uU_yQc.Va2Q901MbRBkHZRlb3P_jwe8efx3uhfyfxLzenJXG2NSOF5PAg7YDDo8XFtfqkz5XAcwL xWnI.YXEa3I4fKAIQhYZUoso_bgz0hsEXiWOyErz4G8XhBS1le_NXbJA7jSGDTUbKqCDQm.o2yl5 tEsuPYRyCkL7jtCXZIChUew7pMZ20sciaEqfIy_Yezj2XhlWjNIHsq4Btko.3tGTz4XUmcZNkHrU gIpJUC9mFOW1H3oYyUE6gaG3VXKEHFhYaJXF1CNrmq58w0eZJTyRHXFr5qD6UiJ4ELXWL1CqQpku nI83K.ghuXVqPViTztj27u.DhcRZLaFXX2pwPtu4u.Poa7.Q2Kj0iGfLegvuPARZBtIpBaKt4W0t aWMQS2oG1AcTyz4C5N8x8ELE.q7ur76UzgObuhrPA78ihsagzgKL4NMIibUE2ZVr19VIOx5_xU16 Ad_7ne5H1fp8OleqYGibkkM2yn.IHEvH3i.l8BNVZkgS1kzRZpvCvf.4Clbi0INGABrNGmCRcIbC 3BlJE1F13kG3ptFFgSg8tjjGOsTvr.XrVYTnLRC_SlVdRKJyFTq0Y5a55yScKxLTwXnq_PCipv7J 2nZDqRSwX102GQJ6VSgMuheNHlAQEnDNJJ9P3FGEZJEN.I7dXVhdMt2ZAJCrb5tp.bMyhtVCyLYm MUrB4e31i1q.YmGoB8N17EF3x5kXRf0IJpJAJgLAkvnF9O48lgg2oGsLqvS5aF69oqrd2gt33hmX 5Y_XXPwWsTkrDjDETD3jg1KF3t5btoms7nfP81uCu6zs7pnHA1pQDUQ_ILmgHqreLVrh7UhKZKgc NDLYrKqSNMziIXOpcjn.X9WzPUHeSXzZIoVsB9irWAkoi8Tr1uUWfEqmrgojo5SIBdpoT63c8vmJ .BmwocxqvIufRtmNAUF3SB5QzjftPrGGulEaF_EJITsf3v7_xz_FYlsYD2h_uEJDCqyBMZ_QGN3p xvsG4yC1MV6JHLw612Jvd23ym9x95cdOkVZ4QkZJXURSmaJxQbYGdo99RW1rrNOV4i0Ck9sRat8f 7YZoV239oTQsGXq3WMpDb.9J0JEAEvW2GZRnMXgFrXNz8Hw.NAZUIwSh9gkeBBpqIs0mBaM_B0By po1Tj5rvooLwIBAV7RU4r92qGEjsjMOQfMTVJhw3mLKLbWKaTqad0iS4y6p_faaL8af5RgyjPKxZ bH3Q7XfVE9Q60x0Mk4EcVBlfQZxaCZe86IX_ZbsR0Ednpl.3OjG8n5fIFkKSAwim9PN0xPfBnT7z rx.dYnypRbQ9AdSlx5INqQvwo7dzs4pA_6VY7yvA1v2Btz.wY2SfO6gGhC4VlFw_F0Hjd8OZjiVD EMdmKrdamMPxr.Nj3Kg7Xn.FS7UWo5.Zw.ouzTYPSh7NRbWj6aL67VM3a4uEs8CzGnt14iUvwHJ8 xs4ZAOEkbg9r2GigOIVDE1pYBbK0XkQQfrc_ClYvJM0E4Zbc9D.lyKGe.KKpIPBfGa_LrS030j4_ LTWgJwx_uTZb576jVgh3iD8ojU_LkV7.kmyvh9xjY5BJbTqyCJthdmQ8TJf9erRFzzzdSilwrVTL ez9qph93WynE.gOHu32XpHaEJaF67cKraEuH5KuY1srPQeP4XU.mQt5pyh9rcShHipV0lxzxKKJA VCbgyTfgLteupGW9.OZ5eWUhG1D5EiuZm3QqCE29eLxt7M0ZssaSjCWsYVKc0PW7Zhr0Xg4NfASV kFLefkblAYbiDqZhNjzoLwjeCPYjVJr7X39Fi3UcPhl7TsjQesWqVPV5KIyolrZMlobARqTOU94A 7btae1FjT1VWr3y4k2aYyWtA9FHnjA7_NTSpi6l.KdteP.Pr9Z4MoarmLgzBZMMCF44A7oDq6xSr eHxNbwsW82aLcNTgHNDLkeYddT9X3PvOUvT0TiybJ.ypbCQw0zPRaT.Jdso.PUhB3O8VR3jNSxsW H65NIGCwEmH8- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Thu, 8 Jul 2021 16:41:20 +0000 Received: by kubenode552.mail-prod1.omega.bf1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID 8fb3070d3cf35945c0c00c484ee80073; Thu, 08 Jul 2021 16:41:15 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting software to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-ports List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-ports@freebsd.org X-BeenThere: freebsd-ports@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.100.0.2.22\)) Subject: Re: Too many pythons in poudriere In-Reply-To: <20210708154558.GB60914@www.zefox.net> Date: Thu, 8 Jul 2021 09:41:13 -0700 Cc: freebsd-ports@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: References: <044A7E63-2734-41F4-A1A2-AE5096C6A62C@yahoo.com> <20210708154558.GB60914@www.zefox.net> To: bob prohaska X-Mailer: Apple Mail (2.3654.100.0.2.22) X-Rspamd-Queue-Id: 4GLMW65V8Bz3sVy X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[] Reply-To: marklmi@yahoo.com From: Mark Millard via freebsd-ports X-Original-From: Mark Millard X-ThisMailContainsUnwantedMimeParts: N On 2021-Jul-8, at 08:45, bob prohaska wrote: > Even with -J1 and no ALLOW_MAKE_JOBS I'm still > seeing five pythons occupying at least 3 GB on > the loose. Actually I just looked and saw: Swapinfo 7.36% (Unlike the 83% or so I saw somewhat around 3 hours ago.) Load Averages (220%) 2.20 2.18 1.76 Elapsed 12:54:56 I do not see a swaplog in http://www.zefox.org/~bob/swaplogs/ to look at. So I can not see how much the peak swap space usage was so far (approximately). > I'm fairly sure this didn't happen > when using make by itself (IIRC it was -j2). > I also got rid of the mistaken directive in > poudriere.d/make.conf. When I look at http://www.zefox.org/~bob/poudriere.d/make.conf now I see: ALLOW_MAKE_JOBS=yes #MAKE_JOBS_NUMBER=2 #.if ${.CURDIR:M*www/chromium} #MAKE_JOBS_NUMBER_LIMIT=2 #.endif #.if ${.CURDIR:M*databases/sqlite3} #MAKE_JOBS_NUMBER_LIMIT=2 #.endif #.if ${.CURDIR:M*www/firefox} #MAKE_JOBS_NUMBER_LIMIT=2 #.endif which does not match your wording. But http://www.zefox.org/~bob/poudriere.conf does show ALLOW_MAKE_JOBS=yes commented out: # By default MAKE_JOBS is disabled to allow only one process per cpu # Use the following to allow it anyway #ALLOW_MAKE_JOBS=yes #MAKE_JOBS_NUMBER=2 # List of packages that will always be allowed to use MAKE_JOBS # regardless of ALLOW_MAKE_JOBS. This is useful for allowing ports # which holdup the rest of the queue to build more quickly. #ALLOW_MAKE_JOBS_PACKAGES="pkg ccache py*" (The interface for seeing the files does not show timestamps so I can not tell when updates were done.) (This is the wrong place for a MAKE_JOBS_NUMBER= use but it is commented out.) To see what is getting CPU time that leads to the load averages being around 2 might take using something like top sorted by cpu time and watching for a while. > There is a > #MAX_MEMORY=8 > in poudriere.conf, presumably GB. Documented as GiB: # How much memory to limit jail processes to for *each builder* # in GiB (default: none) #MAX_MEMORY=8 Per builder, not per-make-process. Within a builder each make-process shares that size space with the others. > That > looks like a good knob to play with. Would > setting it to something like 3 or 4 help? If the memory use exceeds what you set, the builder process is likely killed. It looks to be a way to have one or both of the following true: A) Stop the builder somewhat before the whole system runs out of memory (and swap space). This is likely a better failure mode handling: it could avoid the system hanging up. B) Prevent interfering other processes outside the builder by being sure the system still has available memory for other activities. But when I looked at the scripts it turns into jexec (and jexecd) command line options but man jexec did not document the options that I could see. No found for "man jexecd". So I'm guessing. > RAM plus swap presently totals 4.6 GB. > It looks like the present try will run out of > swap eventually, but I'll let it go for now > to see if anything interesting happens. It may later fail again, but the swap space usage is not large as of when I looked. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)