From nobody Sun Jan 14 04:49:58 2024 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TCNB172lvz573Gt for ; Sun, 14 Jan 2024 04:50:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic310-22.consmr.mail.gq1.yahoo.com (sonic310-22.consmr.mail.gq1.yahoo.com [98.137.69.148]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4TCNB10DK5z4G74 for ; Sun, 14 Jan 2024 04:50:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=aAWDAflb; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.148 as permitted sender) smtp.mailfrom=marklmi@yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1705207814; bh=yMBHKiCwgUuTpDLFH2smQKvNo1/LSuFEQSzPIycrNWo=; h=From:Subject:Date:To:References:From:Subject:Reply-To; b=aAWDAflb01G4Zw2TKEieJyuvOu5YMBfYfNb8MyWur/i4KggIEMT+Y0NMKiMaPzvIxkXIHixkCRbAW9/XAC8tkJ9rm9ouQuE2/fh4orIoFa0dVL9o6HQ9B7RP7onI5LWwHQyxrbVMphi4m7TwAIYxn746n+XI7CrvQtyJrILr3L2g1zCqua7XZJ33VhcKlIB1njYquWU9L0ubR7qiZlilh6fCs3XYvdg1kUf1dh+nGjlD1+7Pam9p3cHmwV9b69AXWyNw7Xl7L2fn1yCA8lKX7XAxO66jCyq+LLAje781WaPQcYS19t9yEaEytWd2bE840pRlh8aq9NBptZiJHE8Yiw== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1705207814; bh=ROMP+UBWHcj+jfAgiKUDdCJsFw2bPFR0LfWXri97f9t=; h=X-Sonic-MF:From:Subject:Date:To:From:Subject; b=jVXIi5X7SYpO6v6h58X/nNQ5MqwUCxADhKBV0sJnPT+mbkgzSWBzjl9r3PopMaRRPVqkMzIb7o1cXIPcOga4eTD17Ouy6lFmtO9E3VNz8OdZ6m4c89HpUCJp4ehx1ud1VEUI81Zk1UI/9NTjuLjIw1RWb24lfd5ZgyQqIFWFJ3+uX08jSBjVEeX95/wGxB6aop19DDE0ytUaa5O/7SooHptiE+L/5g0pca3ocyhnVqIYuty+TEkR6pyg5XQgpxf+uRAksJGz1ZeHgkIix29av0BvmXnt9mg+D24n+YYu7mwIMfcFIc3pddOH3BiPMqNDGbnXKO/SC6KKmx5T1mDMew== X-YMail-OSG: 3IhyUi4VM1l82vgBVR5SlLRZuJ1.Whkr7jtAMmvSir3IvgTwh.iEuZxD6t4poYb emEDlo4mU6r2KhHNp280whmrWTIuqIIIYSr18XxkWpCpkGsin7MbcPDk8sWGctbegF_odw4cDg1x J4zvWe_fjcVqzpJQw.XRytufJGUE5SGVETkLmDtMMoTeLaoMgMFh3ryGEAfB4WMvlBBU8nJOmf14 OuR0H2ulFhk1oenFbOr0jnZ1xVtQZk0HNj.yY_rRoBSJjNLNKBwBgK0comtszN8tRgNKc_MMObZJ wKGtAPi.BoSKqpMbBaqxU5w1x5kKaIHirgpZ.3kSYjImIHsPX92Njh2VEWsHxktV9TkjPmeWlC2Z Ftj1ZB1htLS1dTguehJ_.yADhY0j4W4CP8gfTJsEzhKHwvDmqEhp5YgXgNMsV9DZwWRALJ9BytE2 YZXqSPZY30MXGhSLEUnOpDHVxRVbUL3SBgKFHOr4LFuV5d1fX.oaLJTzhE5R3S7hDLYA46jkq362 z6ZMoSJKBAp6QqY5sz_MD3UgBQrXyMxSy4VaxuGZAGtpaB0x1r6jCkVG8qkRyi5dogs9DdYhkrX0 3LtrZd2VO9H5SR_Fbu7FIrv9bMxnELnZ0NiMNpq8RH7i_mX0NyDrVWav6Gyjz0UEGYmsAMpRZiD. 5v6hQfQNPwD3Z1KKq5c3Ox8Xd0km52AwkkjuIYM1qQXTruGHWd23koYwyXcf0.u4FOOLXQ6kwaPT pXjW5694VHwnmexW2mfY48ie0ZCzvn5YfMcTCkLF1IRX1fCwT82t0cYfRTejbdQicPkKtUtAHdDP kJk8MslUAkmP_WR62pRsN.45ISh7EQB8CCKm1b3brONgRGcc5mniZfMqtQAI5kwK2Lw_tUfgxD3d yf.LP1_4i_X2v2ECgiTFIQCkJjZAjNnAEbjRGiHZGQBLntrtbyADum2wzOeC1qLlNFRNPoeSGWey Yg0.uTZE9uXTqq_uqBgNT9WfmjqQKnPtGqmSYz.mjXhdQnYEBcAoX9OKho3iVggdFOSpbk7p4uDM _ReR8nefnQCoaIE9Cb8gpXFgu2pKlEnW84IqnmD9HUMsiVLUe8bcfOYNrscs7mW86SwYCF5ctUH2 BeyZeVhGnOZqix7q60R3Oq.4d_H_BwmzVKx_9Ogrxdd4NJ690w2q2Tf5pQchKGp9p7SO.1dYkSiH ITSRZ5x8rQFYmMF0bz9QigQYJ65YlT3jNzIYfTrTaMtvhBt6kCsGvZZiVzTXh1mh2ewVoshpyX_n lyJABkn2CUN__UoH5hf.qFxy_xcWBW7p_.tKTmrlxI546qifA6ASN6pQH8Np0fTMbm1Xe.r7hTFE rvOUsdmIJACGmcR_5_LZXCmNspaPapF6pdfuUEjMHmA.CAMDVyBo44brzrKCCMA0z1qAg2jw.BW0 1VtxgTpZLw2WFWWjcVion2DHnWXIVyD5tg4SVDHI_Mgn4ZUre74xuQvvvGd2B4sO.RUhIfiraDjd yEcshHw88bWGKX3HLIYu9DJYB73uFrgnN5RRff.a4cgiJrfhd6wQXFq5Aq2uBJiVS50pEQv1UpA_ 31gWSfUYgmaqiXBnSmuc0sv4US6Gze30QeyCSOrYsjRK58C7UXb9h5eq4AKe0AbYRBOvNI7EkFoR y_McYjwR0RpwUe8BY6G3RGnq1rk3xwnjDuFWYaBeQTpzPX5RVr0J8PuW.Lr8y3SIN1.o5k_gRIEv QR0HKPuBUG6xeB.SZEYBFHKHgCFb7ezaK49ukiWy_zJmrxhQSGNs3lax601UZ8PANhYChIQ4ECAO 9cJeJnBJVVIrXPjytEPvacLnE4LSjfwaGyUafGcQEC5mYo.QyLug0SDOsuuniu6_B977PZLwgbUH r37ARsrZY1tWiiQ0FjhO6Ejew4C0XNSZLs1fcUxlLLgdqQT.Ywhi1MMzb6_CQ3hVaBX7ZvQHR2eU Rw.Y1T1SaTEeFtpG4dIzft42U5w_k23evZ883yrRFfD.X4hCH.e9D8ZDdm2OR74sLD4YIv8D3ZMC XxYFDWmsHhgnh56q7GanboQ7nD2WyiAppDNL3micExdlvMTYeW2LCPbXVe1LDTlUDAcSP2VnRrn7 dOj6YVhLzDXboVWx0xS3HemH5rYBZANzt6CsbhS9KL7txXva85BnjaJxDtAkHBAHVUdLPJNYrfgP D72GweRWq_Ls9Rii5DuE8XsxP7ZzPbiU7EJMXgVZxz6YpLLWNRurta5jG9q31tKy6_2cyp6O18OW sUjPAM.261WPz91x3mVFc9YG_LcK1kt8czIBVx.n8fiTwnU9loG0hAil5qv138o746PaGap5u3.0 8 X-Sonic-MF: X-Sonic-ID: 9d5d579b-09f1-47db-9454-1b0074e3c990 Received: from sonic.gate.mail.ne1.yahoo.com by sonic310.consmr.mail.gq1.yahoo.com with HTTP; Sun, 14 Jan 2024 04:50:14 +0000 Received: by hermes--production-gq1-78d49cd6df-4xktb (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 382a3257c54ea491d7e296657689d9c3; Sun, 14 Jan 2024 04:50:09 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.300.61.1.2\)) Subject: RE: poudriere: swap_pager: out of swap space Message-Id: <60C36CF0-43AC-4ACA-B6A5-6997F4425EC5@yahoo.com> Date: Sat, 13 Jan 2024 20:49:58 -0800 To: lexi@le-fay.org, Current FreeBSD X-Mailer: Apple Mail (2.3774.300.61.1.2) References: <60C36CF0-43AC-4ACA-B6A5-6997F4425EC5.ref@yahoo.com> X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.50 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; MIME_GOOD(-0.10)[text/plain]; RCPT_COUNT_TWO(0.00)[2]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; ARC_NA(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; FREEMAIL_FROM(0.00)[yahoo.com]; FROM_HAS_DN(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; MLMMJ_DEST(0.00)[freebsd-current@freebsd.org]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.148:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.148:from] X-Rspamd-Queue-Id: 4TCNB10DK5z4G74 Lexi Winter wrote on Date: Thu, 11 Jan 2024 02:21:19 UTC : > i'm having a recurring problem with poudriere that i hope someone = might > have an idea about. >=20 > i'm building packages with poudriere on a system with 32GB memory, = with > tmpfs and md disabled in poudriere (so it's using ZFS only) and with = the > ZFS ARC limited to 8GB. >=20 > running poudriere produces many kernel log messages like this: >=20 > Jan 10 21:40:00 ilythia kernel: swap_pager: out of swap space > Jan 10 21:40:00 ilythia kernel: swp_pager_getswapspace(2): failed > Jan 10 22:41:55 ilythia kernel: swap_pager: out of swap space > Jan 10 22:41:55 ilythia kernel: swp_pager_getswapspace(21): failed > Jan 10 23:48:03 ilythia kernel: swap_pager: out of swap space > Jan 10 23:48:03 ilythia kernel: swp_pager_getswapspace(8): failed > Jan 11 00:05:00 ilythia kernel: swp_pager_getswapspace(1): failed > Jan 11 00:21:45 ilythia kernel: swp_pager_getswapspace(10): failed >=20 > this is despite the system having a large amount of "Inact" memory > according to top(1): >=20 > Mem: 3828M Active, 15G Inact, 2921M Laundry, 9263M Wired, 1559M Buf, = 892M Free > ARC: 3113M Total, 994M MFU, 884M MRU, 39M Anon, 49M Header, 1139M = Other > 1296M Compressed, 4130M Uncompressed, 3.19:1 Ratio > Swap: 2048M Total, 2048M Used, 8192B Free, 99% Inuse >=20 > from what i can tell, these swap errors don't cause any issues with = the > poudriere build, but they do seem to hinder interactive usage by = causing > long hangs. >=20 > does anyone have some idea what's going on here? i don't really > understand why the system has used 100% of available swap space when = it > has plenty of Inact memory it could free to fulfill requirements. You seem to be under the impression that "Inact" means "page is not dirty" and so can be freed without being written out to the swap space. Inact pages can be dirty and such pages can not be freed without being written out to the swap space first. If the swap space ends up filled, dirty pages that are not in active use stay or propogate into the Inact or Laundry states, accumulating there (for later potential use). ("Laundry" just means that those pages have been fully prepared to be written out to swap space. Dirty Inact pages have more work to be done before they can be written out --if there is room.) It appears that you had about 15 GiBytes of dirty pages that were not in active use but that would not fit in the 8192 Bytes of free Swap space. So they could not be freed without losing the "dirt" for which there was no other copy of around that could later be restored when needed. You do not report what sort of load average triple the context had, the number of parallel builders that were active, if you were using ALLOW_MAKE_JOBS, or if you had defined MAKE_JOBS_NUMBER or the like to limit the parallelism. I've certainly seen such memory use figures for allowing lots of parallel builders when ALLOW_MAKE_JOBS was also enabled and MAKE_JOBS_NUMBER (or the like) was not used to limit the number of make jobs each parallel builder is allowed to have going in parallel. There are also some port builds that create more parallel activity below each make-job, outside make's control. I've seen an 8 hardware thread system with the maximum observed for each of the 3 load average time frames being the likes of: 43.21, 40.44, 33.70 (from different times during the bulk run, not simulateously). =3D=3D=3D Mark Millard marklmi at yahoo.com