From nobody Tue May 17 21:10:06 2022 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id DBB181AE9BD8 for ; Tue, 17 May 2022 21:10:23 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4L2pg25X44z4d7c for ; Tue, 17 May 2022 21:10:22 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1652821815; bh=HyTxK26SL1H2o/R1INvY62GqxgMzyxnaPrUd+YV54XQ=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=IwDJir2mVs4zyGDTVGPl/oCWJlIlr0WDPBMkFwBosnS/THSY+zHLYXpA5kk5kYwCGMTlx5BJcM5eIS847tVsqStJVzhlr8Q/+7hEPFCOoL1gwwnydHsYlVku5vzlYuS9Q54akNbS/+w4N8F2/3hT71lZbXpJVkIWTftYm+/rZF7UsRk+p5dCvlijMduPeaZDCPlARjdcfAvbwZ8lCrN6MHE7PbV8xwanjfR0nHDQGqxWlO7HKnie6Q2Vi+XnVOCk+VRIjGeojjjPROdTqtFVxM3TZVUkULja/WOVsTYnjgEWDkbglShqm76wIqPWkUXxd8dmCVzRoHZ7zRWUR7aOjQ== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1652821815; bh=YNtPYl4ehTvczz12FCRN+y6X6pyTgssGrUb4IW6a+gi=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=ZKax2KUc7W+RYoGnK5PUsxYgx5/4imcq0erc0wiflzRWSyMqJTO+xyx8/c97FCZl26oW9d0+sGkZIF2NvssNlhthcJyRSw77Dsdlh0vpX1UnNfT+y14hORGCzzdIsd6ecgp5nqIqynr9PleQm4R2oq3DhXPmrxzDDoiMuZnSGM5E06+XpruQiSXtqBNgDw4+ki7lRV06IkLU8Xvr1drR2zYx82ybU0YFnQ3BvbivofR2cPgtS9Q+2KSqT4gmtxT1NzpT+HBOFpxU3qUMDBnwi+y7qJ4eTNJKLlXmxfgW+pw4sckxdI6r9lZHjc7BcdgOaSysuuRqMF01xW9sJnPqDw== X-YMail-OSG: gXGrx7cVM1m5ADX6.ldBjjTeIKaf9IUDtwUZhXu.MQX5O4HS7FKDwjF8Nr_YEeE jvAP4qEfHoVBH3nFqEKqY3O5mhkHORPdfcrqnfOuV_.IZehS2DK7ls7gQ7lFSqYGvd7BswmXAWqz 8huCzaVbjWGcAGW2qmuz0lmFOQCYXvnV2UkQrMLyiALE46OX22EUjpkMVa3ZyR.GTDS86yIKC_DV ZgkEFeluIbbDfndEGMzZlNzy1yVW2SI5RolsfBymK87SSybdypVkREWcfWmzYvxNpKmgH6IjQG7U rDd3EH6oY4RBbEpfsmBqVO7xp_af1HE7767JLg3YqquojI9WP3q9iz1JL.PY8zAsb71aH_I9icqH dH.VnTzZtPn_sd8tzZTuPCMb17Tn0uxPMfwTrrkU9sSpv9hwS1zh6AgYdfU7E5manY2zTfpJF8IC fviGr_yBQspbCZ5u1nySn19aO2StwzRtDfFJ_BJoTqCAXrLYRhc0CFAV_vBjkGWpsaauwXRej3KC 2HBseW8B6FyYJReuaL1KgrqhAH7K2GuBYwpjKpROPCf.pN..195wAPWgsseDlJeL3tBlIRPncqIc .3OKi8KvIW5iDIbNZ00aV7csUXDicAmzbmN63wu0oCgr016zEm4L5v4hrLo4bknsivLzaZD6_EY1 z97ZnLddpHtu1c0u4MxrAsrWnQeIymCSUF_Q1mXkkXcfausvUJaBNGKCHDabj4HlHP3fyNvG5GaY BSGVVas8QjA7s5IDkXfJa3M.4bpeXdEWjh1om1zRMdcJwzGeRuEs_uVWqRpZ3TXt_bsbebjYoHun tyg1a7yq9uYq0oq.jQnHZUt51C6B403.pivUFXPahhxIviVv6mK7IbFkW8UlRIq_Wf0B6WkvNRsV mUmi.itH_vy_FV0SeBx2a.x_u_Sa0fsgNiUcxmEJNXe90RItQxgbN9IKOlZn_G8fb9L6k_EgM22W oNUPa6_Vn1aSQGuanrIpL.QLec5krHUSY5EDNFlwlYj8Ycf_FHb8hfFo90J_mvVOiZJNA.Qi6eie ULP94Y4PwHk4hGK9fcMo6SUT783iYpRgboPtxT0XVL7SCetwfgULaiOxMeL90BD0PSpIYOwOXFc2 clB6NVwxmPT.qXFIUYnykxSntClXYR.jE3c1gyB0fvqf2k4MC1bMQKEggwFT6ZH_Dlw2oedZVqze udkuHSDMHcSyLHxqKBnObGROTPbHs2uwevCTB2aJqU3giguGgtg91w.lKAw_mBBRmDdW5n4hcoQG 7Wq4eJSdtPT_3oXJGk9YRqG5Q2schHnwX.ge6PKasaZyIz5_fHl6yikgcrEL9nRczfJtmUTdZCHL ytm5oE6Qx.1sX97.zAw5lpeJvDb9nM9te2sGYuOrB87VyIiMLmngpTipVaCJ3FoHl2m7OSaxfIr4 sF2oFMXV89EoWcqvU9ebHqCfOG.9.NteKHOvmwZlPAb195dWtcwbUWAxy78N.Qtakhi7bW3dAxuP CZv6NMeHRPamhIu_EfUwS6ZwoWQM8ujwTiip4OKP8e5WZHj.smDZAfy3N7dsJ5hNZ9V63BpB9kWW UrgVKsQuhsi8ad75t2i8kMihCMuEF23BLA9tIfbIAU0GtfeK_2uGc3v5sU61XiDsYuA.9utxWJtG EaDyOIXvSPK5BnXuXFSJMi5SMnJaY4efeLf._zXlftahc110R8ZLJx6NMy9v2sg8EGolcEssxEg2 KduUjiuIbkrNjx.bx1tkdeGHI2FOKy5nyri0Uz_zvSbDvdT9ov4gLCYEGf.bTbK0F4.uww6QUzmz N4DHal41x6d_0IYgLXH9dwPa4SoKi67JQBpAMQPeIkxRkVglO6A2S7UzaJ17yzxWH_s8GGEIPeIh aaJXGSZRXHJ3f8gN3oay.yPiv2Sqtwn8Tslw60lX4dsjoP3fFbBfFj6AiXVYc6PtBfzDmFXk1LpR mg6bpiIkcSGeMeraAKS9.xROmvSGKkGZo0WJhsLxAm4S7PMafcJPl2XqRnpVhdK3S2aeHAvPDjyE hzlUXy9RGYzA4u2eP0XLAzDPYRxC3N.scT6SOlD.k0_XtBs8MO1jXssIYiGQYZLRUM8T9ojv6d8M nLhvYvzOgYEUwvtuwI1Lb_o8ClNffV20GqVKT0sNHRllnyZWGoCUpmyvLV5Ikm6NAcD6638MelzJ 9zYireLHgLsvrl4vx.ePSJWFsPo51WkTZJR2Fwx0L7q3PnKy7.pm7XQd6xHoUitqmdDgbf7WeDxh 6Xe0RRQuf44sJlQ4aN.wjIIDCFgGlpa.MVtKhXyvKj.464QLiiSiAmd3bcRS5r9jUJb1z1HTrwAW tRsB07.VOaNN9uv0t_8vzFhCgzc.JWLhdgJnAi4EnmBNXsuZtMHn.YY3lXbgo X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Tue, 17 May 2022 21:10:15 +0000 Received: by hermes--canary-production-ne1-5bcb78874-tzl4b (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2c16a81d28186a600bac54bf2c89f351; Tue, 17 May 2022 21:10:11 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Can not build kernel on 1GB VM From: Mark Millard In-Reply-To: <2892AE2A-E453-460F-A243-0E98E5DF3C8C@yahoo.com> Date: Tue, 17 May 2022 14:10:06 -0700 Cc: FreeBSD Hackers Content-Transfer-Encoding: quoted-printable Message-Id: <84F2341A-C7AF-41F3-8A69-70FB9D7271CF@yahoo.com> References: <27171A11-13B1-48A8-AF46-605091E1093F.ref@yahoo.com> <27171A11-13B1-48A8-AF46-605091E1093F@yahoo.com> <20220516143753.GY72471@post.wayne47.com> <934C3159-4B1A-4A2F-9C21-93DC7CC90A72@yahoo.com> <20220517135616.GA72471@post.wayne47.com> <2892AE2A-E453-460F-A243-0E98E5DF3C8C@yahoo.com> To: Michael Wayne X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4L2pg25X44z4d7c X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=IwDJir2m; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; NEURAL_HAM_SHORT(-1.00)[-1.000]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; MLMMJ_DEST(0.00)[freebsd-hackers]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_TWO(0.00)[2]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim] X-ThisMailContainsUnwantedMimeParts: N On 2022-May-17, at 12:40, Mark Millard wrote: > On 2022-May-17, at 06:56, Michael Wayne wrote: >=20 >> On Mon, May 16, 2022 at 12:07:18PM -0700, Mark Millard wrote: >>> On 2022-May-16, at 07:37, Michael Wayne = wrote: >>>=20 >>>> More info. I am running UFS so the ZFS should not be an issue >>>>=20 >>>> % pstat -s >>>> Device 1K-blocks Used Avail Capacity >>>> /dev/md99 1048576 0 1048576 0% >>>=20 >>> That may well explain some (or all?) of what is going on: >>> file-system/vnode backed swap spaces are subject to deadlocks. >>=20 >>=20 >>> Which suggested patch(s)? Any patches for . . . >>=20 >> This one line change (patch reduced to only relevant info) that >> was posted earlier on the list. >> --- a/sys/vm/vm_pageout.c >> +++ b/sys/vm/vm_pageout.c >> @@ -1069,7 +1069,7 @@ vm_pageout_laundry_worker(void *arg) >> - if (target =3D=3D 0 && ndirty * isqrt(howmany(nfreed = + 1, >> + if (target =3D=3D 0 && ndirty * = isqrt(howmany(nfreed, >=20 > Why my brain was stuck on "patches for messaging better" > I do not know. Seems obvious now that you show it. > Sorry for the noise. >=20 >>> Can you switch to using a swap partition instead of >>> file-system/vnode backed swap space? >>=20 >> AFAICT, this would require a reinstall as there's no easy way to=20 >> shrink the existing image.=20 >=20 > You may ultimately have a requirement that the image > containing the UFS file system and swap space be no > more than some specific size), implying that the UFS > file system would need to shrink to make room. >=20 > But, for testing, could a copy of the image file for > the VM be made and then be grown larger and then a > freebsd-swap partition added in it for use as a swap > space? Then: switch to using the partition to see what > happens? At least we might learn if the issue by > itself is sufficient to avoid the problem you are > having. >=20 > 18.3 "Resizing and Growing Disks" in: >=20 > https://docs.freebsd.org/en/books/handbook/disks/ >=20 > has notes some notes about doing such growing and > mentions virtual machines --but uses an ada0 > example. It includes adding a swap partition. >=20 > It may be best for such a test to have a larger > than intended swap space as an initial test and > then, if things work, to change the swap partition > to be smaller and see if it still works, possibly > bisecting to find what an approximate minimum > size is and then judging what to use from that. >=20 > (Either way, relative to avoid the existing problem > that you are having, based on my experiences, I would > never recommend file-system/vnode based swap spaces > be used.) >=20 >> Summary of events to date: >> - This was installed as a lightweight machine. It will never hit swap = in >> "normal" operation. >=20 > The toolchain's memory use has been growing as the > versions progress. Fixed RAM+SWAP sizes over long > periods of time are not realistic as stands --unless > room was provided for growth up front. Because of > kernel data structure tradeoffs, SWAP sizing is > effectively constrained by RAM size. For 64-bit > contexts, something like 3.8*RAM avoids notices > of mistuning when the swap is added. >=20 > (An unfortunate property of the mistuning-message > is it makes a suggestion that involves other > tradeoffs for other kernel resources as I understand > --without giving any hint that such a tradeoff is > involved.) >=20 >> - The only reason I added a swap file was that someplace in 11.x = building >> the kernel ran out of memory. >=20 > Toolchain again, yep. >=20 >> - I only build a custom kernel to get options TCP_SIGNATURE for bird. >=20 > Any chance that you could build the kernel outside the > specific VM (in a less constrained context) and somehow > transfer it into the VM? >=20 >> - The swap file worked correctly for all of 11.x until I tried to = build 12.x. >=20 > file-system/vnode backed swap spaces do not fail on > every use but are unreliable. Back when I was isolating > the failures that I was seeing, I found notes from > a wide range of time frames from folks having problems, > as I remember. (Not that I could point you at them any > more.) The problem did not seem to be new, predating 11 > for sure. >=20 > The toolchain memory usage growth over time has probably > lead to reaching failure for file-system backed swap > spaces ever more often over time for ever less > constrained contexts. >=20 >> - There likely out to be a FAQ or handbook page about how to lay >> out lightweight machines. Having used it since 4.x, 1 GB "seems" like >> a pretty big machine, yet these issues arose. >=20 > If partition based swap spaces prove insufficient, > it may require adding messaging to the kernel that > reports on just which of the 4 conditions is > leading to the kills --and possibly related > information for the one that is happening. That > context might be required to make progress. >=20 >=20 I'll note an old buzilla about OOM kills: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D241048 started on 2910-Oct-04. [12.1-RELEASE was released on 2019-Nov-04 --and ended support on 2021-Jan-31.] The bugzilla was resolved as fixed on 2020-07-11, but the criteria is a little indirect and your case might well have prevented such a classification if it was known back then. (Too late now for 12.1-RELEASE-p* .) If you still can, may be it is better to skip having 12.1-RELEASE involved in the upgrade sequence? =3D=3D=3D Mark Millard marklmi at yahoo.com