From nobody Tue Jan 25 20:49:02 2022 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 96003197083E for ; Tue, 25 Jan 2022 20:49:17 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic314-20.consmr.mail.gq1.yahoo.com (sonic314-20.consmr.mail.gq1.yahoo.com [98.137.69.83]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4JjzVN0hD4z4cny for ; Tue, 25 Jan 2022 20:49:16 +0000 (UTC) (envelope-from marklmi@yahoo.com) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1643143749; bh=W824uPDRPdYHBa4GojcB4PuBu5LV23jray3ZUueu2mo=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=i358Ht7tf0IWFtrVlO0+4JaIVj4r8IT9V1kcapDz5TJeLxWfQSrF9QgYYubYwRxpPLRAHA8qRECg5PUFaajDBz6NNFbFJ8dmElGadDjC/wmYFiFt/LjlsOYcxKw2xfwKEi48blsgl21PY6SSBZVfwHZwUCNdAlAT0+JLS0ShdOEvkncKyQI39SaC7+nQeRfc+FQDybXhnB6O2zXu1idyJv82SMJtxvvkMk4ItArtZ6JLauPaXgmMRstedDxu5ai8MApD3+x0BItmSJqJGckf6KKFdBjfShub5mKdrIXEhqeCDYzVt94m/iscZZ0OyWkI2PfhiolSM6iDMuY7j9bP1A== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1643143749; bh=Whx5OD+AU/aKso6BS37cfjDnIKB17+QgKGj2ye03Q+R=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=rdDp9LKGY1R6cT+nF4E7CR5ywtDG6/aFuXde38RvwXJXr7rAihMnIXJy4ajrEAyTpszC2BVVs7hB8e+CtguYFNw57yk0JaUZHcRs8Ff4mKWN4lRoFV7DW/CMjb3NvmSqxV/kW05g1/98LXsA5b59ij7OOmOyetns6wDh/EVaS298EWnrggd47b6xsmWkNx3U6TAycfJFsXzUksAkwdYqJYDnHFQeplIYkQHudzW2I16SzCLVPGPJHjjBCQSqo+Jg/rCIuKzDH9OTnDpSyPT1+dgZ+0NZwA54Rfq7B5TU0cNvULxZpVQ39Fh3JiBkEoho1v0NOThDWN0RdOFnjIQ+Zg== X-YMail-OSG: a.9.rhoVM1nFT66iVfSqhPVVXkFAg2mP3xbGb5W7LHbMMIfdso83k9sjMGxeiGG 5G6LBkcCkshiTZ1COU1IFoiELFpyU6jsxsi0WUXTrpjzD2QFZ3Hu1_NoxUZYq7hB1.rfaxWWkFPa w2dsC.4.blRjGOC6A2Fglr734aSn2sLDd0BgufS33671hYXsm0WFIINU5BW4RgDOd68Azi1tXG6A acv7xrU94EGanT3zja_2ODIeELEctQbIOHtBkPIfLIVUo1REYq3dqvhXc1bGPBLNF69_5TbKti6R ctKQJqIYyOBjpRoV_7ogSwl86WNwAO4Fx3_P3GW1JBNn7dCfezFQMdiwiQC3BGGq4ApiEyHMW8L3 jIsih_UR8AyIHAR6wy4jAnbGr0kwYk.DVPdYk8.cLjtwHtakfNwPdOCDH9QfM0OErt8h2SqywnBG cjCBnhlIpOHAC383KI_QaCDPd84sGawuWNnLw_CV_bQy3VbLPnBdUYYcfJexbqibztTVvKRf2c6z Wk3NpcLeKX1uXQ0m7uNN.ca2LaRli.NAO7yr22DIQ5ZwQEUMfFduZFaDUJMocioR1Gt0hJkccHk1 iIDqMdZpgGHvPaz2pzG8QBc.JgOSN6GPUpZRo8fH1syw8hxZMESmr4tg3xeX83dbDHvuDrkHlhno _Bg7HvSNN5v1S8a2_JMMt.NBVAtl6yEPURrfWWAFAYNRkbCuFu7ljyGn3uiw65HsQqBzExTAfYvn ZTjQilYt..1i3UoMQG1wRdUCg1Yh6N6bG5zovP_IEUgltcwk5NS5XoWx7yDv6rIrhLs2ieut3XOX 9ed09y9IuPodNOWkwyuVUjateMO0AgOJVggd_aMEy81qlYq9TuUkESUxfq8eol2B8QVENsiROlx2 6lvyr6g0C5n.FZIkCiK6SL.g3oVjMpMYyrXsF0dY2iTS3zsekYKS7gfbfrNSaNaxaDsDQ8EYey0c QxgHAGLdcl2.Q9kmrOe4WIRc4Al808NZ8nEi5ZQytk_aThCxY1ykhbz1YeLqKYpk5.NRsU6GsMPD .94WDQNTEeBl9O7QpcceH9zr4bW14v0BfW_niaqUdef5oiELNh0vj28PYCjNZ8jcVMLmJuxzAu54 KlTf.Xnkzp5VI4mHBiYwRgWE9G1grmJdA0WJ96jkkfOwnFrbKlJRZmXz6oLwIqqOfpUZ_TGZlYHz AENUTPHc0K4CCW1EYL9EWdin6D0N4jVYaN_353NhLLAPMkOgnuLyud7Z468RaBjLRgT_7EUtUd4y D7P73iKvj0dT8UT6qOuKezNSqmA9rqHMVF._QmqFxlvTiWa3I_8a6vIZ5LrBfnuP3kykP5KvIWb5 PQbr2KCCe8aAF3kbJlClg.enoprE8YGD5Wcbj6.avj4XkCqKerkg30lacXJv5eXLIGpvwfNtUFlC E7Xvkw.BctVPUfGm1dL_I__WijI8eoOdaXwRqpJlVo1SqVpMJt2lAcmXWZMoRqohhV8m.z9wNt5R nWhn7eh7oCNZ7kTasQ.WTZjKCGUr0IrkABPPviK9rogFyWvGOcY8x_v6AtAy_HZmOtJsIuAvHVXP ME0nuFAsauSHxAFFohcPBrtC17kxf7Dg.pFGHFgaovx4cj7qeITFGyAT6gBlRidb9UdnVSnqeoCa 1PiBT5nAwEj.9YeQkrhiKN.j5kk5QIo5Vtexh3Cb.gPnPHLPSIG8Zqoy87Mirn5npmsU6AOgrDJJ 5Y9ObC9ldjlJXsUWXtdt1qBPEjMXNNwfCLclLikyIVxuoomDxx3M0YalNGXi7qIFuJ7EWx3hEWhJ osupigLbaq4zWEukNyhe.RAA0eGPznEyRcKFJ6OEiuxMOpjcfUfah.A53zTqS4DIB.8FrofbNzeG mB2fmGkS480ANzrDcMhUGwTNNIKQOoRd9aFPWvHVvTdUusuwO9OoQLJfOB8ayZUYYgB0U_wN4_6. r.kPP9tWPLxb5cX6ju9RbJuPeUiNt5WUjV.ncI4OQ_5FxPvy72r6M7Md8AOuF5jME8Fftc_hq1eI yWTdhy3932qaRotvK0SUf6VN298wBPyP65jy3Yz9J4QJgcfSPxoajKmbcOqGZupUyPaOdRc0bC5J PBNOcQOUXK6f84bVMybjCVztrxg3lTthridpc_imbQp651ni0bhA_Sj2.PdOfVYSLE6FPRCcy.bB BWLIdJxR5PMgpFAT2QRzQmi6Vl4ESuAsOEgZ0QaeG5yIErSuxxJS0kKV2fjOh6JKMXFkpAFqFAbe C4JVxcbjLy1peO94o68zqcwca8eD.r1TWj6OBJjPdigR__aYYdwXFDtFGSs0MGbzqnQcaFnJESWS eQDN9M62eMPpznw-- X-Sonic-MF: Received: from sonic.gate.mail.ne1.yahoo.com by sonic314.consmr.mail.gq1.yahoo.com with HTTP; Tue, 25 Jan 2022 20:49:09 +0000 Received: by kubenode503.mail-prod1.omega.ne1.yahoo.com (VZM Hermes SMTP Server) with ESMTPA ID d0559bb82ce130373eaa6df502402384; Tue, 25 Jan 2022 20:49:04 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.120.0.1.13\)) Subject: Re: Troubles building world on stable/13 From: Mark Millard In-Reply-To: <20220125180823.GB43635@www.zefox.net> Date: Tue, 25 Jan 2022 12:49:02 -0800 Cc: Free BSD Content-Transfer-Encoding: 7bit Message-Id: <35046946-7FE4-4E44-950F-BF9CCA72D8F0@yahoo.com> References: <20220121031601.GA26308@www.zefox.net> <8595CFBD-DC65-4472-A0A1-8A7BE1C031D6@yahoo.com> <20220124165449.GA39982@www.zefox.net> <5FAC2B2C-7740-435E-A183-FB3EF1FCE7F9@yahoo.com> <1CB4EDCD-0998-4363-8CEA-14854EB76FA3@yahoo.com> <20220125162245.GA43635@www.zefox.net> <61A3CF79-552C-4884-A8EA-85003B249856@yahoo.com> <20220125180823.GB43635@www.zefox.net> To: bob prohaska X-Mailer: Apple Mail (2.3654.120.0.1.13) X-Rspamd-Queue-Id: 4JjzVN0hD4z4cny X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b=i358Ht7t; dmarc=pass (policy=reject) header.from=yahoo.com; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.69.83 as permitted sender) smtp.mailfrom=marklmi@yahoo.com X-Spamd-Result: default: False [-3.50 / 15.00]; FREEMAIL_FROM(0.00)[yahoo.com]; MV_CASE(0.50)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; NEURAL_HAM_SHORT(-1.00)[-1.000]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[98.137.69.83:from]; MLMMJ_DEST(0.00)[freebsd-arm]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.69.83:from]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N Status: O Content-Length: 4363 Lines: 125 On 2022-Jan-25, at 10:08, bob prohaska wrote: > On Tue, Jan 25, 2022 at 09:13:08AM -0800, Mark Millard wrote: >> >> -DBATCH ? I'm not aware of there being any use of that symbol. >> Do you have a documentation reference for it so that I could >> read about it? >> > It's a switch to turn off dialog4ports. I can't find the reference > now. Perhaps it's been deprecated? A name like -DUSE_DEFAULTS would > be easier to understand anyway. I've never had buildworld buildkernel or the like try to use dialog4ports. I've only had port building use it. buildworld and buildkernel can be done with no ports installed at all. dialog4ports is a port. I think -DBATCH was ignored for the activity at hand. > On a whim, I tried building devel/llvm13 on a Pi4 running -current with > 8 GB of RAM and 8 GB of swap. To my surprise, that stopped with: > nemesis.zefox.com kernel log messages: > +FreeBSD 14.0-CURRENT #26 main-5025e85013: Sun Jan 23 17:25:31 PST 2022 > +swap_pager: indefinite wait buffer: bufobj: 0, blkno: 1873450, size: 4096 > +swap_pager: indefinite wait buffer: bufobj: 0, blkno: 521393, size: 4096 > +swap_pager: indefinite wait buffer: bufobj: 0, blkno: 209826, size: 12288 > +swap_pager: indefinite wait buffer: bufobj: 0, blkno: 1717218, size: 24576 > +pid 56508 (c++), jid 0, uid 0, was killed: failed to reclaim memory > > On an 8GB machine, that seems strange. -j build? -j4 ? Were you watching the swap usage in top (or some such)? Note: The "was killed" related notices have been improved in main, but there is a misnomer case about "out of swap" (last I checked). An environment that gets "swap_pager: indefinite wait buffer" notices is problematical and the I/O delays for the virtual memory subsystem can lead to kills, if I understand right. But, if I remember right, the actual message for a directly I/O related kill is now different. I think that being able to reproduce this case could be important. I probably can not because I'd not get the "swap_pager: indefinite wait buffer" in my hardware context. > Per the failure message I restarted the build of devel/llvm13 with > make -DBATCH MAKE_JOBS_UNSAFE=YES > make.log & Just like -DBATCH is for ports, not buildworld buildkernel, MAKE_JOBS_UNSAFE= is for ports, not buildworld buildkernel, at least if I understand right. In other words, it probably would have been the same result without the two arguments. > It seems to be running with only one thread so far, not sure if that's > by design or happenstance. > >>> However, restarting buildworld using -j1 appears to have worked past >>> the former point of failure. >> >> Hmm. That usually means one (or both) of two things was involved >> in the failure: >> >> A) a build race where something is not (fully) ready when >> it is used >> >> B) running out of resources, such as RAM+SWAP >> > > The stable/13 machine is short of swap; it has only 2 GB, which > used to be enough. So RAM+SWAP is 1 GiByte + 2 GiByte, so 3 GiByte on that RPi3*? (That would have been good to know earlier, such as for my attempts at reproduction.) -j for the RPi3* when it was failing? Did you havae failures with the .cpp and .sh (so no make use involved) in the RAM+SWAP context? > Maybe that's the problem, but having an error > report that says it's a segfault is a confusing diagnostic. > >> But, as I understand, you were able to use a .cpp and >> .sh file pair that had been produced to repeat the >> problem on the RPi3B --and that would not have been a >> parallel-activity context. >> > > To be clear, the reproduction was on the same stable/13 that > reported the original failure. An attempt at reproduction > on a different Pi3 running -current ran without any errors. > Come to think of it, that machine had more swap, too. How much swap? >>> It's in the building libraries phase now. >>> Based on log size I'd guess it's about halfway through buildworld. >>> >> >> Well, hopefully you will not be stuck with -j1 builds in >> the future as well. >> > Indeed! At this point, I expect that the failure was tied to the RAM+SWAP totaling to 3 GiBytes. Knowing that context we might have a reproducible report that can be made based on the .cpp and .sh files, where restricting the RAM+SWAP use allowed is part of the report. === Mark Millard marklmi at yahoo.com