From nobody Tue Jun 27 16:59:40 2023 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Qr9tj4nZqz4kjMy for ; Tue, 27 Jun 2023 16:59:57 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-20.consmr.mail.gq1.yahoo.com (sonic317-20.consmr.mail.gq1.yahoo.com [98.137.66.146]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4Qr9th0YClz3CGd for ; Tue, 27 Jun 2023 16:59:56 +0000 (UTC) (envelope-from marklmi@yahoo.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=yahoo.com header.s=s2048 header.b="geF/Fyme"; spf=pass (mx1.freebsd.org: domain of marklmi@yahoo.com designates 98.137.66.146 as permitted sender) smtp.mailfrom=marklmi@yahoo.com; dmarc=pass (policy=reject) header.from=yahoo.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1687885194; bh=rayNNksDHDPBbIDPDXmdHF6OXDYiwCGJQ/+W9Bgr2cE=; h=Subject:From:In-Reply-To:Date:Cc:References:To:From:Subject:Reply-To; b=geF/Fymera5WRJgb3f8E+cFw/rXvFpcc0fvzCsc1d0jQQ+aZY56k6RJ3OCnYeiVtRUtbVLsAbHna7qlm3hGUgc+Tg4c46qen5TKuWL8fsNR2WhdmG3yesE2axLYk8aeYFmXf87q5CU369ZUem4oEu7kZOR8uthcha0LsocYqR2R41fzYXBZcgMEKHqiTkWe7cML2I4bQ3k/01rXNjFJSczE4um3EskWUQjNhyWVn/SZHVhHgl12OSitSHSsew83wOWiKtgfk4WW35gpjJ5ChQQmWRmvUMRZgw7pfi4YWme/3fC+O8sHXM2QnnLnbc5c+IFqrDj7mxIBfbhxXP9af+Q== X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048; t=1687885194; bh=Z7sB7Ir9n7cr5wqZgSdDMRx6bX8Fbfo3rkcY9AoMLqQ=; h=X-Sonic-MF:Subject:From:Date:To:From:Subject; b=eKyx1uFXP6fDNdW7Mk4atLH7yDxIKnEqHVwTg9eZioJ/tIsyTltuyK9PAIK7XJ1+9KdjaXWHpCW12/g9sht1+vZMxK8VlBXiloS2Pud2SsKpm+cAgeLLFy9HuL51WHvYcOrXzrwFWdlH1qf4ot+2ooItfUHSKK0C6ta6RO3L7KNBu8DFNVMvPpobICukJzD8AgPSz0pJKro8X8Ba2OBKjx1V86tBROyq7BffbkQSw1eDqQGyHkIT8nudy4aWbcCCtlp+mrNQRU4XmLbWdVIrNKtTCkhmkwUkj8diqzYWgopBoPbw/+LteDGgGtPEWSOiaYVetbc3jijld0jZMfYl5g== X-YMail-OSG: jqRDE1gVM1nqTNiBA_Nihzr.Powta.WXge_KGdxVMC1F28PJieyaU_e0kXz4Fd3 lYl_OPT1buOiCl8zk2E3jyMy3a8GRIb0A2JPoJ_gTMJvf.FjSKVoL9rF3.J1orqwLlKcCjLP6akH Vyq1iCto4oNcpsqObZ749eNMF_KS9qci9QfXYscjN0cHucJVBcyycKS_d_AkQNai4L4JmkdRmck6 7fntfy74WFzCch5x0KHHrm.pQ2D7sUj5mSLCnvSURvwKd_2G6grFHi2fVS4T0SdOQ.fQN5mLJz1w _CqbW.CXXZmLfO1Cyo.Y0LkAGF1cwCi1bl4EP2h9yrdc.mVNqFKQ3kfGoC4ncb86WxCdERyALYnE UAD11WUCopK2YGq9sVA5yOniyeGcDfDzksEgYDDj9IDyUZ9G_tz3O39lYhpyyQ_XwtSrzZHsx2XQ PgsftROMhC60kphXP2QYr5P4b4voerJin4kJ1OUjjxdMF1QlIgwxUXEg0uM5txiY9sKk3EDnTYXF 6IY_fLVxvirfIPTle0s.pl5szZkYbVcbi36SgNIHWH4gHJtKxu4vBaCuMh.uies1LHsWrn_rhnU. JPSPxsr1wWPtFNtV0kcylvblQlr2opO4JzmcM6hlPBsU9eLp9LAodmSxmlp2f2TTcO8TrdQwQirn Bt7kau.JdKtHw9_QY4cEF1Gc.U2gKy6DpudeSocIN2GWB8Em8wJF_VBw7K0Dlu_aybfzmySahinh eKTslXPWc5j7nfnwDv.YtKkE._2O66NfO72RZpwTOdemYtvRI9krktfnoud592.hjDf6frercqIj MeWnaeA8Q7v0Icm9O1Fx7YgKAqvfelhxHeJ5MbP.8r8NW..N9hDq8w8SBVlyroE3FDdOsPK0L_BP 5NYSpDFqHrW4IolmHERLnHIWa5bqGZ1K0qoYcoij3roq00znyCu4ba2btAk7sPteBXMkj4VmYa.U NwfhdOgi91I5VmVSF.O7TJF2Fxxm4ZIkokvRT7_JCJZ.277IO7HKShQl8R.QdKiiLYMBm3ndTMF9 NaIiB2hd0NNFE9jHHLDCGxiAHrutA3w4srX55XlZzD4aIxmQpTJRA053Sls2JSWvwsTOo8RVMs4P 2q1b6NJhcRK9bCb9n15LnP0JuJsf3pdS.CFW_fqZZZozIwVkHCN33gR_qMg238iR90MpupUkVvJj aUua8Enuq166dR04old9h_YWiUHQDG0kWTB6OM7mG_E_hiZ_avk52QO_rYEdQBHG3jjsnXutwB4_ NfW37.rWg154ZAVeVnDRgBqWhrs9sk7n94o7pVi40Yveq.rj9o3ptq6tXd7HKp_8eJHk7zil8kyj b1WW_sj2MrFI_u6rc8ctQpYpdeWULKgGeYHcqIGUelV3Pcf1uwNAV4c8S0s04i.XJzc785BdP7Hq qnsQf61Jb94kX5DdhbIBjtlDW_cuo6oVgKamrHSgXVXlX_zJYUE8dJlF5ByM2INkzkI8hkBzylaU w9jH2WcnS2NkTmisd0TAJhoINQgd94qbW2cc7PypobWIpdCMwtKq5ndjC4_iVzr54OFrk1Hm6bEU Fw5pXGggsdzewmJqHjdrTbjBdMWGkKeAIzFJHXpCgjXEbsONakYnjCjcYSn.S4qbe9rUw3InEGvf A1nLY2kwg9yG8QaDC3NOVifBBVPVFpbdEToOfQIOjyvhX8pTW5yZrCkV_cCnGrL.vVzDU5VMfgzS dhAr9GT2rX7GU0lYiPWf583KRZv8UL6tzN4.ROVpvRCqgsjg2ntbuD.u1y610Dtsq5D783d7N0Qj qCFyCURWC0Fz_x0D6BGtK2zJojFbwEsl6fn4sc4Neuw0OrnrgPwXDD5pjeN.4Pd_5YkFJ27VLZU. pWQXXG1RMP9UM7ZUgIh3cbZdNg6s7sxEZL46.n90yE_rNNP7ac7OrTyCALIT1tIVEXnJzLlt56sj .emu4NIvafUmdJ6tvkuYy4KoVUYUsAtSlkfEKv3nn9Oh3VmJl5D5Azms1znsHPOGKXfU1qfuMseE C.wXcEYsm2BwcfQo7TUSnYqbDEtEGfw0sT3WoPMefEBSRObVP6YIcpP5hmge6wYgpZGKEuiZq_zQ mywPTOte5FQgNV_LKfkxvKYaTx1wDSRUcHpztxvgDFYNw1py_Ktzi.ER1jGwMUusJH4CB3jElIX7 CJo.QZ6QrQKnIcWlgRsPN..goBXUGRv9zWoHMTSB47UkfZ96TWgKLLdQa6Lp_2E7iPWNkYZ6b1nH jBK6uUcuzeiPhFQOCLrwCx7_yvy2jarYc0j0MbkVhSfo6yIuJ3ID4ZfR5LRSPvLz.CuEwyf21i4I vpQ-- X-Sonic-MF: X-Sonic-ID: 485a7c88-a301-4d18-a420-1bbfe6698aa7 Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.gq1.yahoo.com with HTTP; Tue, 27 Jun 2023 16:59:54 +0000 Received: by hermes--production-gq1-65744d4b95-w7swl (Yahoo Inc. Hermes SMTP Server) with ESMTPA ID 2ae622afe81b2a22df628f7907b70b16; Tue, 27 Jun 2023 16:59:51 +0000 (UTC) Content-Type: text/plain; charset=us-ascii List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@freebsd.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.600.7\)) Subject: Re: -current on armv7 stuck with flashing disk light From: Mark Millard In-Reply-To: Date: Tue, 27 Jun 2023 09:59:40 -0700 Cc: freebsd-arm@freebsd.org Content-Transfer-Encoding: 7bit Message-Id: <066FD282-1637-448C-99FF-BA62718386F0@yahoo.com> References: To: bob prohaska X-Mailer: Apple Mail (2.3731.600.7) X-Spamd-Result: default: False [-0.99 / 15.00]; NEURAL_SPAM_SHORT(0.81)[0.809]; NEURAL_HAM_MEDIUM(-0.75)[-0.750]; NEURAL_HAM_LONG(-0.55)[-0.551]; MV_CASE(0.50)[]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[98.137.66.146:from]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/20, country:US]; RCVD_COUNT_THREE(0.00)[3]; FREEMAIL_FROM(0.00)[yahoo.com]; TO_DN_SOME(0.00)[]; MLMMJ_DEST(0.00)[freebsd-arm@freebsd.org]; DKIM_TRACE(0.00)[yahoo.com:+]; FROM_EQ_ENVFROM(0.00)[]; RCPT_COUNT_TWO(0.00)[2]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; MIME_TRACE(0.00)[0:+]; RWL_MAILSPIKE_POSSIBLE(0.00)[98.137.66.146:from] X-Rspamd-Queue-Id: 4Qr9th0YClz3CGd X-Spamd-Bar: / X-ThisMailContainsUnwantedMimeParts: N On Jun 27, 2023, at 09:47, Mark Millard wrote: > On Jun 27, 2023, at 09:29, bob prohaska wrote: > >> On Mon, Jun 26, 2023 at 07:57:05PM -0700, Mark Millard wrote: >>> On Jun 26, 2023, at 19:12, bob prohaska wrote: >>> >>>> A Pi2 freshly updated to >>>> FreeBSD 14.0-CURRENT #41 main-c3e58ace31: Mon Jun 26 17:06:01 PDT 2023 >>>> bob@www.zefox.com:/usr/obj/usr/src/arm.armv7/sys/GENERIC arm >>>> got stuck with a flashing USB disk LED after starting a -j3 buildworld. >>>> No response to debugger escape, had to pull the plug. > > I'm confused. > > That says "stuck with a flashing USB disk LED". But: > > http://nemesis.zefox.com/~bob/fbsd/rpi2/20230623/readme > > says: "the disk had gone to sleep mode. Both LEDs were off" > > Are these two different examples with variable behavior > across the examples? > >>> If I understand right, the LED flashing means the disk >>> had not stopped doing I/O: the system was still running, >>> doing disk activity. (But I do not have a description >>> of what your drive documentation says about how the >>> drive handles the LED and what various patterns/colors >>> may mean.) >>> >>> If the processes associated with processing input that >>> would identify the debugger escape had the kernel stacks >>> involved swapped out to swap space, I doubt that the >>> debugger escape would work until/unless the kernel >>> stacks are brought back into kernel RAM. >>> >>> Avoiding the specific way of losing control is why I >>> have in /etc/sysctl.conf : >>> >>> # >>> # Together this pair avoids swapping out the process kernel stacks. >>> # This avoids processes for interacting with the system from being >>> # hung-up by such. >>> vm.swap_enabled=0 >>> vm.swap_idle_enabled=0 >>> >> >> This combination was tried and didn't seem to have any consistent >> effect. It's commented out at the moment. > > By not having them, we have no way to know if the > relevant kernel stacks had been moved to swap space. > Having them is part of problem isolation/identification > even when other forms of loss of control happen. > > The 2 lines serve more than one goal. > >>> (No claim such is the only way to lose control.) >>> >>> You might be able to get a clue if their was disk I/O going >>> on based on modification times on files you know would have >>> been modified periodically for some time (minutes) before >>> you pulled the plug --but not modified on reboot and later >>> activity. May be a log file that would only be modified by >>> the build that you had been trying to do? >>> >> >> There are log files for build and disk activity (for a cold >> hang, no disk activity at all) at >> http://nemesis.zefox.com/~bob/fbsd/rpi2/20230623/ > > So this is a different hangup? j4swapscript.log has internal timestamp pairs: Wed Jun 21 16:34:06 PDT 2023 . . . Fri Jun 23 07:26:10 PDT 2023 It would be interesting to know if "Jun 23 07:26:10" was after the appearent hangup was identified vs. before. >> In this case the top window was via ssh. Lately I've >> taken to running top on the serial console in hopes >> that will help distinguish system hangs from USB hangs. > > If you want to identify system hangs, please > put back: > > vm.swap_enabled=0 > vm.swap_idle_enabled=0 > > otherwise all you may be seeing is the relevant > kernel stacks having been moved to swap space. > That is not a form of system hang relative to > overall activity, leaving more uncertainty about > what top no longer displaying updates implies. > > You can use sysctl to adjust the live context > as well. > >> >>> (You did not indicate how long you let it run with the >>> status "possibly hung up".) >>> >> IIRC it was about half an hour. It was already stuck, so I >> don't know the actual time > > No logs or other files with modification times that > might indicate if there was activity during that > around 0.5 hr? (Timestamps in files can also serve.) > >>>> Reboot with kernel.old, >>>> FreeBSD 14.0-CURRENT #40 main-c1cbabe8ae: Tue Jun 20 03:58:47 PDT 2023 >>>> bob@www.zefox.com:/usr/obj/usr/src/arm.armv7/sys/GENERIC arm >>>> seems ok, I'll try to run buildworld with that. >> >> The kernel.old -j3 buildworld is still running, no complaints so far. >> If it succeeds I'll experiment with usbtop. === Mark Millard marklmi at yahoo.com