From owner-freebsd-ppc@freebsd.org Wed Feb 19 23:17:49 2020 Return-Path: Delivered-To: freebsd-ppc@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0CA7D24800F for ; Wed, 19 Feb 2020 23:17:49 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic304-25.consmr.mail.gq1.yahoo.com (sonic304-25.consmr.mail.gq1.yahoo.com [98.137.68.206]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48NDCb6Dnpz3JrL for ; Wed, 19 Feb 2020 23:17:47 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: rppPzVIVM1nWefn01L3hmyzQ0ZBbBEwh4gJfc5.Yg8u4lMk.0Ky3qyyifFOba10 vcD0pRdoHnlbCKHBsJw7ODiq9J08uh_AwB1FtxpBAyk7SXIpdlQIHOGL4fEPPRnMPDyJsDj2_E7O Dn.RyUeS6ILFoZD7Y5mBql8C0HSB_PfJ._HHjxdZeImlu7fvG8qlQV.q2.iYHf36P1xBthMwbd86 ubndTIdBtBMMFwQuHdN3CpQqfDrVG8ODdFyn4vqCy7ygSrolxtdZLZWL71sNUjaNBSC_oJ0HFH3_ vStL1fHqEWMp6UOswNfmugml1t9mbaCjdSD03cwMSKc3.mGULewPXO0X7pRtgaRwSIOe4EZXpeZv 4dXElPXt7NmfnLNi_9Sk.bQtEY8esSoFDEfacFoSJj7p_BcZLJWSBwPHp6SwikjSilR_UUddux3w VCZQqI2g5tzQzwQrHXmAwu0svILNPR_UUGcNIE_w9IlDyXEP8.QDmRIBoX8KzI9iPLgs00AOpnvz 71ocfDo59.ruKyzGelFSuRUH4usrMwuEqL1HMgZQ3w8M8uCIbm7diutJ.6M1JHamA_R3tsQ.SPUO 3ASyUU1NMfXllTR6xn_Nw0z10LpR.u3maMO7BPR030x_Atk73y5zg7Ugq6.dym02IvQDh.lHw.gn OSrZuQL6H9ELxdpod8cCrn_m0.BljQQO12Whf4n9clYaBUK17El9sYfxgfO4b8Iy0AGH1EGwhvMN OkFHOGNEBIOR50jOE.Mvfi1269CmRG05MWgGC6Y8iMqk6oK2ICD6GPGMeUx56xXDpoTQkecFO9yl nMWr3oETnoO_qVc.ebqxYVVGQtdkjU9pNfyHW1_P0mCW08jnfUEcUvDcuQMAVs9jtI3T9AEn_dpU gG80FXTRW48OMKz4lEKWzFBxas.UMOaIkcP1YPh4Xa76OjugLhwTmMQ285pYrNdLEj7FWL_KZDCy 9IOzHSOpJFhxO77W1.LYOf4d2IY28jhaNMqWcf9fxkFXWGpNgjHbw0MKHvynSe_Vv52JgmIeeBlg G5vjBOC7TA7NEepZK.cyUktutVSfDx9CHhbNnAL1GLFibAXAkdWL9xyxmSvy0ZUVOAn6Uy5gCLfF GvaWP6AbClAsc34XAbeaYCDDWOui9_WaJjkb4PI9Mti_.JMiDYZrJbWdmWh8fge0dwu.9r_L9.Lb Vlm0EBB9Tjg1m7XpgIgUX8rJsGovGvjw9idH0.x1Vl2kjuaKFcVUNip5shjDce_IZyoINxi_Pnsu sXoOUQu96Qva4hYECKAjpCKGeualvo.Qxxrko0oKC13jws2VfjRUPFpUxjpUi.H9Q6rHwt4AmbI8 bRBV2sffkhMnGbz5eQM17xsF5HqWP.8hJFHTYTODtdb5k2LZ.cVjc3HiRcGDo7UizvNZvW9VfWu5 vsej3K_J1latHGYrmpVF.Ula_FlBFyVIi0yP7248lMvqsMAYF11gm Received: from sonic.gate.mail.ne1.yahoo.com by sonic304.consmr.mail.gq1.yahoo.com with HTTP; Wed, 19 Feb 2020 23:17:45 +0000 Received: by smtp432.mail.gq1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 9faf9e6ce4d92b5db5966c5fb7ba34c8; Wed, 19 Feb 2020 23:17:41 +0000 (UTC) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: head -r358094 and -r357948 vs. powerpc64 or 32-bit powerpc multi-socket TB (time) mismatches: lots of temporary system hangs Message-Id: Date: Wed, 19 Feb 2020 15:17:41 -0800 To: FreeBSD PowerPC ML X-Mailer: Apple Mail (2.3608.60.0.2.5) References: X-Rspamd-Queue-Id: 48NDCb6Dnpz3JrL X-Spamd-Bar: -- X-Spamd-Result: default: False [-2.38 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.88)[-0.883,0]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; FREEMAIL_FROM(0.00)[yahoo.com]; MIME_GOOD(-0.10)[text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-0.999,0]; IP_SCORE_FREEMAIL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; RCVD_IN_DNSWL_NONE(0.00)[206.68.137.98.list.dnswl.org : 127.0.5.0]; MV_CASE(0.50)[]; IP_SCORE(0.00)[ip: (0.87), ipnet: 98.137.64.0/21(0.83), asn: 36647(0.66), country: US(-0.05)]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36647, ipnet:98.137.64.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-ppc@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Porting FreeBSD to the PowerPC List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Feb 2020 23:17:49 -0000 [The below is mostly for normal FreeBSD, without any patching to cause TB values to approximately respect some cause->effect ordering sequence across example sockets/cores.] Head -r358094 checked in the fix for head -r357549 breaking booting on (some?) powerpc machines, such as PowerMac G5 dual-socket ones. This means one can again grab artifact.ci kernels and test them, for example. (For me, that avoids my patches being involved.) That in turn exposes kib's -r357948 check-in: QUOTE Consolidate read code for timecounters and fix possible overflow in bintime()/binuptime(). END QUOTE This leads to mismatched power/powerpc TB values across sockets/cores causing the system to temporarily hang/wait from the overflow being handled and leading to longer times being involved. Temporary here is not necessarily momentary but possibly minutes and it may only be minutes or less between such hangups. The hangups can start before the login prompt is reached or while typing in the username to log into. I see this both on multi-socket PowerMac G4s and multi-socket PowerMac G5s using the matching -r358094 kernels from artifact.ci . It is the same multi-socket PowerMac behavior that I saw for a trail versions of kib's patch back around 2019-Mar/Apr or so when I then tested without a patch for the TB values. (So I was expecting such a result from such a check-in.) I've not (yet?) figured out how to fit a variant of my TB patch into the code base as requested or to cover things like suspend/resume if there is such for multi-core powerpc systems that also get the TB value relationship problem. (I only fairly recently learned that the TB value relationships issue is not historically limited to PowerMacs.) Without the long-sustained available time like I had when I came up with the existing patch (weeks back then), it is not clear how soon it would be for me to have a more general and more acceptable patch for the FreeBSD code base. My test context also does not span lots of sockets/cores or NUMA variability in memory access timing --or any suspend/resume contexts. Even for what I did cover, I'm not sure how well it generalizes for such issues. I, of course, normally run with my existing PowerMac patch. So far I've not seen problems from -r357948 for that context for the G4s or G5s (but have seen the problem for not using such a patch). === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)