From owner-freebsd-hackers@freebsd.org Mon Mar 4 20:58:26 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6447E1522B61 for ; Mon, 4 Mar 2019 20:58:26 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic317-34.consmr.mail.ne1.yahoo.com (sonic317-34.consmr.mail.ne1.yahoo.com [66.163.184.45]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 236EB879B3 for ; Mon, 4 Mar 2019 20:58:25 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: Av7ZzPMVM1l7UlZBbiUW8idImHi0nHz4ITc3Jyo21R27rl_HGqXYs41UrIuhVHR BH0YrPEQ9Fm0UeS6UuNHdJ9Enbwsx0MyVCLaZVgg10n5hzowe3_n3FiKUbWi30GFxlIWSKPOufWa mzYT6yxrjrYX3C9HQZYWM6M.87.FH1lGBa8CkzSIj0tCwJB4UUcwg9OWmwT6aInR7bK_.qMYiQHl vZL5v6PAzqhAqvTRQcTrUh4lHaVYj2.wbhjMSG9Cgz7xRHTEzK1ERYYOZaAkCrAlXQNj1wC7iVvh PUJDJ_2iurCK0M_HL_LcDOEf5P3pwH9HhBHq8wIp4Tx1VrhHmtgRM0F61A0NdPP.7nBw1hg3HKrj RggP9i6Tjb.NvwL1o_lmo8pqT3AoNkomzm66.H1HqzyHh7BeXG8RN7miZOY6_nhzZ4geXrU6l4tI 3_ypiTtgepCy1ogdW.BqjFwG7Ds0o1OZ5ESDA0cAkkPgjA.i_6PLz_pfcSmFvB42iA4mK_ORiry5 A82qjPSCJWJmlFH2BV2FOiFbhMSSgaB85Sq2XoqAYnQwba0aVZRL_vo0xAU_GKqrgDRC8HEcZaeY QSXAPNKSjXdv8UzKdFi37fVCwPn8GpYXkyZA7tpiCsTO1oN6LgSlC3MYxz73qCkqaaWSLHoBLFqv vsHoau68oZDuuN8_preWIrBlzxOEMqTfJ3AzaItnQubExOZHzVcvO_ZBxZIj.dBKzuSO_JAGR87V Rtma0p_xcGI9pUKysOMuWUUdqaLReL_YOBCgaMSpFVupNAc3SKRdbkU2jb2StJ_vnyGPhUwNajjW aqKitxLmSrKAGNyRH0Hz.eKfhw1YiwlVgq37FZE3VPgKSB6KBRfQbsj7MMgZWvthI.OOX7DvoUC8 yQfCKPczTar5VcW0S1BeZw_Vn6nFGx.Fj9w8Nb1IgGckb1GBH64.2tcLxo3bFQJsOdJdZIAkhwgd s8g7C4dbBozuTbGFs8w_7a.W7IKHEMrcobqYm1O5pdyTI7TRDJAksNSnEDtD3Dhl3Qme6dVg3tdZ igPjHHwaTKd853_QYquCMKOfTPILL8VEgxMpgRJA3Vv90BpZUvARBYhpK8ETy6ja0Els- Received: from sonic.gate.mail.ne1.yahoo.com by sonic317.consmr.mail.ne1.yahoo.com with HTTP; Mon, 4 Mar 2019 20:58:23 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp417.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID d6e36c6b74fad01663fd179bddcbc796; Mon, 04 Mar 2019 20:58:16 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] From: Mark Millard In-Reply-To: <20190305031010.I4610@besplex.bde.org> Date: Mon, 4 Mar 2019 12:58:14 -0800 Cc: Konstantin Belousov , freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: 7bit Message-Id: References: <20190301194217.GB68879@kib.kiev.ua> <20190302071425.G5025@besplex.bde.org> <20190302105140.GC68879@kib.kiev.ua> <20190302225513.W3408@besplex.bde.org> <20190302142521.GE68879@kib.kiev.ua> <20190303041441.V4781@besplex.bde.org> <20190303111931.GI68879@kib.kiev.ua> <20190303223100.B3572@besplex.bde.org> <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> To: Bruce Evans X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 236EB879B3 X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.06 / 15.00]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_SPF_ALLOW(-0.20)[+ptr:yahoo.com]; MV_CASE(0.50)[]; FREEMAIL_FROM(0.00)[yahoo.com]; RCVD_COUNT_THREE(0.00)[3]; TO_DN_ALL(0.00)[]; MX_GOOD(-0.01)[cached: mta6.am0.yahoodns.net]; DKIM_TRACE(0.00)[yahoo.com:+]; DMARC_POLICY_ALLOW(-0.50)[yahoo.com,reject]; FREEMAIL_TO(0.00)[optusnet.com.au]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[yahoo.com]; ASN(0.00)[asn:36646, ipnet:66.163.184.0/21, country:US]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[yahoo.com.dwl.dnswl.org : 127.0.5.0]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[yahoo.com:s=s2048]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; NEURAL_SPAM_SHORT(0.62)[0.622,0]; MIME_GOOD(-0.10)[text/plain]; IP_SCORE(1.22)[ip: (3.86), ipnet: 66.163.184.0/21(1.29), asn: 36646(1.03), country: US(-0.07)]; NEURAL_SPAM_MEDIUM(0.13)[0.130,0]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(0.60)[0.598,0]; RCVD_IN_DNSWL_NONE(0.00)[45.184.163.66.list.dnswl.org : 127.0.5.0]; RWL_MAILSPIKE_POSSIBLE(0.00)[45.184.163.66.rep.mailspike.net : 127.0.0.17]; FREEMAIL_CC(0.00)[gmail.com] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Mar 2019 20:58:26 -0000 On 2019-Mar-4, at 10:17, Bruce Evans wrote: >> . . . > > I think it is just a kludge that reduced the problem before it was fixed > properly using fences. > > Cross-socket latency is over 100 cycles according to jhb's tscskew > benchmark: on Haswell 4x2: > > CPU | TSC skew (min/avg/max/stddev) > ----+------------------------------ > 0 | 0 0 0 0.000 > 1 | 24 49 84 14.353 > 2 | 164 243 308 47.811 > 3 | 164 238 312 47.242 > 4 | 168 242 332 49.593 > 5 | 168 243 324 48.722 > 6 | 172 242 320 52.596 > 7 | 172 240 316 53.014 > > freefall is similar. Latency is apparently measured relative to CPU 0. > It is much lower to CPU 1 since that is on the same core. > You may want to look at: https://lists.freebsd.org/pipermail/freebsd-hackers/2019-March/054218.html for cruder, but somewhat related, information for the old Powermac G5 2-socket with 2 cores each, given how FreeBSD tries to synchronize the tbr's across cores as it starts up the CPUs. It may give some idea of a ball-park scale involved for such context, especially the reports of what happened for varying one figure in the source code. As stands, I've only done the experiments with a debug kernel build. I built using devel/powerpc64-xtoolchain-gcc related infrastructure, not gcc 4.2.1 . (This is typical for me.) === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)