From owner-freebsd-hackers@freebsd.org Wed Mar 13 21:47:45 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B36CF1542D57 for ; Wed, 13 Mar 2019 21:47:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) Received: from sonic303-22.consmr.mail.ne1.yahoo.com (sonic303-22.consmr.mail.ne1.yahoo.com [66.163.188.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 30AB271E00 for ; Wed, 13 Mar 2019 21:47:45 +0000 (UTC) (envelope-from marklmi@yahoo.com) X-YMail-OSG: njhf4UMVM1konPYelGtCvG2CLqQbU2S165OHROnVAP9wg9HyjO51PkO4DNHPCzI 502D0uLj2YBDLZmRA64XBORhlQJw5Ghxwr2gLhSzvHb_gQL3VRU3lGMQiIEzuLn_SzfxW1etQXNW yIVlJDl6uWdELzd2j1r83.q0QvvLd0k3IwatDyRvhvfKaxv3NWSYMLzY8brSfeWi9amknabxYt.f RGKf2eIYU4WPvXQ9Zp3RrrndCSssCZuXEDikRFwPma0E1AzP1ygGxw3NMHG74HmWbiBSiRxI8XfW nsygbRMt93oqyoITKZ0peYNCB_YteN1CdB3tZ53QUMHJlNsn44fEG.75fxhlCmUBhukHw5sUVe7X _JdU.R40EsJEWnbMANXaBI.s6YQStGfcG21wFDdARTsnTmBDCB9NuDXE6hcaGzEZNeY7nfX4P3GS MTQUSARJBRzaxQ8V2vWeQP7xHTLITF1C7mir4XZmYAbw3Ut4lzODVY1Y9z.Pzs7dq_rAzOJ4InyK nz49Ir5bbQI8gaF11U.j.bD7Vvj_sQLOYdxmz92sPGA1qy0Bjpx0j2DETPgtBKnua48EOoBbaKV8 rkQRaUq1zn_4aKNYEdB9312tIFxpiBNOy9IXSlvpUXpqQ4TYjpVYDgyK.UJP5Q92qXFjpt9y6OkB L8bO8IjmtHdAQMC_5YTWp4GC.PTqBHnNdxmhpZdQkLk2HEev9Rm6f44IjBBwDqdAH6uMQg6qyE2O e9Wa_yQoPnuQYw4baEiRkTrTmNnmsVMBCllD9jusuuAVuauUqmngUd2oExmBBAx.Jk6RlrdQod_9 CMpr.e.2nSxYibH29sKLyN3WXrVm.lnB_kG3naWAED6TFUTv2rx9HtZLCEMnN5cyoDd0OL9SPElN x1AQ_ZdG8p7tw47KkugHcVuUM37Yq8n5uJ8bOAXyRyxvgEeriaJLIJWm8U.k7xbWCLodywlk.Ocf V18.t7SfBDwwkqwCF4.GMO3cYMxu8Fsl6GhT08K7PtDUon7iHZOPstheXTNtw9488k0CA3nw89Go rQse7h_WjkxY0YjmaBF5vrMYS38APcIPTD9Ef7md2lgHfeud1Crld._gOAuRU3OrfjvONKOi2t8M rvYiXNAM- Received: from sonic.gate.mail.ne1.yahoo.com by sonic303.consmr.mail.ne1.yahoo.com with HTTP; Wed, 13 Mar 2019 21:47:38 +0000 Received: from c-67-170-167-181.hsd1.or.comcast.net (EHLO [192.168.1.113]) ([67.170.167.181]) by smtp409.mail.ne1.yahoo.com (Oath Hermes SMTP Server) with ESMTPA ID 58284d86e6b91e2c7d115888f3682c37; Wed, 13 Mar 2019 21:47:37 +0000 (UTC) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 12.2 \(3445.102.3\)) Subject: Re: TSC "skew" (was: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed]) From: Mark Millard In-Reply-To: <20190313190558.GB2492@kib.kiev.ua> Date: Wed, 13 Mar 2019 14:47:35 -0700 Cc: Bruce Evans , freebsd-hackers Hackers , FreeBSD PowerPC ML Content-Transfer-Encoding: 7bit Message-Id: <3C3486AE-DA3A-49DF-BAA5-139D4E99FADB@yahoo.com> References: <20190302225513.W3408@besplex.bde.org> <20190302142521.GE68879@kib.kiev.ua> <20190303041441.V4781@besplex.bde.org> <20190303111931.GI68879@kib.kiev.ua> <20190303223100.B3572@besplex.bde.org> <20190303161635.GJ68879@kib.kiev.ua> <20190304043416.V5640@besplex.bde.org> <20190304114150.GM68879@kib.kiev.ua> <20190305031010.I4610@besplex.bde.org> <20190305223415.U1563@besplex.bde.org> <20190313190558.GB2492@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.3445.102.3) X-Rspamd-Queue-Id: 30AB271E00 X-Spamd-Bar: ------ X-Spamd-Result: default: False [-6.99 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; REPLY(-4.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.992,0] X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Mar 2019 21:47:46 -0000 On 2019-Mar-13, at 12:05, Konstantin Belousov wrote: >> . . . > I am not sure I follow. MFENCE is documented by wording that implies, > without any doubts, that store buffers are flushed before the > instruction is retired. It is not so obvious for SFENCE, which > sounds like a real fence instead of full flush, at least for normal > write-back memory where it is NOP as far as ISA is considered. > > It is known and documented in optimization manuals that locked > operations are much more efficient, but locked ops are only documented > to ensure ordering and not flush. So SFENCE is not suitable as our > barrier. What I've seen in papers for the C++ Load/Store Seq Cst mappings to processors is: For write-fencing style: Load Seq Cst: MOV from memory Store Seq Cst alternative 0: XCHG (which as an implicit lock prefix) Store Seq Cst alternative 1: MOV into memory; MFENCE For read-fencing style: Load Seq Cst alternative 0: LOCK XADD(0) Load Seq Cst alternative 1: MFENCE; MOV from memory Store Seq Cst: MOV into memory There is also: Seq Cst Fence: MFENCE I've never seen SFENCE (or LFENCE) suggested for any of the above. I would expect for C++ Seq Cst that the XCHG and the LOCK XADD(0) would need to flush store buffers in order for those alternatives to be valid for C++ Seq Cst. I've seen a reference to a "locked identity operation to the top of stack" as another form of locked style of Seq Cst fencing. (write-fencing and read-fencing can not be generally mixed for Seq Cst: they do not inter-operate.) > And, the second point, LFENCE there does not work as barrier for IPC. > It only ensures that RDTSC is not started earlier than the previous > instructions. No store buffer flushing is done. === Mark Millard marklmi at yahoo.com ( dsl-only.net went away in early 2018-Mar)