From owner-freebsd-hackers@freebsd.org Fri Apr 5 13:21:38 2019 Return-Path: Delivered-To: freebsd-hackers@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AAA831576E5C; Fri, 5 Apr 2019 13:21:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 1F96490C20; Fri, 5 Apr 2019 13:21:38 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x35DLSfb011961 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO); Fri, 5 Apr 2019 16:21:31 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x35DLSfb011961 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x35DLSX2011960; Fri, 5 Apr 2019 16:21:28 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Fri, 5 Apr 2019 16:21:28 +0300 From: Konstantin Belousov To: Bruce Evans Cc: Michael Tuexen , freebsd-hackers Hackers , FreeBSD PowerPC ML Subject: Re: powerpc64 head -r344018 stuck sleeping problems: th->th_scale * tc_delta(th) overflows unsigned 64 bits sometimes [patched failed] Message-ID: <20190405132128.GD1923@kib.kiev.ua> References: <20190306172003.GD2492@kib.kiev.ua> <20190308001005.M2756@besplex.bde.org> <20190307222220.GK2492@kib.kiev.ua> <20190309144844.K1166@besplex.bde.org> <20190324110138.GR1923@kib.kiev.ua> <20190403070045.GW1923@kib.kiev.ua> <20190404011802.E2390@besplex.bde.org> <20190405113912.GB1923@kib.kiev.ua> <20190405230717.D3383@besplex.bde.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20190405230717.D3383@besplex.bde.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Apr 2019 13:21:38 -0000 On Fri, Apr 05, 2019 at 11:52:27PM +1100, Bruce Evans wrote: > On Fri, 5 Apr 2019, Konstantin Belousov wrote: > > > On Thu, Apr 04, 2019 at 02:47:34AM +1100, Bruce Evans wrote: > >> I noticed (or better realized) a general problem with multiple > >> timehands. ntpd can slew the clock at up to 500 ppm, and at least an > >> old version of it uses a rate of 50 ppm to fix up fairly small drifts > >> in the milliseconds range. 500 ppm is enormous in CPU cycles -- it is > >> 500 thousand nsec or 2 million cycles at 4GHz. Winding up the timecounter > >> every 1 msec reduces this to only 2000 cycles. > >> ... > >> The main point of having multiple timehands (after introducing the per- > >> timehands generation count) is to avoid blocking thread N during the > >> update, but this doesn't actually work, even for only 2 timehands and > >> a global generation count. > > > > You are describing the generic race between reader and writer. The same > > race would exist even with one timehand (and/or one global generation > > counter), where ntp adjustment might come earlier or later of some > > consumer accessing the timehands. If timehand instance was read before > > tc_windup() run but code consumed the result after the windup, it might > > appear as if time went backward, and this cannot be fixed without either > > re-reading the time after time-depended calculations were done and > > restarting, or some globabl lock ensuring serialization. > > With 1 timehand, its generation count would be global. I think its ordering > is strong enough to ensure serialization. Yes, single timehands result in global generation. But it would not solve the same race appearing in slightly different manner, as I described above. If reader finished while generation number in th was not yet reset, but caller uses the result after tc_windup(), the effect is same as if we have two th's and reader used the outdated one. > > I think the fix in the kernel to use a global generation count (with > 1 > timehands) is simply s/th->th_generation/tc_generation/g. Oops, that > makes multiple timehands useless and gives some blocking. The critical > case is when a new timehands is under construction. The old timehands > becomes unsafe to use when the writer (tc_windup()) updates the offset. > tc_windup() currently sets th_generation to 0 to indicate that the new > timehands is unsafe to use. Doing the same with a global tc_generation > would give serialization at the cost of busy-waiting for tc_generation > to become nonzero. It would indicate that all timehands are unsafe > to use. > > In the library, does it just work to put the global generation count in > the shared page? libc always reload tk_current in the loop, so it works with any number of vdso timehands greater or equal to one.