From owner-freebsd-stable@FreeBSD.ORG Wed Jun 6 21:27:39 2007 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C0A6B16A41F for ; Wed, 6 Jun 2007 21:27:39 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.freebsd.org (Postfix) with ESMTP id AA98E13C465 for ; Wed, 6 Jun 2007 21:27:39 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.13.8/8.13.7) with ESMTP id l56LRYD6090138; Wed, 6 Jun 2007 14:27:34 -0700 (PDT) Received: (from dillon@localhost) by apollo.backplane.com (8.13.8/8.13.4/Submit) id l56LRYTe090137; Wed, 6 Jun 2007 14:27:34 -0700 (PDT) Date: Wed, 6 Jun 2007 14:27:34 -0700 (PDT) From: Matthew Dillon Message-Id: <200706062127.l56LRYTe090137@apollo.backplane.com> To: Dmitry Morozovsky References: <20070606153542.Y76617@woozle.rinet.ru> <20070606231940.T91939@woozle.rinet.ru> Cc: freebsd-stable@freebsd.org, Ivan Voras Subject: Re: calcru: runtime went backwards, RELENG_6, SMP X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Jun 2007 21:27:39 -0000 :IV> > Upd: on GENERIC/amd64 kernel I got the same errors. :IV> :IV> Do you perhaps run with TSC timecounter? (that's the only cause I've notice :IV> that can generate this message). : :Nope: : :marck@ct-new:~> sysctl kern.timecounter :kern.timecounter.tick: 1 :kern.timecounter.choice: TSC(-100) ACPI-fast(1000) i8254(0) dummy(-1000000) :kern.timecounter.hardware: ACPI-fast :... kgdb your live kernel and 'print cpu_ticks'. See what the cpu ticker is actually pointing at, because it might not be the time counter. It could still be TSC. The TSC isn't synchronized between the cores on a SMP box, not even on multi-core parts. It can't be used to calculate delta times for any thread that has the possibility of migrating between cpu's. Not only will the absolute offset be off between cpus, but the frequency will also be slightly different (at least on SMP multi-core parts), so you get frequency drift too. There is also possibly an issue with tc_cpu_ticks(), which seems to be using a static 64 bit variable to handle rollover instead of a per-cpu variable. I don't see how that could possibly be MP safe, especially if the timecount is not synchronized between cpus and causes multiple rollover events. In fact, I can *barely* use the TSC on DragonFly for KTR logging, and even then I have to have some kernel threads sitting there doing nothing but figuring out the drift between the cpus so it can correct the TSC values when it logs information... and even with all of that I can't get them synchronized any closer then around 500ns from each other. I'd recommend that FreeBSD do what we did years ago with calcru ... stop trying to calculate the time down to the nanosecond and just do it statistically. It works just fine and takes the whole mess out of the critical path. -Matt