From owner-freebsd-current@freebsd.org Wed Jun 27 17:05:41 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8C6B4102D6CE for ; Wed, 27 Jun 2018 17:05:41 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 0944D87241; Wed, 27 Jun 2018 17:05:40 +0000 (UTC) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w5RH5WLp009270; Wed, 27 Jun 2018 10:05:32 -0700 (PDT) (envelope-from freebsd-rwg@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd-rwg@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w5RH5WZ7009269; Wed, 27 Jun 2018 10:05:32 -0700 (PDT) (envelope-from freebsd-rwg) From: "Rodney W. Grimes" Message-Id: <201806271705.w5RH5WZ7009269@pdx.rh.CN85.dnsmgr.net> Subject: Re: TSC calibration in virtual machines In-Reply-To: To: Alan Somers Date: Wed, 27 Jun 2018 10:05:32 -0700 (PDT) CC: Jung-uk Kim , Andriy Gapon , FreeBSD Current X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jun 2018 17:05:41 -0000 > On Wed, Jun 27, 2018 at 10:36 AM, Jung-uk Kim wrote: > > > On 06/27/2018 03:14, Andriy Gapon wrote: > > > > > > It seems that TSC calibration in virtual machines sometimes can do more > > harm > > > than good. Should we default to trusting the information provided by a > > hypervisor? > > > > > > Specifically, I am observing a problem on GCE instances where calibrated > > TSC > > > frequency is about 10% lower than advertised frequency. And apparently > > the > > > advertised frequency is the right one. > > > > > > I found this thread with similar reports and a variety of workarounds > > from > > > administratively disabling the calibration to switching to a different > > timecounter: > > > https://lists.freebsd.org/pipermail/freebsd-cloud/2017- > > January/000080.html > > > > We already do that for VMware hosts since r221214. > > > > https://svnweb.freebsd.org/changeset/base/221214 > > > > We should do the same for each hypervisor. > > > > Jung-uk Kim > > > > > We probably should. But why does calibration fail in the first place? If > it can fail in a VM, then it can probably fail on bare metal too. It would > be worth investigating. No, the failure in a VM is unique to a VM, it has to do with the fact your have the hypervisor timeslicing a CPU that you believe to be 100% dedicated to you. There are several white papers, including one from VMWare about what they have done to help with the time keeping problems. What is suggested above would be a correct thing to do. Bhyve creates these issues as well, and use of certain timers in a bhyve guest can cause you nightmares with ntp. -- Rod Grimes rgrimes@freebsd.org