From owner-freebsd-current@freebsd.org Wed Jun 27 17:06:21 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2CF82102D78F for ; Wed, 27 Jun 2018 17:06:21 +0000 (UTC) (envelope-from asomers@gmail.com) Received: from mail-lj1-x22d.google.com (mail-lj1-x22d.google.com [IPv6:2a00:1450:4864:20::22d]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 89A9987321; Wed, 27 Jun 2018 17:06:20 +0000 (UTC) (envelope-from asomers@gmail.com) Received: by mail-lj1-x22d.google.com with SMTP id a6-v6so2218283ljj.7; Wed, 27 Jun 2018 10:06:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:sender:in-reply-to:references:from:date:message-id :subject:to:cc; bh=x6OEF2U6mBR49GHZweGF3GdhgIRB5wkYk0AGkGIkFZI=; b=PJwFIdsYRwd1sKCxlyFA/2Skoma8T19gaVH7Y7JMGukPYyTeVMgGluWpGb7SPYBOn6 owCBpZPqPlf3gyOCjm0SfbM44C7N+hCiKaJuuY28EZ+Ol2DDMIlm1NQmtiMhyW+JXREk xqKPs43aIjm1x8zufsZgDrc/DJESEfQJQfrQgF0QBUXWEhOTaEndZRfVGyXtxwYE0p9R kBAI0tySLgETJq0j+yP7YoUjEnt/kN63GOiO8/y78wXv/42703zWOQRip2nqCkt14FS2 z7dI5HxhiPVfAoassEhlPojFX9mP/eYyUPOnqKPJ8aD2Hef0BV4QhCBDflsmqG6iRNV4 OszA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:sender:in-reply-to:references:from :date:message-id:subject:to:cc; bh=x6OEF2U6mBR49GHZweGF3GdhgIRB5wkYk0AGkGIkFZI=; b=eu8oAT+RLd4qxGQwgr/a3DcNb10R7/cUHmyAkXXDmZOMwPIBkCsqFKnSXXGjjGiMda 2hf2unvzobd0s9XahHzpXKCX5kG2khwkLM9HuDRUzkdHFL39jo7ZTJT51YXZu0E4FDkM M8628kjNWugCHxuJWbW9A0zquCSGqFTmd9iITc7tnK02aZNIviMab9MazJSjsW5QrOY3 ZaappL5lsPhyPHkwI5mu5Ci5QwMK0O0tWC5ttHWdcCy2SQsHjufaQxgDPql0J2lVgOGY FvWyuAlcIOrAMShT0f9WW38BlU/zohyNbH47gMfTJk9V5N30oI01yMMeoc7zGRGoU1T9 rXWw== X-Gm-Message-State: APt69E2i5bn/blToUUU8wtfQrG15XDxZOIv2kg2Z2b+DCllSt4F/dbzu uILMSc+NnrVgSFYvNk+mZXijyyC6mNAqS0OB68U= X-Google-Smtp-Source: AAOMgpfxD7G+0yGB3v+6qSL8Cc24AH+AjH4TvaLfCtO35JGQKpfU5qdhd5hZtRf3HDx3vTN6EP1Ctj36k0k8/eKrXq8= X-Received: by 2002:a2e:29da:: with SMTP id p87-v6mr2907356ljp.12.1530119178950; Wed, 27 Jun 2018 10:06:18 -0700 (PDT) MIME-Version: 1.0 Sender: asomers@gmail.com Received: by 2002:ab3:1b91:0:0:0:0:0 with HTTP; Wed, 27 Jun 2018 10:06:18 -0700 (PDT) In-Reply-To: <0984f859-bce2-3dcb-40ba-58bb61598e1b@FreeBSD.org> References: <8ac353c5-d188-f432-aab1-86f4ca5fd295@FreeBSD.org> <4d7957f6-9497-19ff-4dbb-436bb6b05a56@FreeBSD.org> <0984f859-bce2-3dcb-40ba-58bb61598e1b@FreeBSD.org> From: Alan Somers Date: Wed, 27 Jun 2018 11:06:18 -0600 X-Google-Sender-Auth: 80ULbhP0dX6zXehV0aknJ1RWav4 Message-ID: Subject: Re: TSC calibration in virtual machines To: Jung-uk Kim Cc: Andriy Gapon , FreeBSD Current Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.26 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Jun 2018 17:06:21 -0000 On Wed, Jun 27, 2018 at 11:05 AM, Jung-uk Kim wrote: > On 06/27/2018 12:47, Alan Somers wrote: > > On Wed, Jun 27, 2018 at 10:36 AM, Jung-uk Kim > > wrote: > > > > On 06/27/2018 03:14, Andriy Gapon wrote: > > > > > > It seems that TSC calibration in virtual machines sometimes can do > more harm > > > than good. Should we default to trusting the information provided > by a hypervisor? > > > > > > Specifically, I am observing a problem on GCE instances where > calibrated TSC > > > frequency is about 10% lower than advertised frequency. And > apparently the > > > advertised frequency is the right one. > > > > > > I found this thread with similar reports and a variety of > workarounds from > > > administratively disabling the calibration to switching to a > different timecounter: > > > https://lists.freebsd.org/pipermail/freebsd-cloud/2017- > January/000080.html > > January/000080.html> > > > > We already do that for VMware hosts since r221214. > > > > https://svnweb.freebsd.org/changeset/base/221214 > > > > > > We should do the same for each hypervisor. > > > > We probably should. But why does calibration fail in the first place? > Because multiple guests are sharing same physical CPUs and guest OS has > no control, timing cannot be 100% accurate. > > > If it can fail in a VM, then it can probably fail on bare metal too. It > > would be worth investigating. > It does not "fail" in bare metal because we have almost complete control. > > Jung-uk Kim > > Makes sense. I didn't realize that it ran before the scheduler or interrupts were started.