From owner-svn-src-head@freebsd.org Thu Mar 24 09:09:26 2016 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6F4B4ADAE7A; Thu, 24 Mar 2016 09:09:26 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 195ED176D; Thu, 24 Mar 2016 09:09:25 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id u2O99J5b061993 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Thu, 24 Mar 2016 11:09:19 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua u2O99J5b061993 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id u2O99HCB061992; Thu, 24 Mar 2016 11:09:17 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 24 Mar 2016 11:09:17 +0200 From: Konstantin Belousov To: John Baldwin Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org, "'rstone@freebsd.org'" Subject: Re: svn commit: r297039 - head/sys/x86/x86 Message-ID: <20160324090917.GC1741@kib.kiev.ua> References: <201603181948.u2IJmndg063765@repo.freebsd.org> <1866602.Bp7VFd5f42@ralph.baldwin.cx> <20160323075842.GX1741@kib.kiev.ua> <2922763.uITxoCVqGR@ralph.baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2922763.uITxoCVqGR@ralph.baldwin.cx> User-Agent: Mutt/1.5.24 (2015-08-30) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Mar 2016 09:09:26 -0000 On Wed, Mar 23, 2016 at 02:21:59PM -0700, John Baldwin wrote: > As you noted, the issue is if a timecounter needs locks (e.g. i8254) though > outside of that I think the patch is great. :-/ Of course, if the TSC > isn't advertised as invariant, DELAY() is talking to the timecounter > directly as well. > > However, I think we probably can use the TSC. The only specific note I got > from Ryan (cc'd) was about the TSC being unstable as a timecounter under KVM. > That doesn't mean that the TSC is non-mononotic on a single vCPU. In fact, > thinking about this more I have a different theory to explain how the TSC > can be out of whack on different vCPUs even if the hardware TSC is in sync > in the physical CPUs underneath. In fact, if we can use TSC with the only requirement of being monotonic, I do not see why do we need TSC at all. We can return to pre-r278325 loop, but calibrate the number of loop iterations for known delay in 1us, once on boot. Do you agree with this ? > > One of the things present in the VCMS on Intel CPUs using VT-x is a TSC > adjustment. The hypervisor can alter this TSC adjustment during a VM-exit to > alter the offset between the TSC in the guest and the "real" TSC value in the > physical CPU itself. One way a hypervisor might use this is to try to > "pause" the TSC during a VM-exit by taking TSC timestamps at the start and > end of a VM-exit and adding that delta to the TSC offset just before each > VM-entry. However, if you have two vCPUs, one of which is running in the > guest and one of which is handling a VM-exit in the hypervisor, the TSC on > the first vCPU will run while the effective TSC of the second vCPU is paused. > When the second vCPU resumes after a VM-entry, it's TSC will now "unpause", > but it will lag the first vCPU by however long it took to handle its VM-exit. > > It wouldn't surprise me if KVM was doing this. bhyve does not do this to my > knowledge (so the TSC is actually still usable as a timecounter under bhyve > for some value of "usable"). However, even with this TSC pausing/unpausing, > the TSC would still increase monotonically on a single vCPU. For the purposes > of DELAY() (and other spin loops on a pinned thread such as in > lapic_ipi_wait()), that is all you need. BTW, Intel exported this mechanism to the non-VT environment as well, on recent chips. So I would be not too surprised if SMM handlers start 'compensate' for some long delays in near future.