From owner-freebsd-virtualization@freebsd.org Tue Oct 24 22:14:11 2017 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DE034E30314 for ; Tue, 24 Oct 2017 22:14:11 +0000 (UTC) (envelope-from ian@freebsd.org) Received: from outbound1b.ore.mailhop.org (outbound1b.ore.mailhop.org [54.200.247.200]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id B6DF5731D1 for ; Tue, 24 Oct 2017 22:14:11 +0000 (UTC) (envelope-from ian@freebsd.org) X-MHO-User: b36b2837-b908-11e7-a938-4f970e858fdb X-Report-Abuse-To: https://support.duocircle.com/support/solutions/articles/5000540958-duocircle-standard-smtp-abuse-information X-Originating-IP: 73.78.92.27 X-Mail-Handler: DuoCircle Outbound SMTP Received: from ilsoft.org (unknown [73.78.92.27]) by outbound1.ore.mailhop.org (Halon) with ESMTPSA id b36b2837-b908-11e7-a938-4f970e858fdb; Tue, 24 Oct 2017 22:14:28 +0000 (UTC) Received: from rev (rev [172.22.42.240]) by ilsoft.org (8.15.2/8.15.2) with ESMTP id v9OME8TV002019; Tue, 24 Oct 2017 16:14:08 -0600 (MDT) (envelope-from ian@freebsd.org) Message-ID: <1508883248.34364.52.camel@freebsd.org> Subject: Re: host, bhyve vm and ntpd From: Ian Lepore To: Boris Samorodov , Michael Voorhis Cc: freebsd-current@FreeBSD.org, freebsd-virtualization@FreeBSD.org Date: Tue, 24 Oct 2017 16:14:08 -0600 In-Reply-To: References: <2931f1cc-6574-b58d-4b94-5f77fa5cdb85@passap.ru> <1508512327.1383.55.camel@freebsd.org> <39bf2426-2edf-d485-7c81-519e931154be@passap.ru> <1508517160.1383.63.camel@freebsd.org> <76ff7afb-3d3a-96f6-1275-89472ff5683d@passap.ru> <1508522667.1383.69.camel@freebsd.org> <30992c14-7b78-ab9f-5693-931e6ca41f1b@passap.ru> <1508523696.1383.75.camel@freebsd.org> <23019.46875.929719.481108@atom.mcvau.net> <1508624153.1383.107.camel@freebsd.org> Content-Type: text/plain; charset="koi8-r" X-Mailer: Evolution 3.18.5.1 FreeBSD GNOME Team Port Mime-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Oct 2017 22:14:12 -0000 On Wed, 2017-10-25 at 00:43 +0300, Boris Samorodov wrote: > Hi Ian, All! > > 22.10.2017 01:15, Ian Lepore ÐÉÛÅÔ: > > > > On Sat, 2017-10-21 at 17:07 -0400, Michael Voorhis wrote: > > > > > > Ian Lepore writes: > > > > > > > > > > > > Beyond that, I'm not sure what else to try. šIt might be necessary to > > > > get some bhyve developers involved (I know almost nothing about it). > > > NTPD behaves more normally on uniprocessor VMs. > > > > > > A FreeBSD bhyve-guest running on a freebsd host will select a > > > different timecounter depending on whether it is a multiprocessor or a > > > uniprocessor.ššMy uniprocessor bhyve-vm selected TSC-low as the best > > > timecounter in a uniprocessor.ššNTP functions there as expected. > > > > > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000) > > > kern.timecounter.hardware: TSC-low > > > > > > The very same VM, when given two total CPUs, selected HPET (if I > > > recall) and the timekeeping with NTPD was unreliable, with many > > > step-resets to the clock. > > > > > Hmm, I just had glance at the code inšsys/amd64/vmm/io/vhpet.c and it > > looks right. šI wonder if this is just a simple roundoff error in > > converting between 10.0MHz and SBT units? šIf so, that could be wished > > away easily by using a power-of-2 frequency for the virtual HPET. šI > > wonder if the attached patch is all that's needed? > I suppose the answer is "yes", the patch helped. Here are two samples > (host for bhyve VM without your patch and after patching): > --- > https://poudriere.passap.ru/misc/ntpd.jot.log-HPET.frequency.10000000.txt > https://poudriere.passap.ru/misc/ntpd.jot.log-HPET.frequency.16777216.txt > --- > > The command was: > % for t in `jot 1000`; do ntpq -pn; sleep 64; done > > The patch made the system to stabilize the process. > Ian, thank you! > Hmmm. šThe startup behavior wasn't great, it took a long time and several clock steps for it to figure out the frequency error and get the clock under control. šBut, as you say, it did eventually stabilize this time. Can you show the /var/db/ntpd.drift file contents of the host and guest? šIdeally, now that it's stable, the two values should be very close. šIf they're not, maybe this isn't the right fix. -- Ian