From owner-freebsd-current@freebsd.org Sun Oct 22 15:38:59 2017 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69623E54FA1; Sun, 22 Oct 2017 15:38:59 +0000 (UTC) (envelope-from bsam@passap.ru) Received: from forward106j.mail.yandex.net (forward106j.mail.yandex.net [IPv6:2a02:6b8:0:801:2::109]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "forwards.mail.yandex.net", Issuer "Yandex CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 02C126A573; Sun, 22 Oct 2017 15:38:58 +0000 (UTC) (envelope-from bsam@passap.ru) Received: from mxback13g.mail.yandex.net (mxback13g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:92]) by forward106j.mail.yandex.net (Yandex) with ESMTP id 273BB18056D0; Sun, 22 Oct 2017 18:38:46 +0300 (MSK) Received: from smtp3p.mail.yandex.net (smtp3p.mail.yandex.net [2a02:6b8:0:1472:2741:0:8b6:8]) by mxback13g.mail.yandex.net (nwsmtp/Yandex) with ESMTP id ZKbttz1MG3-cjvKQAvG; Sun, 22 Oct 2017 18:38:46 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=passap.ru; s=mail; t=1508686726; bh=FYZqtmDoaEhbr53uk4NqahytsU9FFTmtNB/8unK74IM=; h=Subject:To:Cc:References:From:Message-ID:Date:In-Reply-To; b=KFpevbm7iZ5n75/+Mhspd9NnH149MZoIqgnEG+LXERsrpOdWqsvNMWzwOfoWEToOj l0YBIRsCVoPeISxZLJyKOmHQGyQ4ZPC/Sdnt98kgZ5x3+kr0eO1EDMmfXShYIE/cLf JYW15JSwwlgcs0slZ77VN0m5d6Ovx8XNYdxLjefA= Received: by smtp3p.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id hCirSmNIQ7-cihSflGe; Sun, 22 Oct 2017 18:38:44 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client certificate not present) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=passap.ru; s=mail; t=1508686724; bh=FYZqtmDoaEhbr53uk4NqahytsU9FFTmtNB/8unK74IM=; h=Subject:To:Cc:References:From:Message-ID:Date:In-Reply-To; b=qEqlZvL7NsLnM5PfXTiPvqW75XcQWk+ZuOZWlC9WxfJPrTAYD9uMwOEpPyMA9bfIG OA9Glq8XVAIATvBoYpMeqEtbmZHqBwZaB04kmHYemR31+vxBegQ/6t5DBrvLIQYJjk KMU+uSrJoHnyYNv1ivni5UXsuECa/Ly0u6PLFgS4= Authentication-Results: smtp3p.mail.yandex.net; dkim=pass header.i=@passap.ru Subject: Re: host, bhyve vm and ntpd To: "Rodney W. Grimes" Cc: Ian Lepore , Michael Voorhis , freebsd-current@freebsd.org, freebsd-virtualization@freebsd.org References: <201710221522.v9MFMGi6039118@pdx.rh.CN85.dnsmgr.net> From: Boris Samorodov Message-ID: <6ddef049-3e24-5cac-cccc-d99db9a835af@passap.ru> Date: Sun, 22 Oct 2017 18:38:44 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <201710221522.v9MFMGi6039118@pdx.rh.CN85.dnsmgr.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Oct 2017 15:38:59 -0000 22.10.2017 18:22, Rodney W. Grimes пишет: > [ Charset UTF-8 unsupported, converting... ] >> 22.10.2017 01:15, Ian Lepore ?????: >>> On Sat, 2017-10-21 at 17:07 -0400, Michael Voorhis wrote: >>>> Ian Lepore writes: >>>>> >>>>> Beyond that, I'm not sure what else to try. ?It might be necessary to >>>>> get some bhyve developers involved (I know almost nothing about it). >>>> NTPD behaves more normally on uniprocessor VMs. >>>> >>>> A FreeBSD bhyve-guest running on a freebsd host will select a >>>> different timecounter depending on whether it is a multiprocessor or a >>>> uniprocessor.??My uniprocessor bhyve-vm selected TSC-low as the best >>>> timecounter in a uniprocessor.??NTP functions there as expected. >>>> >>>> kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) dummy(-1000000) >>>> kern.timecounter.hardware: TSC-low >>>> >>>> The very same VM, when given two total CPUs, selected HPET (if I >>>> recall) and the timekeeping with NTPD was unreliable, with many >>>> step-resets to the clock. >>>> >>> >>> Hmm, I just had glance at the code in?sys/amd64/vmm/io/vhpet.c and it >>> looks right. ?I wonder if this is just a simple roundoff error in >>> converting between 10.0MHz and SBT units? ?If so, that could be wished >>> away easily by using a power-of-2 frequency for the virtual HPET. ?I >>> wonder if the attached patch is all that's needed? >> I've tried the patch (at bhyve guest) and nothing has changed. Should >> the patched system be tested at bhyve guest or bhyve host? > > I believe the suggested patch would have to be made to the bhyve > host OK, I'd do it tomorrow and report back. >. Also on the host and guest what are the values of > sysctl kern.timecounter.tc.HPET > sysctl kern.timecounter.tc.i8254 Here they are: --- bhyve-host% sysctl kern.timecounter.tc.HPET kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 14318180 kern.timecounter.tc.HPET.counter: 2138094157 kern.timecounter.tc.HPET.mask: 4294967295 bhyve-host% sysctl kern.timecounter.tc.i8254 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 54883 kern.timecounter.tc.i8254.mask: 65535 --- bhyve-guest% sysctl kern.timecounter.tc.HPET kern.timecounter.tc.HPET.quality: 950 kern.timecounter.tc.HPET.frequency: 10000000 kern.timecounter.tc.HPET.counter: 969429421 kern.timecounter.tc.HPET.mask: 4294967295 bhyve-guest% sysctl kern.timecounter.tc.i8254 kern.timecounter.tc.i8254.quality: 0 kern.timecounter.tc.i8254.frequency: 1193182 kern.timecounter.tc.i8254.counter: 39893 kern.timecounter.tc.i8254.mask: 65535 --- > Getting good ntpd behavior in a VM guest of any kind is sometimes a > non trivial thing to do. As a side note, I have a CentOS-7 bhyve VM at the same host. And it was enough to run chronyd with default config. Which stepped twice and is stable (no messages) for several days, current log: --- Oct 19 16:01:03 c.vpn systemd[1]: Starting NTP client/server... Oct 19 16:01:03 c.vpn chronyd[27043]: chronyd version 3.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SECHASH +SIGND +ASYNCDNS +IPV6 +DEBUG) Oct 19 16:01:03 c.vpn chronyd[27043]: Frequency 0.000 +/- 1000000.000 ppm read from /var/lib/chrony/drift Oct 19 16:01:03 c.vpn systemd[1]: Started NTP client/server. Oct 19 16:01:07 c.vpn chronyd[27043]: Selected source XX.XX.XX.1 Oct 19 16:01:07 c.vpn chronyd[27043]: System clock wrong by -44.392782 seconds, adjustment started Oct 19 16:00:23 c.vpn chronyd[27043]: System clock was stepped by -44.392782 seconds Oct 19 16:00:34 c.vpn chronyd[27043]: System clock was stepped by 0.000001 seconds --- -- WBR, bsam