From owner-freebsd-virtualization@freebsd.org Tue Apr 26 13:50:57 2016 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D5F33B1D76B for ; Tue, 26 Apr 2016 13:50:57 +0000 (UTC) (envelope-from ifebi@schema31.it) Received: from stricnina.roma.schema31.it (stricnina.schema31.it [2.228.74.189]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "stricnina.roma.schema31.it", Issuer "stricnina.roma.schema31.it" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 562D310EF for ; Tue, 26 Apr 2016 13:50:56 +0000 (UTC) (envelope-from ifebi@schema31.it) Received: from smtp.schema31.it (localhost [127.0.0.1]) by stricnina.roma.schema31.it (8.14.9/8.14.9) with ESMTP id u3QDkNgU098819; Tue, 26 Apr 2016 15:46:24 +0200 (CEST) (envelope-from ifebi@schema31.it) Subject: Re: bhyve clock issue To: Neel Natu References: <8EE098BA-6C8F-41F4-B9CD-88F3D587B10F@gmail.com> Cc: abrancatelli@schema31.it, "freebsd-virtualization@freebsd.org" From: Ilario Febi Message-ID: <571F7127.1050100@schema31.it> Date: Tue, 26 Apr 2016 15:46:15 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.7.2 MIME-Version: 1.0 In-Reply-To: <8EE098BA-6C8F-41F4-B9CD-88F3D587B10F@gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Content-Filtered-By: Mailman/MimeDel 2.1.21 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Apr 2016 13:50:57 -0000 Yes, I found errors in /var/log/messages and fix adding: kern.timecounter.hardware=TSC-low into /etc/sysctl.conf file Thanks for support. Il 14/04/16 18:13, Neel Natu ha scritto: > Hi, > > Did you see anything interesting in the ntpd log file around the time the large skew happened? > > Best > Neel > >> On Apr 13, 2016, at 3:44 AM, Ilario Febi wrote: >> >> Hi All >> >> I have a strange issue about the clock of some FreeBSD Bhyve guests, >> maybe someone of you can help me to diagnose it and work on a >> solution. >> >> In detail, sometimes (without an apparent regularity) one of four >> bhyve guest's clock goes exactly 7 minutes and 10 seconds in the >> future. >> >> This clock stays unaligned for about 20 minutes, then, without any >> human intervention, the clock realign to proper condition. >> >> My configuration is: >> >> envrm02 (bhyve HOST, 10.2-RELEASE-p14) >> -> og-qa (bhyve guest - 10.2-RELEASE-p14) >> -> og-ci (bhyve guest - 10.2-RELEASE-p14) >> -> og-test (bhyve guest - 10.2-RELEASE-p14) >> -> og-prod-rm (bhyve guest - 10.2-RELEASE-p14) >> All the og-* machines just have a running copy of tomcat. >> >> I built a simple test to catch the error with python/fabric... the test is: >> >> fab -R ENV,OG,OG-PROD -- date | awk -F ': ' '/2016/{print $2, $1}' >> >> which simply run the command "date" on all hosts. >> >> So, the result is: >> @Mac-mini ~/p/f/> ./check_clock.sh >> Thu Apr 7 17:38:00 CEST 2016 [envrm02] out >> Thu Apr 7 17:38:00 CEST 2016 [og-qa] out >> Thu Apr 7 17:38:00 CEST 2016 [og-ci] out >> Thu Apr 7 17:38:00 CEST 2016 [og-test] out >> Thu Apr 7 17:38:00 CEST 2016 [og-prod-rm] out >> >> @Mac-mini ~/p/f/> ./check_clock.sh >> Thu Apr 7 18:38:00 CEST 2016 [envrm02] out >> Thu Apr 7 18:45:10 CEST 2016 [og-qa] out >> Thu Apr 7 18:38:00 CEST 2016 [og-ci] out >> Thu Apr 7 18:38:00 CEST 2016 [og-test] out >> Thu Apr 7 18:38:00 CEST 2016 [og-prod-rm] out >> >> @Mac-mini ~/p/f/> ./check_clock.sh >> Thu Apr 7 18:41:05 CEST 2016 [envrm02] out >> Thu Apr 7 18:48:15 CEST 2016 [og-qa] out >> Thu Apr 7 18:41:05 CEST 2016 [og-ci] out >> Thu Apr 7 18:41:05 CEST 2016 [og-test] out >> Thu Apr 7 18:41:05 CEST 2016 [og-prod-rm] out >> >> @Mac-mini ~/p/f/> ./check_clock.sh >> Thu Apr 7 18:46:51 CEST 2016 [envrm02] out >> Thu Apr 7 18:46:52 CEST 2016 [og-qa] out >> Thu Apr 7 18:46:52 CEST 2016 [og-ci] out >> Thu Apr 7 18:46:53 CEST 2016 [og-test] out >> Thu Apr 7 18:46:53 CEST 2016 [og-prod-rm] out >> >> The configuration is the same for all hosts and is this: >> @Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/rc.conf | grep ntpd >> [envrm02] out: ntpd_enable="YES" >> [og-qa] out: ntpd_enable="YES" >> [og-qa] out: ntpd_sync_on_start="YES" >> [og-ci] out: ntpd_enable="YES" >> [og-ci] out: ntpd_sync_on_start="YES" >> [og-test] out: ntpd_enable="YES" >> [og-test] out: ntpd_sync_on_start="YES" >> [og-prod-rm] out: ntpd_enable="YES" >> [og-prod-rm] out: ntpd_sync_on_start="YES" >> >> @Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/ntp.conf | grep -v 'out: #' >> [envrm02] Executing task '' >> [envrm02] run: cat /etc/ntp.conf >> [envrm02] out: >> [envrm02] out: server 0.freebsd.pool.ntp.org iburst >> [envrm02] out: server 1.freebsd.pool.ntp.org iburst >> [envrm02] out: server 2.freebsd.pool.ntp.org iburst >> [envrm02] out: >> [envrm02] out: restrict default limited kod nomodify notrap nopeer noquery >> [envrm02] out: restrict -6 default limited kod nomodify notrap nopeer noquery >> [envrm02] out: restrict 127.0.0.1 >> [envrm02] out: restrict -6 ::1 >> [envrm02] out: restrict 127.127.1.0 >> >> [og-qa] Executing task '' >> [og-qa] run: cat /etc/ntp.conf >> [og-qa] out: >> [og-qa] out: server 0.freebsd.pool.ntp.org iburst >> [og-qa] out: server 1.freebsd.pool.ntp.org iburst >> [og-qa] out: server 2.freebsd.pool.ntp.org iburst >> [og-qa] out: >> [og-qa] out: restrict default limited kod nomodify notrap nopeer noquery >> [og-qa] out: restrict -6 default limited kod nomodify notrap nopeer noquery >> [og-qa] out: restrict 127.0.0.1 >> [og-qa] out: restrict -6 ::1 >> [og-qa] out: restrict 127.127.1.0 >> >> [og-ci] Executing task '' >> [og-ci] run: cat /etc/ntp.conf >> [og-ci] out: >> [og-ci] out: server 0.freebsd.pool.ntp.org iburst >> [og-ci] out: server 1.freebsd.pool.ntp.org iburst >> [og-ci] out: server 2.freebsd.pool.ntp.org iburst >> [og-ci] out: >> [og-ci] out: restrict default limited kod nomodify notrap nopeer noquery >> [og-ci] out: restrict -6 default limited kod nomodify notrap nopeer noquery >> [og-ci] out: restrict 127.0.0.1 >> [og-ci] out: restrict -6 ::1 >> [og-ci] out: restrict 127.127.1.0 >> >> [og-test] Executing task '' >> [og-test] run: cat /etc/ntp.conf >> [og-test] out: >> [og-test] out: server 0.freebsd.pool.ntp.org iburst >> [og-test] out: server 1.freebsd.pool.ntp.org iburst >> [og-test] out: server 2.freebsd.pool.ntp.org iburst >> [og-test] out: >> [og-test] out: restrict default limited kod nomodify notrap nopeer noquery >> [og-test] out: restrict -6 default limited kod nomodify notrap nopeer noquery >> [og-test] out: restrict 127.0.0.1 >> [og-test] out: restrict -6 ::1 >> [og-test] out: restrict 127.127.1.0 >> >> Some observation: >> * The clock of envrm02 (the HOST) always seems to be correct. >> * The issue happens on a (apparently) random guest (not always the same one). >> * The issue happens always on one host at a time. >> >> The bhyve command line is, roughly, this one: >> >> /usr/sbin/bhyveload -m 2048 -d /san_storage/VMfs/$machine/$machine.img $machine >> /usr/sbin/bhyve -c 2 -m 2048 -A -H -P -s 0,hostbridge -s >> 2,ahci-hd,/san_storage/VMfs/$machine/$machine.img -s 4,virtio-net,$tap >> -s 31,lpc -l com1,stdio $machine >> >> Thanks in advance. >> _______________________________________________ >> freebsd-virtualization@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization >> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe@freebsd.org" -- *Ing. Ilario Febi Schema 31 S.p.a. Socio Unico* *System Administrator *BO - FI - ROMA - PA ITALY Tel: +39. 06.98.358.472 Cell: +39. 366.69.45.384 Fax: +39. 055.71.880.466 Società del Gruppo *SC31 ITALIA*