From nobody Thu Jun 15 22:22:19 2023 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4QhxcL1Mqdz4df4q for ; Thu, 15 Jun 2023 22:22:26 +0000 (UTC) (envelope-from sean@rogue-research.com) Received: from mail.rogue-research.com (mail.rogue-research.com [207.115.102.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.rogue-research.com", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4QhxcK5gFFz3GDh for ; Thu, 15 Jun 2023 22:22:25 +0000 (UTC) (envelope-from sean@rogue-research.com) Authentication-Results: mx1.freebsd.org; none Received: from localhost (localhost [127.0.0.1]) by mail.rogue-research.com (Postfix) with ESMTP id 37504E47FE36; Thu, 15 Jun 2023 18:22:24 -0400 (EDT) X-Virus-Scanned: amavisd-new at rogue-research.com Received: from mail.rogue-research.com ([127.0.0.1]) by localhost (kingu.rogue-research.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8Hi4sPTlftfr; Thu, 15 Jun 2023 18:22:21 -0400 (EDT) Received: from [172.16.6.1] (dullahan.rogue-research.com [10.29.19.120]) by mail.rogue-research.com (Postfix) with ESMTPSA id A3D67E47FE13; Thu, 15 Jun 2023 18:22:21 -0400 (EDT) From: Sean McBride To: jason@tubnor.net Cc: freebsd-virtualization@FreeBSD.org Subject: Re: Ubuntu in bhyve Date: Thu, 15 Jun 2023 18:22:19 -0400 X-Mailer: MailMate (1.14r5937) Message-ID: In-Reply-To: <002401d99fd2$4c6d2470$e5476d50$@tubnor.net> References: <1AC64CF0-2CCF-4B72-9E44-75104E813CF6@rogue-research.com> <002401d99fd2$4c6d2470$e5476d50$@tubnor.net> List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-virtualization@freebsd.org X-BeenThere: freebsd-virtualization@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="=_MailMate_C7833C97-3B71-4A0C-A381-0BD751521718_=" Embedded-HTML: [{"plain":[783,2097],"uuid":"385C24F8-818F-4661-910E-FEEBF47434BA"}] X-Rspamd-Queue-Id: 4QhxcK5gFFz3GDh X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:11478, ipnet:207.115.96.0/20, country:CA] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-ThisMailContainsUnwantedMimeParts: N --=_MailMate_C7833C97-3B71-4A0C-A381-0BD751521718_= Content-Type: text/plain; format=flowed Jason, Thanks for your reply. We run NTP on our router (a pfsense) and both the host (TrueNAS) and guests are using it, correctly I believe. On TrueNAS: # ntpdate -q time.rogue-research.com server 10.xx.xx.1, stratum 2, offset -0.000149, delay 0.02580 15 Jun 18:19:11 ntpdate[1239]: adjust time server 10.xx.xx.1 offset -0.000149 sec On Ubuntu: $ timedatectl Local time: Thu 2023-06-15 18:17:02 EDT Universal time: Thu 2023-06-15 22:17:02 UTC RTC time: Thu 2023-06-15 22:17:02 Time zone: America/Toronto (EDT, -0400) System clock synchronized: yes NTP service: active RTC in local TZ: no I believe both indicate that it's working... Sean On 15 Jun 2023, at 17:42, jason@tubnor.net wrote: > Make sure your guests (and host) are syncing to a reliable time > source. I > have heard time to time of some having this issue, but we haven't seen > it > though we do use OpenNTPD across our BSD/*nix guests (windows has its > own > time sync service). > > > > Cheers, > > > > Jason. > > > > From: owner-freebsd-virtualization@freebsd.org > On Behalf Of Sean McBride > Sent: Friday, June 16, 2023 5:20 AM > To: freebsd-virtualization@FreeBSD.org > Subject: Ubuntu in bhyve > > > > Hi all, > > I am running bhyve (via TrueNAS Core 13) and have some Ubuntu 20.04 > and > 22.04 guests. > > One of them rather regularly seizes up hard, where I cannot ssh to it, > then > after a few minutes works again. This happens repeatedly and on and > off. In > the Ubuntu logs I see messages like: > > clocksource: timekeeping watchdog on CPU2: Marking clocksource 'tsc' > as > unstable because the skew is too large: > clocksource: 'hpet' wd_nsec: 536417782 wd_now: > 638cb3ff wd_last: 63036152 mask: ffffffff > clocksource: 'tsc' cs_nsec: 536821277 cs_now: > 225a9f9e1250b cs_last: 225a9b3891749 mask: ffffffffffffffff > clocksource: 'tsc' is current clocksource. > tsc: Marking TSC unstable due to clocksource watchdog > TSC found unstable after boot, most likely due to broken BIOS. Use > 'tsc=unstable'. > sched_clock: Marking unstable (12899572906362, > 9192186981)<-(12908811630850, > -46250627) > clocksource: Checking clocksource tsc synchronization from CPU 2 to > CPUs > 0-1,3. > clocksource: Switched to clocksource hpet > loop8: detected capacity change from 0 to 114000 > > Although I don't fully grok these messages, they sound like something > that > could be bhyve's fault. Anyone know if that may indeed be the case? > > There is a thread about this on the TrueNAS forum too, but with no > answers > really: > > > 76/> > https://www.truenas.com/community/threads/debian-vms-under-bhyve-clock.10837 > 6/ > > Thanks, > > Sean --=_MailMate_C7833C97-3B71-4A0C-A381-0BD751521718_= Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable

Jason,

Thanks for your reply.

We run NTP on our router (a pfsense) and both the host (T= rueNAS) and guests are using it, correctly I believe.

On TrueNAS:

ntpdate -q time.rogue-research.com

server 10.xx.xx.1, stratum 2, offset -0.000149, delay 0.0= 2580
15 Jun 18:19:11 ntpdate[1239]: adjust time server 10.xx.xx.1 offset -0.00= 0149 sec

On Ubuntu:

$ timedatectl
Local time: Thu 2023-06-15 18:17:02 EDT
Universal time: Thu 2023-06-15 22:17:02 UTC
RTC time: Thu 2023-06-15 22:17:02
Time zone: America/Toronto (EDT, -0400)
System clock synchronized: yes
NTP service: active
RTC in local TZ: no

I believe both indicate that it's working...

Sean

On 15 Jun 2023, at 17:42, jason@tubnor.net wrote:

Make su= re your guests (and host) are syncing to a reliable time source. I have h= eard time to time of some having this issue, but we haven=E2=80=99t seen = it though we do use OpenNTPD across our BSD/*nix guests (windows has its = own time sync service).

 <= /span>

Cheers,=

 <= /span>

Jason.<= /span>

 <= /span>

From: owner-freebsd-virtualization@freebsd.org <owner-freebsd-vi= rtualization@freebsd.org> On Behalf Of Sean McBride
Sent: Friday, June 16, 2023 5:20 AM
To: freebsd-virtualization@FreeBSD.org
Subject: Ubuntu in bhyve

 

Hi all,

I am running = bhyve (via TrueNAS Core 13) and have some Ubuntu 20.04 and 22.04 guests.<= /span>

One of them r= ather regularly seizes up hard, where I cannot ssh to it, then after a fe= w minutes works again. This happens repeatedly and on and off. In the Ubu= ntu logs I see messages like:

clocksource: timekeeping watchdog on CPU2: Marking cl=
ocksource 'tsc' as unstable because the skew is too large:<=
/pre>

clocksource:                       'hpet' wd_ns=
ec: 536417782 wd_now: 638cb3ff wd_last: 63036152 mask: ffffffff


clocksource:                       'tsc' cs_nse=
c: 536821277 cs_now: 225a9f9e1250b cs_last: 225a9b3891749 mask: fffffffff=
fffffff


clocksource:                       'tsc' is cur=
rent clocksource.


tsc: Marking TSC unstable due to clocksource wa=
tchdog


TSC found unstable after boot, most likely due =
to broken BIOS. Use 'tsc=3Dunstable'.


sched_clock: Marking unstable (12899572906362, =
9192186981)<-(12908811630850, -46250627)


clocksource: Checking clocksource tsc synchroni=
zation from CPU 2 to CPUs 0-1,3.


clocksource: Switched to clocksource hpet


loop8: detected capacity change from 0 to 11400=
0


Although I do=
n't fully grok these messages, they sound like something that could be bh=
yve's fault. Anyone know if that may indeed be the case?


There is a th=
read about this on the TrueNAS forum too, but with no answers really:


https://www.truenas.com/community/thre=
ads/debian-vms-under-bhyve-clock.108376/


Thanks,


Sean
--=_MailMate_C7833C97-3B71-4A0C-A381-0BD751521718_=--