From nobody Tue Jan 20 12:32:56 2026 X-Original-To: freebsd-arm@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dwRZ270G4z6PW3k for ; Tue, 20 Jan 2026 12:33:14 +0000 (UTC) (envelope-from bounce.k3gv00ayaen5gj3=fklulsakozfh=izwgkv3v5myatt@em790814.fubar.geek.nz) Received: from e3i341.smtp2go.com (e3i341.smtp2go.com [158.120.85.85]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4dwRZ23D4Tz3Zf9 for ; Tue, 20 Jan 2026 12:33:13 +0000 (UTC) (envelope-from bounce.k3gv00ayaen5gj3=fklulsakozfh=izwgkv3v5myatt@em790814.fubar.geek.nz) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fubar.geek.nz; i=@fubar.geek.nz; q=dns/txt; s=s790814; t=1768912387; h=from : subject : to : message-id : date; bh=4Su5GkFZFDuatCMHI/prlZC8m0nCKatdcHFusFRiDok=; b=ZnlUk0AO3IfakrQ4OoVTvpZrw67hLWY4ALWNpOMe6VZuU0KwbGWOhI7gKUioOU9LNk6SI SLdbk8BKkKDuMfeVpFh31Z4JYj+wxvoBCRxNKAQ9xPuP/2O6B0TmWdqHFsfnRETM+BniZw3 7aGQ5fw0gVPuYlK33KOquTx3lNT+lIWufc1WnyQFqs3yNjyLVZmHs/6PT92AU32b5WCKCKA pmut+Lxhwrz+fLCQM5QcCazQgDmV0BqPvKBNW9We1T8CpN+bVHJbkQ5Q8Y8SC1Sh30e5NEe eufX98jg3TZkt3HdL7TM3tHLyK5LzmwyPsM5TbpdXKHZFfRlPG2wYD9x+0oA== Received: from [10.99.243.232] (helo=morbo.fubar.geek.nz) by smtpcorp.com with esmtpsa (TLS1.3:ECDHE_X25519__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim 4.98.1-S2G) (envelope-from ) id 1viAuz-AIkwcC8qmiX-HM2X; Tue, 20 Jan 2026 12:33:09 +0000 Received: from smtpclient.apple (unknown [IPv6:2a02:8012:a6a8:0:fc64:fee4:9318:c77]) by morbo.fubar.geek.nz (Postfix) with ESMTPSA id B40D04A435; Tue, 20 Jan 2026 12:33:07 +0000 (UTC) From: Andrew Turner Message-Id: <0EE0EFAA-68BF-4BA5-A42E-2D72454B3E4A@fubar.geek.nz> Content-Type: multipart/alternative; boundary="Apple-Mail=_6ECD286B-5D76-4F8A-B0B8-9226167CCB81" List-Id: Porting FreeBSD to ARM processors List-Archive: https://lists.freebsd.org/archives/freebsd-arm List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-arm@FreeBSD.org Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3826.700.81\)) Subject: Re: Quartz64: vmm: No vgic found Date: Tue, 20 Jan 2026 12:32:56 +0000 In-Reply-To: Cc: Mark Johnston , freebsd-virtualization@freebsd.org, freebsd-arm@freebsd.org To: Roman Bogorodskiy References: X-Mailer: Apple Mail (2.3826.700.81) X-Report-Abuse: Please forward a copy of this message, including all headers, to Feedback-ID: 790814m:790814amQcrys:790814sLMQjd3Rv5 X-smtpcorp-track: Ochydt4YA-Bu.0tF_rsgvv28G._3N0HwDilOV X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:23352, ipnet:158.120.84.0/22, country:US] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4dwRZ23D4Tz3Zf9 --Apple-Mail=_6ECD286B-5D76-4F8A-B0B8-9226167CCB81 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On 19 Jan 2026, at 18:03, Roman Bogorodskiy wrote: >=20 > Mark Johnston wrote: >=20 >> On Sat, Jan 17, 2026 at 06:19:47PM +0100, Roman Bogorodskiy wrote: >>> Mark Johnston wrote: >>>=20 >>>> On Thu, Jan 15, 2026 at 06:42:10PM +0100, Roman Bogorodskiy wrote: >>>>> But bhyve does not start: >>>>> [...] >>>>> Does the board need something in addition to GICv3 that is missing = that >>>>> is needed to make vmm(4) work, or is that a configuration or = vmm(4) >>>>> issue? >>>>=20 >>>> The vgic driver is refusing to attach, apparently because we don't >>>> detect virtualization support when probing the controller. When = booting >>>> from a FDT we look for an "interrupts" property to determine this, = but >>>> it's not clear to me whether that's totally right. Can you share = the >>>> device tree you're using with this board? >>>=20 >>> Not sure what's the right way to share the device tree, so I used = the >>> command I found on wiki = (https://wiki.freebsd.org/FlattenedDeviceTree), >>> that is: sysctl -b hw.fdt.dtb | dtc -I dtb >>>=20 >>> It's lengthy, so I uploaded it here: >>>=20 >>> https://people.freebsd.org/~novel/misc/quartz64_devicetree.txt >>=20 >> So, from what I can tell, gic_v3_ofw_bus_attach() should be attaching = a >> child vgic device. Are you loading vmm.ko after the kernel has = booted? >> Does anything change if you load it from loader.conf, or compile vmm >> into the kernel? >=20 > I was loading vmm.ko manually, i.e. by running `kldload vmm`. > Nothing changes when I load it via vmm_load=3D"YES" in = /boot/loader.conf: >=20 > vmm: No vgic found > module_register_init: MOD_LOAD (vmm, 0xffff0000014391f0, 0) error 19 >=20 > I'll test adding that to the kernel once I have an established = procedure > of building a kernel for this board. Can you try the patch in https://reviews.freebsd.org/D54791? It fixes an = issue where the vgic driver isn=E2=80=99t attached if there is no ITS on = the board. This appears to be the case with the Quartz64. Andrew --Apple-Mail=_6ECD286B-5D76-4F8A-B0B8-9226167CCB81 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On 19 Jan 2026, at 18:03, Roman Bogorodskiy = <novel@freebsd.org> wrote:

 Mark Johnston wrote:

On Sat, Jan 17, 2026 at 06:19:47PM +0100, Roman Bogorodskiy = wrote:
 Mark Johnston = wrote:

On Thu, Jan 15, 2026 at = 06:42:10PM +0100, Roman Bogorodskiy wrote:
But bhyve does not start:
[...]
Does the board need = something in addition to GICv3 that is missing that
is needed to make = vmm(4) work, or is that a configuration or = vmm(4)
issue?

The vgic driver is refusing to = attach, apparently because we don't
detect virtualization support = when probing the controller.  When booting
from a FDT we look = for an "interrupts" property to determine this, but
it's not clear to = me whether that's totally right.  Can you share the
device tree = you're using with this board?

Not sure what's the = right way to share the device tree, so I used the
command I found on = wiki (https://wiki.freebsd.org/FlattenedDeviceTree),
that is: sysctl = -b hw.fdt.dtb | dtc -I dtb

It's lengthy, so I uploaded it = here:

https://people.freebsd.org/~novel/misc/quartz64_devicetree.tx= t

So, from what I can tell, gic_v3_ofw_bus_attach() = should be attaching a
child vgic device.  Are you loading vmm.ko = after the kernel has booted?
Does anything change if you load it from = loader.conf, or compile vmm
into the kernel?

I was loading vmm.ko manually, i.e. by = running `kldload vmm`.
Nothing = changes when I load it via vmm_load=3D"YES" in = /boot/loader.conf:

vmm: No = vgic found
module_register_init: = MOD_LOAD (vmm, 0xffff0000014391f0, 0) error 19

I'll test adding that to the kernel once I = have an established procedure
of = building a kernel for this board.

Can you try = the patch in https://reviews.freebsd.org/D5= 4791? It fixes an issue where the vgic driver isn=E2=80=99t = attached if there is no ITS on the board. This appears to be the case = with the = Quartz64.

Andrew

= --Apple-Mail=_6ECD286B-5D76-4F8A-B0B8-9226167CCB81--