Date: Tue, 20 Jan 2026 12:32:56 +0000 From: Andrew Turner <andrew@fubar.geek.nz> To: Roman Bogorodskiy <novel@freebsd.org> Cc: Mark Johnston <markj@freebsd.org>, freebsd-virtualization@freebsd.org, freebsd-arm@freebsd.org Subject: Re: Quartz64: vmm: No vgic found Message-ID: <0EE0EFAA-68BF-4BA5-A42E-2D72454B3E4A@fubar.geek.nz> In-Reply-To: <aW5x45wvWb2lSWNy@tulp> References: <aWkm8jwP6SRvT7iW@tulp> <aWqf5qNu_IlMMiEt@nuc> <aWvEs2_2aN0Gz67l@tulp> <aW5KzB0F1QhKf2Om@nuc> <aW5x45wvWb2lSWNy@tulp>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] > On 19 Jan 2026, at 18:03, Roman Bogorodskiy <novel@freebsd.org> wrote: > > Mark Johnston wrote: > >> On Sat, Jan 17, 2026 at 06:19:47PM +0100, Roman Bogorodskiy wrote: >>> Mark Johnston wrote: >>> >>>> On Thu, Jan 15, 2026 at 06:42:10PM +0100, Roman Bogorodskiy wrote: >>>>> But bhyve does not start: >>>>> [...] >>>>> Does the board need something in addition to GICv3 that is missing that >>>>> is needed to make vmm(4) work, or is that a configuration or vmm(4) >>>>> issue? >>>> >>>> The vgic driver is refusing to attach, apparently because we don't >>>> detect virtualization support when probing the controller. When booting >>>> from a FDT we look for an "interrupts" property to determine this, but >>>> it's not clear to me whether that's totally right. Can you share the >>>> device tree you're using with this board? >>> >>> Not sure what's the right way to share the device tree, so I used the >>> command I found on wiki (https://wiki.freebsd.org/FlattenedDeviceTree), >>> that is: sysctl -b hw.fdt.dtb | dtc -I dtb >>> >>> It's lengthy, so I uploaded it here: >>> >>> https://people.freebsd.org/~novel/misc/quartz64_devicetree.txt >> >> So, from what I can tell, gic_v3_ofw_bus_attach() should be attaching a >> child vgic device. Are you loading vmm.ko after the kernel has booted? >> Does anything change if you load it from loader.conf, or compile vmm >> into the kernel? > > I was loading vmm.ko manually, i.e. by running `kldload vmm`. > Nothing changes when I load it via vmm_load="YES" in /boot/loader.conf: > > vmm: No vgic found > module_register_init: MOD_LOAD (vmm, 0xffff0000014391f0, 0) error 19 > > I'll test adding that to the kernel once I have an established procedure > of building a kernel for this board. Can you try the patch in https://reviews.freebsd.org/D54791? It fixes an issue where the vgic driver isn’t attached if there is no ITS on the board. This appears to be the case with the Quartz64. Andrew [-- Attachment #2 --] <html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><br id="lineBreakAtBeginningOfMessage"><div><br><blockquote type="cite"><div>On 19 Jan 2026, at 18:03, Roman Bogorodskiy <novel@freebsd.org> wrote:</div><br class="Apple-interchange-newline"><div><meta charset="UTF-8"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;"> Mark Johnston wrote:</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><blockquote type="cite" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">On Sat, Jan 17, 2026 at 06:19:47PM +0100, Roman Bogorodskiy wrote:<br><blockquote type="cite"> Mark Johnston wrote:<br><br><blockquote type="cite">On Thu, Jan 15, 2026 at 06:42:10PM +0100, Roman Bogorodskiy wrote:<br><blockquote type="cite">But bhyve does not start:<br>[...]<br>Does the board need something in addition to GICv3 that is missing that<br>is needed to make vmm(4) work, or is that a configuration or vmm(4)<br>issue?<br></blockquote><br>The vgic driver is refusing to attach, apparently because we don't<br>detect virtualization support when probing the controller. When booting<br>from a FDT we look for an "interrupts" property to determine this, but<br>it's not clear to me whether that's totally right. Can you share the<br>device tree you're using with this board?<br></blockquote><br>Not sure what's the right way to share the device tree, so I used the<br>command I found on wiki (https://wiki.freebsd.org/FlattenedDeviceTree),<br>that is: sysctl -b hw.fdt.dtb | dtc -I dtb<br><br>It's lengthy, so I uploaded it here:<br><br>https://people.freebsd.org/~novel/misc/quartz64_devicetree.txt<br></blockquote><br>So, from what I can tell, gic_v3_ofw_bus_attach() should be attaching a<br>child vgic device. Are you loading vmm.ko after the kernel has booted?<br>Does anything change if you load it from loader.conf, or compile vmm<br>into the kernel?<br></blockquote><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">I was loading vmm.ko manually, i.e. by running `kldload vmm`.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">Nothing changes when I load it via vmm_load="YES" in /boot/loader.conf:</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">vmm: No vgic found</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">module_register_init: MOD_LOAD (vmm, 0xffff0000014391f0, 0) error 19</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">I'll test adding that to the kernel once I have an established procedure</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;">of building a kernel for this board.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: 400; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"></div></blockquote><br></div><div>Can you try the patch in <a href="https://reviews.freebsd.org/D54791">https://reviews.freebsd.org/D54791</a>? It fixes an issue where the vgic driver isn’t attached if there is no ITS on the board. This appears to be the case with the Quartz64.</div><div><br></div><div>Andrew</div><br></body></html>home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?0EE0EFAA-68BF-4BA5-A42E-2D72454B3E4A>