Date: Sat, 18 Jul 2026 11:22:52 +0000 From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 296872] bhyve guest hangs at start_init (100%+ vCPU spin in lock_delay) on AMD Ryzen (Zen2/SVM), 15.1-RELEASE host; identical guest boots on 14.3 Intel host Message-ID: <bug-296872-27103@https.bugs.freebsd.org/bugzilla/>
index | next in thread | raw e-mail
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296872 Bug ID: 296872 Summary: bhyve guest hangs at start_init (100%+ vCPU spin in lock_delay) on AMD Ryzen (Zen2/SVM), 15.1-RELEASE host; identical guest boots on 14.3 Intel host Product: Base System Version: 15.1-RELEASE Hardware: amd64 OS: Any Status: New Severity: Affects Some People Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: ozgur@kazancci.com Created attachment 272954 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=272954&action=edit The zip file contains: 01-host-info.txt - 02-bhyve-cmdline.txt - 03-bhyvectl-getall-1 - 03-bhyvectl-getall-2 - 03-bhyvectl-getall-3 - 04-guest-console Overview -------- A FreeBSD 15.1-RELEASE amd64 guest under bhyve reaches userland (the kernel prints "start_init: trying /sbin/init") and then hangs permanently. The bhyve process pins ~100% CPU per vCPU (~213% for 4 vCPUs, ~120% for 1 vCPU). The guest never reaches multi-user; no login prompt, no network. The identical guest disk image boots normally on a different host (Intel Haswell, 14.3-RELEASE), isolating the problem to the 15.1 host's vmm/SVM path on AMD. bhyvectl shows the guest spinning in lock_delay() with millions of intercepted PAUSE instructions and no interrupt-window activity — the classic signature of a waited-for timer/interrupt never being delivered to the guest. Host environment ---------------- - Host: FreeBSD 15.1-RELEASE-p1 amd64 (GENERIC) - CPU: AMD Ryzen 7 3700X (Zen2), Family=0x17 Model=0x71, AMD Features2=<...,SVM,...> - vmm.ko loaded; managed via vm-bhyve 1.7.4 (also reproduced with 1.7.99.87) - hw.vmm.svm.num_asids=32768, features=1293567, vmcb_clean=1023 - hw.vmm.amdvi.enable=0 (no PCI passthrough / IOMMU in use) Guest ----- - FreeBSD 15.1-RELEASE amd64 GENERIC (official 15.1 dist sets; also repro from the stock 15.1 bootonly install ISO's own installer kernel). - loader: bhyveload (NOT UEFI -> not an edk2 firmware issue). - Reproduced with disk = nvme AND virtio-blk; backing = zvol AND raw file; cpu=4 AND cpu=1. bhyve invocation (verbatim, from vm-bhyve) ------------------------------------------ bhyve -c 4 -m 8G -AHPw -U <uuid> -u \ -s 0,hostbridge -s 31,lpc \ -s 0:4:0,virtio-blk,/zroot/vm/web1/disk0.img \ -s 0:5:0,virtio-net,tap0,mac=... \ -l com1,/dev/nmdm-web1.1A web1 (-H yield-on-HLT and -w ignore-unimplemented-MSR are both present.) Steps to reproduce ------------------ 1. On the AMD Ryzen 3700X / 15.1-RELEASE-p1 host, create a FreeBSD 15.1 amd64 guest (bhyveload loader, virtio-blk or nvme disk, virtio-net). 2. Boot the guest. Actual result ------------- Kernel loads, probes devices, mounts root, prints: Trying to mount root from ufs:/dev/ufs/rootfs []... start_init: trying /sbin/init ...then NO further output. bhyve spins ~100% CPU per vCPU indefinitely. No /etc/rc output, no login, no network. Guest is unresponsive (destroyable via bhyvectl --destroy). Expected result --------------- Guest completes /etc/rc and reaches login (as it does on a 14.3 Intel host with the same image). Diagnostic evidence (bhyvectl --get-all, twice ~4s apart while hung) ------------------------------------------------------------------- rip[0] 0xffffffff80bd77b2 (sample 1) rip[0] 0xffffffff80bd77b0 (sample 2) -> RIP oscillates within 2 bytes == tight PAUSE spin loop. -> nm on guest kernel: 0xffffffff80bd77a0 = lock_delay() => spinning in lock_delay(). number of vm exits due to exceptions : 0 -> 0 (rules out MSR/#GP loop) vm exits due to external interrupt : 11390 -> 13712 (~580/s) number of times pause was intercepted : 9415553 -> 10025216 (~600k/s) vm exits due to interrupt window opening: 0 -> 0 vm exits due to nested page fault : 11657 -> 11657 (no growth) Interpretation: ~600k PAUSE/s with RIP fixed in lock_delay() => guest stuck on a spinlock; exceptions=0 => not an MSR loop; interrupt-window=0 with a lock owner waiting => a timer/callout interrupt is not being delivered; no forward progress. Cross-check / isolation (key data point) ---------------------------------------- The SAME 15.1 amd64 guest image, created with the identical procedure, boots CLEANLY to a login prompt on: Intel Core i7-4790 (Haswell, VT-x), FreeBSD 14.3-RELEASE-p15. => Guest image/config is correct; the fault is on the AMD/15.1 host side. Additional observations ------------------------ - Reproduced with cpu=1 (rules out AP-launch/SMP-startup-only bugs; points at the interrupt/timer delivery layer). - Warm reboot AND cold power-cycle of the AMD host do not change behaviour (expected for a deterministic software issue; argues against stuck hardware/firmware state). - Preceded by guest-side "nvme: Resetting controller due to a timeout" during heavy fsync I/O — a SYMPTOM (I/O completion interrupts not arriving), not the cause; reproduces afterwards on every boot including with virtio-blk (no nvme). Workarounds attempted that did NOT help (guest still hangs in lock_delay) ------------------------------------------------------------------------- guest hw.x2apic_enable=0; guest kern.eventtimer.timer=HPET; cpu=1; nvme & virtio-blk; zvol & raw file; vm-bhyve 1.7.4 & 1.7.99.87; vmm kldunload/kldload. (hw.vmm.svm.num_asids=1 made the guest fail to run entirely — inconclusive.) Probable root cause ------------------- A regression in the 15.x vmm(4) AMD SVM path delivering a timer/APIC interrupt (or in event-injection / interrupt-window handling) to the guest. The guest reaches early userland, a thread blocks on a callout/timer, the interrupt is never injected, the lock is never released, another context spins in lock_delay() forever. The 15.x vmm refactor is known to have produced at least one AMD-relevant regression (FreeBSD-EN-25:20.vmm, PCI-passthru IOMMU); this appears to be a separate regression in the same subsystem on the non-passthru interrupt path. Willing to test patches / a rebuilt vmm.ko and provide additional bhyvectl --get-all dumps or a bhyve -G gdb-stub session on request. -- You are receiving this mail because: You are the assignee for the bug.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?bug-296872-27103>