Date: Tue, 9 Dec 2025 11:49:18 -0800 From: Taro <taro@desudesu.org> To: freebsd-virtualization@freebsd.org Subject: [vmm] patch for nvidia pci passthru on releng/15 Message-ID: <20251209114918.2212e7b4@blade>
index | next in thread | raw e-mail
[-- Attachment #1 --]
Hello!
When I updated from 14.3 to 15.0, my nvidia 5090 passthru to an Ubuntu
guest broke (nvidia-smi on the guest returns "no devices"). Emitted to
the host's dmesg:
ivhd0: EVT INTR 0 Status:0x1a EVT Head:0x0 Tail:0x10]
[CMD Total 0x42] Tail:0x420, Head:0x420.
ivhd0: [Event0: Head:0x0 Tail:0x10]
[IO_PAGE_FAULT EVT: devId:0x100 DomId:0x1 Addr:0x1271c0000 0x0]
(The Ubuntu guest is configured with 32GB of RAM. I later noted that it
works correctly when the value is set less than VM_LOWMEM_LIMIT.)
git bisect indicated the first bad commit was 08c7dd2f
After some trial and error, I added printfs to vm_mmap_memseg to see if
there was any difference in the working/bad versions, and there was:
when splitting a segment mapping, the previous working version would
first map the higher, then the lower. The broken version swapped the
ordering:
## WORKING ##
vmm: mmap{gpa=4294967296, segid=0, segoff=4294967296, len=31138512896}
vmm: mmap{gpa=0, segid=0, segoff=0, len=3221225472}
vmm: mmap{gpa=4291313664, segid=8, segoff=13123584, len=3653632}
vmm: mmap{gpa=4278190080, segid=8, segoff=0, len=4096}
vmm: mmap{gpa=3288334336, segid=9, segoff=0, len=33554432}
## BROKEN ##
vmm: mmap{gpa=0, segid=0, segoff=0, len=3221225472}
vmm: mmap{gpa=4294967296, segid=0, segoff=3221225472, len=31138512896}
vmm: mmap{gpa=4291313664, segid=8, segoff=13123584, len=3653632}
vmm: mmap{gpa=4278190080, segid=8, segoff=0, len=4096}
vmm: mmap{gpa=3288334336, segid=9, segoff=0, len=33554432}
Putting the ordering back (patch attached) fixes the problem for me. I
don't know why.
I do note that the segoff is also different between working/broken, but
am trying not to think about it. I am a humble web programmer, lost and
afraid.
Thanks!
[-- Attachment #2 --]
diff --git a/lib/libvmmapi/vmmapi.c b/lib/libvmmapi/vmmapi.c
index 77f0f8f5c58..a86b444b642 100644
--- a/lib/libvmmapi/vmmapi.c
+++ b/lib/libvmmapi/vmmapi.c
@@ -562,18 +562,20 @@ vm_setup_memory_domains(struct vmctx *ctx, enum vm_mmap_style vms,
if (gpa <= VM_LOWMEM_LIMIT &&
gpa + dom->size > VM_LOWMEM_LIMIT) {
low_len = VM_LOWMEM_LIMIT - gpa;
+ /* Map remainder first */
+ error = map_memory_segment(ctx, segid, VM_HIGHMEM_BASE,
+ dom->size - low_len, low_len, baseaddr);
+ if (error)
+ return (error);
+ /* Then map lower segment */
error = map_memory_segment(ctx, segid, gpa, low_len, 0,
baseaddr);
if (error)
return (error);
+ /* Update vars to post-remainder setup */
ctx->lowmem_size = VM_LOWMEM_LIMIT;
- /* Map the remainder. */
gpa = VM_HIGHMEM_BASE;
len = dom->size - low_len;
- error = map_memory_segment(ctx, segid, gpa, len,
- low_len, baseaddr);
- if (error)
- return (error);
} else {
len = dom->size;
error = map_memory_segment(ctx, segid, gpa, len, 0,
help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20251209114918.2212e7b4>