From nobody Thu Jun 4 11:23:17 2026 X-Original-To: freebsd-virtualization@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4gWJzL66vGz6gNHT for ; Thu, 04 Jun 2026 09:23:58 +0000 (UTC) (envelope-from marietto2008@gmail.com) Received: from mail-pl1-x629.google.com (mail-pl1-x629.google.com [IPv6:2607:f8b0:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "WR4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4gWJzK4SbZz45N8 for ; Thu, 04 Jun 2026 09:23:57 +0000 (UTC) (envelope-from marietto2008@gmail.com) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20251104 header.b=KEwlnPlk; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of marietto2008@gmail.com designates 2607:f8b0:4864:20::629 as permitted sender) smtp.mailfrom=marietto2008@gmail.com; arc=pass ("google.com:s=arc-20240605:i=1") Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-2c0c3184c71so3681885ad.1 for ; Thu, 04 Jun 2026 02:23:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1780565035; cv=none; d=google.com; s=arc-20240605; b=hzUzbPjAJOUz7cKPzsovaCaPQFWguMFOZMmijW7qUu2E/qO+xjBfC5tOV87OI357IS DLterGzzhwj7J2geN59xlLHTi5l9bV0SMtme35jUN/fj+pcxPvNhQTp5/YRW0B0tK/xI GCPH8sosW/QyHi8IiZI59QvNcALbgnDRmOsEm4GodpfE2tQeVxaNhKB/g3nLzdTOltC6 DoTZlIuJLjXPVker736bG8Zi3cc4ibFScjO5U5XXsU7byIUu+OzHh6Glwr6EVp+ZinOK kqQZ2r3xqQZ+ei2pheHawCm8SS5z1y+ugPupA3EugZws8IsS5iktPcqBiuQes1W8X5qr 8trA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=to:subject:message-id:date:from:mime-version:dkim-signature; bh=RvIwHdZIpH4xu9ezZ+Wezv3ReHn8toUMURKWNcabW4I=; fh=osQwXtP9Ab2JFfOXWv4VwkLw7RibrxElrQNob6r72pI=; b=lUrV1ORyD+Li+L0xdTJvh8vOb9MP8KXKE3TxkLK4SE+s08QTbZd3+/hqyOEB5Uj6zA Yf4xqrOdscaaRZQHNJXjzzXPLSNH7TnGGd3gO+LGEk9EweVpkNoNX9PqbGyPF9U5sb1O FljrYdLWxBPRI1XGqIjf4Ny/i1RPJDjldZkzLDy2bm1Bq0c+fl6B/AqW/T8CQsGq23H/ JoFs6kkp0Du4Gj2hneq2TbdWJbaf0+tmiFodjMGBUvnRuOsk176+/UKINzwS9ZeUEkWt QAvVYmPgJ89yyVmeQKHdC6hy8kSW3I1jOb7eU/8OobFZUVJ888iDQKdjEKUWi5oNLOQt PDvg==; darn=freebsd.org ARC-Authentication-Results: i=1; mx.google.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1780565035; x=1781169835; darn=freebsd.org; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=RvIwHdZIpH4xu9ezZ+Wezv3ReHn8toUMURKWNcabW4I=; b=KEwlnPlkYksIzNXHD7GE+7SdYRQPj/ygNS62vPBm2jKv5gYbW5G5qFFwbap9wD6tCN 9G+lQaP8S9aeeWhTeUKmER3enZcZQP/FMSFlsS0vaFyRt5NOGl/VcTmN1+3937dQHwC5 Ckej7UC/fQ9Cxn1D95s/WHyVocmCNy50m3n8gXBsLpljeMMAA7s3CBWLPbpyrQteJnCl ypyCfE5arMGAYQxy7s4j2MolVwlQ1DE9carHw8QOzyExxiXPTQnn3FZzFWzj/bM/SD3L 95v/vQcBVWxn8xDEXm6SfBcrY/17PGJACineBS3nDWn0qWTybRa8f6smfWXI7BUjBd2y p3tw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780565035; x=1781169835; h=to:subject:message-id:date:from:mime-version:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=RvIwHdZIpH4xu9ezZ+Wezv3ReHn8toUMURKWNcabW4I=; b=FUtBYtB3/hq0+R5zivKi0ZyJobyP66zwrOBz7WUH9jlrEj9au2ua7ree59AeglZ4tA G0bgL+7tH60pXi9nXwVJBLjEUST0o2rGNgWXlFrLpSHxcboDyUKe5w7w5yFEqxHji3AS dy7Dy8bnd2NHjk9BMaUokclcrtVotILMaRQ8RQ9zo4rL4QHMX4tUbpMkA7bSInIwxbgV 4eSVsJtTAT5cMImwGrIGRnV5wtx/F93A3vgwBYBHGECyG/LmzlX3bIbalF5ZAblhmEC1 8Su7PlLu0tdrskjkO2jO+6AE6YonizfJqvZq7j5Bva7G0UrUr1Ti4dcq9ruOBUcisldV H+nA== X-Gm-Message-State: AOJu0Yx1+KuLQT0UqQuEPcCnClmCrpr5rYJGx+gBwTozOmBlew9B4Ka5 HpMngBoJdCHXs0tOxm0hU6rWHWE6vY6gAoUM/m0vZlNg38e/xnZIqmtPBPv9tg1meNYhK1ycSNb 9cyCng0M48Ah2aqgncktmLtPgEqjlLqKlMlkm1Qo= X-Gm-Gg: Acq92OGWefo303t163QVt5zIWvtFY2UvttWl66JTdTDEhxqgd+a5/yySF7lJRF7plfw brzvIIYSkbEbBoreSgdq//0gVGAwo0UP0oAEZ0yILBEY9AJ+ilLXOvayI/x+GMzcm8eg3Phw3YY 8nSP+SXjBScHU837BnFfvGUfE5meEKd56NJk4jLMMfa20vQ2ZIxyOJnYlviYftHwsSi7og/IuTW S80LX7WJNLfXMsB2KP3onp4mFK/4wSOp/X9P/W3aQdAeXQcyCyQVlfZD2izhPfK/OsDzm/37OIL scTcvtxg8qDRDeyK1wnxz6TVlHRjbrwcAQy02WS2PTNz034J7mq0K1GCZBrgcTBkRcxnbNBlaI6 4NW7jb+7a5DyZeWWDssz4aML2RgVMoHICUBxJwq3aI5E/mySLrCT3N2ryLmhDZivfOdWYC3lgi6 +u31QD+Ntyv9dVGrJ1DDjEgRCVvRyKF4zeW6pI2Eq4ct8vZ72lOIaq5NnIgQVv30GR+ocwARxvx +PUilx2rz7YLebNG5hG5pYPampbCivRmGSnQvNPVuVv/2sJyOFuy20u/94BER+pVStgw8QFWp2C PmEoz4R6Rkct7ngQFSw+I2RJKnMtm+L+h5RFlBtVy8hOXA== X-Received: by 2002:a17:902:e74d:b0:2bf:2256:7334 with SMTP id d9443c01a7336-2c163fad73emr76605755ad.24.1780565035297; Thu, 04 Jun 2026 02:23:55 -0700 (PDT) List-Id: Discussion List-Archive: https://lists.freebsd.org/archives/freebsd-virtualization List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: freebsd-virtualization@freebsd.org Sender: owner-freebsd-virtualization@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 From: Mario Marietto Date: Thu, 4 Jun 2026 13:23:17 +0200 X-Gm-Features: AVHnY4JYBbz1izLoid6RHjePPFi3Z_Cc9gWSPdxyAbJP3mupwhmwR_qoEa8OLBE Message-ID: Subject: =?UTF-8?Q?=2A=2A=5BHelp=5D_bhyve=3A_bootrom=5Falloc=3A_vm=5Fmmap=5Fmapseg=3A_Inval?= =?UTF-8?Q?id_argument_=E2=80=94_NVIDIA_passthrough_with_Corvin=27s_branch_on_F?= =?UTF-8?Q?reeBSD_15=2E0=2A=2A?= To: FreeBSD virtualization Content-Type: multipart/alternative; boundary="0000000000006dad3a06536a1a6a" X-Spamd-Result: default: False [-2.91 / 15.00]; ARC_ALLOW(-1.00)[google.com:s=arc-20240605:i=1]; HTTP_TO_IP(1.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; URI_COUNT_ODD(1.00)[7]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; NEURAL_HAM_SHORT(-0.92)[-0.915]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4864::/56:c]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20251104]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_DN_ALL(0.00)[]; RCVD_TLS_LAST(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; TO_MATCH_ENVRCPT_ALL(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; MID_RHS_MATCH_FROMTLD(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-virtualization@freebsd.org]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; MLMMJ_DEST(0.00)[freebsd-virtualization@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::629:from] X-Spamd-Bar: -- X-Rspamd-Queue-Id: 4gWJzK4SbZz45N8 --0000000000006dad3a06536a1a6a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi everyone, I'm trying to get NVIDIA GPU passthrough working with bhyve on FreeBSD 15.0-RELEASE-p5, using Corvin K=C3=B6hne's nvidia-wip branch: https://github.com/Beckhoff/freebsd-src/tree/phab/corvink/15.0/nvidia-wip The VM fails to start with the following error before the guest even boots: bhyve: bootrom_alloc: vm_mmap_mapseg: Invalid argument --- **Hardware** - Motherboard: Gigabyte Z390 AORUS PRO - CPU: Intel Core i9-9900K - GPU: ZOTAC RTX 2080 Ti (0x10de:0x1e04, subvendor 0x19da, subdevice 0x2503) --- **Host** - FreeBSD 15.0-RELEASE-p5 (kernel GENERIC, amd64) - vmm.ko compiled from Corvin's `phab/corvink/15.0/nvidia-wip` branch - bhyve compiled from the same branch - libvmmapi.so.6 produced by the Corvin branch build --- **bhyve command** --- bhyve-lin -S -c sockets=3D8,cores=3D1,threads=3D1 -m 32G -w -H -A \ -s 0,hostbridge \ -s 1,ahci-hd,/mnt/zroot-133/bhyve/img/Linux/Ubuntu-2404-KDE6-Wayland.img,booti= ndex=3D1 \ -s 7,passthru,5/0/0 \ -s 8:0,passthru,2/0/0 \ -s 8:1,passthru,2/0/1 \ -s 8:2,passthru,2/0/2 \ -s 8:3,passthru,2/0/3 \ -s 11,hda,play=3D/dev/dsp,rec=3D/dev/dsp \ -s 13,virtio-net,tap4 \ -s 14,virtio-9p,sharename=3D/ \ -s 29,fbuf,tcp=3D0.0.0.0:5900,w=3D1650,h=3D900,wait \ -s 30,xhci,tablet \ -s 31,lpc \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \ vm0:4 < /dev/null 2>&1 & --- **Root cause analysis (via ktrace)** --- Tracing bhyve with `ktrace` reveals the following sequence: 1. `openat("/dev/vmmctl", O_RDWR)` =E2=86=92 success 2. `ioctl(VM_MMAP_GETNEXT)` =E2=86=92 success 3. `ioctl(VM_MMAP_MEMSEG)` =E2=86=92 **errno 22 (EINVAL)** The failure is inside `bootrom_alloc()` =E2=86=92 `vm_mmap_memseg()` when t= rying to map the `VM_BOOTROM` memory segment. Digging into `lib/libvmmapi/vmmapi.c`, the function `vm_alloc_memseg()` first calls `vm_get_memseg()` to check if the segment already exists. If it returns success with `len =3D=3D 0` (segment not yet allocated), it should proceed to call `VM_ALLOC_MEMSEG` ioctl to create the segment. However, `VM_ALLOC_MEMSEG` is **never called** in the ktrace output. The result is that `seg->object =3D=3D NULL` in the kernel when `vm_mmap_memseg()` is later called for `VM_BOOTROM`, which returns `EINVAL`. --- **Workaround found** --- Replacing `libvmmapi.so.6` (from the Corvin build) with the system stock `libvmmapi.so.7` fixes the bootrom issue =E2=80=94 the VM boots successfull= y. However, this introduces a new problem: `bhyvectl --vm=3Dname --create` returns `Operation not permitted` (EPERM) because the stock libvmmapi uses different ioctls that vmm.ko Corvin doesn't handle correctly. What is the correct fix for `vm_alloc_memseg()` in the Corvin libvmmapi to ensure `VM_ALLOC_MEMSEG` is properly called for `VM_BOOTROM`? Is there a known patch or workaround that keeps compatibility with vmm.ko Corvin while fixing the bootrom allocation? Thanks! --- *Relevant source: `lib/libvmmapi/vmmapi.c`, function `vm_alloc_memseg()` and `vm_create_devmem()`* *vmm kernel side: `sys/dev/vmm/vmm_mem.c`, function `vm_mmap_memseg()`* --=20 Mario. --0000000000006dad3a06536a1a6a Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
=
=

Hi everyone,

I'm trying to get NVIDIA GPU passthrough working with bhyve on=20 FreeBSD 15.0-RELEASE-p5, using Corvin K=C3=B6hne's nvidia-wip branch:

https://github.com/Beckhoff/freebsd-src/tree/p= hab/corvink/15.0/nvidia-wip

The VM fails to start with the following error before the guest even = boots:

bhyve: bootrom_alloc: vm_mmap_mapseg: Invalid argument

---

**Hardware**

- Motherboard: Gigabyte Z390 AORUS PRO

- CPU: Intel Core i9-9900K

- GPU: ZOTAC RTX 2080 Ti (0x10de:0x1e04, subvendor 0x19da, subdevice = 0x2503)

---

**Host**

- FreeBSD 15.0-RELEASE-p5 (kernel GENERIC, amd64)

- vmm.ko compiled from Corvin's `phab/corvink/15.0/nvidia-wip` br= anch

- bhyve compiled from the same branch

- libvmmapi.so.6 produced by the Corvin branch build

---

**bhyve command**

---

bhyve-lin -S -c sockets=3D8,cores=3D1,threads=3D1 -m 32G -w -H = -A \

-s 0,hostbridge \

-s 1,ahci-hd,/mnt/zroot-133/bhyve/img/Linux/Ubuntu-2404-KDE6-Wa= yland.img,bootindex=3D1 \

-s 7,passthru,5/0/0 \

-s 8:0,passthru,2/0/0 \

-s 8:1,passthru,2/0/1 \

-s 8:2,passthru,2/0/2 \

-s 8:3,passthru,2/0/3 \

-s 11,hda,play=3D/dev/dsp,rec=3D/dev/dsp \

-s 13,virtio-net,tap4 \

-s 14,virtio-9p,sharename=3D/ \

-s 29,fbuf,tcp=3D0.0.0.0:5900,w=3D1650,h=3D900,wait \

-s 30,xhci,tablet \

-s 31,lpc \

-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI_CODE.fd \<= /code>

vm0:4 < /dev/null 2>&1 &


---

**Root cause analysis (via ktrace)**

---

Tracing bhyve with `ktrace` reveals the following sequence:


  1. `openat("/dev/vmmctl", O_RDWR)` =E2=86=92 success

  2. `ioctl(VM_MMAP_GETNEXT)` =E2=86=92 success

  3. `ioctl(VM_MMAP_MEMSEG)` =E2=86=92 **errno 22 (EINVAL)**

=

The failure is inside `bootrom_alloc()` =E2=86=92 `vm_mmap_memseg()` = when trying to map the `VM_BOOTROM` memory segment.

Digging into `lib/libvmmapi/vmmapi.c`, the function=20 `vm_alloc_memseg()` first calls `vm_get_memseg()` to check if the=20 segment already exists. If it returns success with `len =3D=3D 0` (segment= =20 not yet allocated), it should proceed to call


`VM_ALLOC_MEMSEG` ioctl to create the segment. However, `VM_ALLOC_MEM= SEG` is **never called** in the ktrace output.


<= p dir=3D"auto"> The result is that `seg->object =3D=3D NULL` in the kernel when=20 `vm_mmap_memseg()` is later called for `VM_BOOTROM`, which returns=20 `EINVAL`.

---

**Workaround found**

---


Replacing `libvmmapi.so.6` (from the Corvin build) with the system stock `libvmmapi.so.7` fixes the bootrom issue =E2=80=94 the VM boots=20 successfully. However, this introduces a new problem: `bhyvectl=20 --vm=3Dname --create` returns `Operation not permitted` (EPERM) because=20 the stock libvmmapi uses different ioctls that vmm.ko Corvin doesn't=20 handle correctly.

What is the correct fix for `vm_alloc_memseg()` in the Corvin=20 libvmmapi to ensure `VM_ALLOC_MEMSEG` is properly called for=20 `VM_BOOTROM`? Is there a known patch or workaround that keeps=20 compatibility with vmm.ko Corvin while fixing the bootrom allocation?

Thanks!

---


*Relevant source: `lib/libvmmapi/vmmapi.c`, function `vm_alloc_memseg= ()` and `vm_create_devmem()`*

*vmm kernel side: `sys/dev/vmm/vmm_mem.c`, function `vm_mmap_memseg()= `*



--
Mario.
--0000000000006dad3a06536a1a6a--