Date: Wed, 9 Aug 2023 15:00:23 +0200 From: Santiago Martinez <sm@codenetworks.net> To: skorpio dr <drskorp@gmail.com>, freebsd-current@freebsd.org, Michael Dexter <editor@callfortesting.org> Subject: Re: Bhyve NIC Passthru broken Message-ID: <c0cd4888-b1e8-0cea-7577-14e88ab0bd1e@codenetworks.net> In-Reply-To: <CAEug4ZiSd2U5w4XdMg94A3Efp6TwMUeSq0auPt5hjbydCaq9AQ@mail.gmail.com> References: <CAEug4ZiSd2U5w4XdMg94A3Efp6TwMUeSq0auPt5hjbydCaq9AQ@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi there! I'm seeing the same, but at the end the machine end crashing. Seems that a complete power-off/on of the hardware clears it, until it random happen again when loading vmm and doing passthru. With 13.1p7 it was solid. I'm adding Micheal as he probably know who is the right person to look into it. Santi On 8/3/23 06:48, skorpio dr wrote: > Hello, > > Starting a vm (FreeBSD 13.2) with passthru NIC results a constant > flood of error messages on the host (FreeBSD 14-CURRENT). > The NIC refuses to work in the vm (it works on the host if I turn off > the passthru). > This problem is persistent since 13.1-Release. > (Sidenote: the passthru mostly works under the EOL 13.0-p13 with the > same config, only occasionally showing the symptoms after multiple vm > reboots, and a host reboot solves the problem unlike in 13.1-release > 13.2-release and now 14.0-current) > > Ruling out NIC/board hardware fault and specific NIC driver problems, > I've tried with the following NICs: > - Intel I350-t4 > - Intel 82580 > - Broadcom BCM5720 > > ...in different desktop and server boards (with the latest BIOS): > - Asus M5A99X EVO R2.0 > - Asus M4A87TD EVO > - Asus KGPE-D16 > > # freebsd-version -kru > 14.0-CURRENT > 14.0-CURRENT > 14.0-CURRENT > > /boot/loader.conf: > vmm_load="YES" > nmdm_load="YES" > hw.vmm.amdvi.enable=1 > pptdevs="8/0/0 8/0/1" > > # vm passthru|grep ppt0 > ppt0 8/0/0 Yes 82580 Gigabit Network Connection > > the vm config file for vm-bhyve: > loader="bhyveload" > cpu=2 > memory=4G > disk0_type="virtio-blk" > disk0_name="ptproba.img" > # igb0: > passthru0="8/0/0" > uuid="2333ba09-dfe1-11ed-9ed6-9c5c8e54ad77" > > vm-bhyve.log: > > Aug 03 02:25:16: initialising > Aug 03 02:25:16: [loader: bhyveload] > Aug 03 02:25:16: [cpu: 2] > Aug 03 02:25:16: [memory: 4G] > Aug 03 02:25:16: [hostbridge: standard] > Aug 03 02:25:16: [com ports: com1] > Aug 03 02:25:16: [uuid: 2333ba09-dfe1-11ed-9ed6-9c5c8e54ad77] > Aug 03 02:25:16: [debug mode: no] > Aug 03 02:25:16: [primary disk: ptproba.img] > Aug 03 02:25:16: [primary disk dev: file] > Aug 03 02:25:16: booting > Aug 03 02:25:16: bhyveload -c /dev/nmdm-ptproba.1A -S -m 4G -e > autoboot_delay=3 -d /root/vm-bhyve/ptproba/ptproba.img ptproba > Aug 03 02:25:22: [bhyve options: -c 2 -m 4G -AHP -U > 2333ba09-dfe1-11ed-9ed6-9c5c8e54ad77 -u -S] > Aug 03 02:25:22: [bhyve devices: -s 0,hostbridge -s 31,lpc -s > 4:0,virtio-blk,/root/vm-bhyve/ptproba/ptproba.img -s 5:0,passthru,8/0/0] > Aug 03 02:25:22: [bhyve console: -l com1,/dev/nmdm-ptproba.1A] > Aug 03 02:25:22: starting bhyve (run 1) > > The first few dozen messages are like the followings (couple of > hundred lines): > > ivhd0: EVT INTR 0 Status:0x1a EVT Head:0x0 Tail:0x10] > [CMD Total 0x24] Tail:0x240, Head:0x240. > ivhd0: [Event0: Head:0x0 Tail:0x10] > [INV_DTE devid:0xa0 addr:0xfdf9103300 type:0x3 tr:0] > ivhd0: EVT INTR 1 Status:0x1a EVT Head:0x10 Tail:0x20] > [CMD Total 0x26] Tail:0x260, Head:0x260. > ivhd0: [Event0: Head:0x10 Tail:0x20] > [INV_DTE devid:0xa0 addr:0xfdf9103300 type:0x3 tr:0] > ivhd0: EVT INTR 2 Status:0x1a EVT Head:0x20 Tail:0x30] > [CMD Total 0x26] Tail:0x260, Head:0x260. > ivhd0: [Event0: Head:0x20 Tail:0x30] > [INV_DTE devid:0xa0 addr:0xfdf9103300 type:0x3 tr:0] > > ...then the constant flood (about 35 lines per second in > /var/log/messages): > > ivhd0: [Event63: Head:0x3e0 Tail:0x8d0] > [INV_DTE devid:0xa0 addr:0xfdf9103300 type:0x3 tr:0] > ivhd0: [Event64: Head:0x3f0 Tail:0x8d0] > [INV_DTE devid:0xa0 addr:0xfdf9103300 type:0x3 tr:0] > ivhd0: [Event65: Head:0x400 Tail:0x8d0] > [INV_DTE devid:0xa0 addr:0xfdf9103300 type:0x3 tr:0] > > Any idea? > > Best
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c0cd4888-b1e8-0cea-7577-14e88ab0bd1e>