From owner-freebsd-virtualization@freebsd.org Mon Feb 24 10:19:38 2020 Return-Path: Delivered-To: freebsd-virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 444FF2580EE for ; Mon, 24 Feb 2020 10:19:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.nyi.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id 48QyjQ0Vc0z4DPx for ; Mon, 24 Feb 2020 10:19:38 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.nyi.freebsd.org (Postfix) id E216D2580ED; Mon, 24 Feb 2020 10:19:37 +0000 (UTC) Delivered-To: virtualization@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E1B542580EC for ; Mon, 24 Feb 2020 10:19:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 48QyjP4SMZz4DP8 for ; Mon, 24 Feb 2020 10:19:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2610:1c1:1:606c::50:1d]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7BB7A23B22 for ; Mon, 24 Feb 2020 10:19:37 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.5]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id 01OAJbg3030849 for ; Mon, 24 Feb 2020 10:19:37 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id 01OAJbqO030848 for virtualization@FreeBSD.org; Mon, 24 Feb 2020 10:19:37 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 244363] Assertion failed: error == 0, file pci_emul.c, line 517, function modify_bar_registration Date: Mon, 24 Feb 2020 10:19:37 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: new X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: 12.0-STABLE X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: sjorge+signup@blackdot.be X-Bugzilla-Status: New X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: bug_id short_desc product version rep_platform op_sys bug_status bug_severity priority component assigned_to reporter Message-ID: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Feb 2020 10:19:38 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D244363 Bug ID: 244363 Summary: Assertion failed: error =3D=3D 0, file pci_emul.c, line 517, function modify_bar_registration Product: Base System Version: 12.0-STABLE Hardware: amd64 OS: Any Status: New Severity: Affects Some People Priority: --- Component: bhyve Assignee: virtualization@FreeBSD.org Reporter: sjorge+signup@blackdot.be I'm running into an issue when using PCI passthrough with bhyve. I am running into this on SmartOS but when googling the problem I found a f= ew FreeBSD users also running into the same problem. After chatting with Micha= el Dexter, I as suggested to also fine a bug bere. List of other people hitting the problem on FreeBSD: - https://forums.freebsd.org/threads/vm-bhyve-windows-2012-r2-and-passthru.60= 832/ - https://www.ixsystems.com/community/threads/bhyve-pci-passthrough-errors.57= 284/ - https://forums.freebsd.org/threads/bhyve-passthru-issue-with-lsi-logic-sas2= 008-- pci-express-fusion-mpt-sas-2.65269/ - bug #211062, comment #8 SmartOS bug report: - https://github.com/joyent/smartos-live/issues/901 Bhyve is dumping core when the following assert is tripped: Assertion failed: error =3D=3D 0, file pci_emul.c, line 517, function modify_bar_registration This is only tripped when the guest OS is windows (10). I do not have issues with a freebsd, illumos, or linux guest. So something windows is doing triggeres it. I also noticed I am only getting this on PCIe devices that have multiple BAR entries. I captured the output in a FreeBSD guest that got the PCIe devices via passthrough, you can see they have 2 BARs. ( Found of no easy way to capture this info on SmartOS, but this is probably easier for here anyway ) ``` ixl0@pci0:0:8:0: class=3D0x020000 card=3D0x37d215d9 chip=3D0x37d2808= 6 rev=3D0x09 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X722 for 10GBASE-T' class =3D network subclass =3D ethernet bar [10] =3D type Prefetchable Memory, range 64, base 0xc1000000, size 16777216, enabled bar [1c] =3D type Prefetchable Memory, range 64, base 0xc2000000, size 32768, enabled cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 05[50] =3D MSI supports 1 message, 64 bit, vector masks cap 11[70] =3D MSI-X supports 8 messages, enabled Table in map 0x1c[0x0], PBA in map 0x1c[0x1000] cap 10[a0] =3D PCI-Express 2 endpoint max data 256(512) FLR RO link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) cap 03[e0] =3D VPD VPD ident =3D 'Example VPD' ixl1@pci0:0:8:1: class=3D0x020000 card=3D0x37d215d9 chip=3D0x37d2808= 6 rev=3D0x09 hdr=3D0x00 vendor =3D 'Intel Corporation' device =3D 'Ethernet Connection X722 for 10GBASE-T' class =3D network subclass =3D ethernet bar [10] =3D type Prefetchable Memory, range 64, base 0xc3000000, size 16777216, enabled bar [1c] =3D type Prefetchable Memory, range 64, base 0xc4000000, size 32768, enabled cap 01[40] =3D powerspec 3 supports D0 D3 current D0 cap 05[50] =3D MSI supports 1 message, 64 bit, vector masks cap 11[70] =3D MSI-X supports 8 messages, enabled Table in map 0x1c[0x0], PBA in map 0x1c[0x1000] cap 10[a0] =3D PCI-Express 2 endpoint max data 256(512) FLR RO link x1(x1) speed 2.5(2.5) ASPM disabled(L0s/L1) cap 03[e0] =3D VPD VPD ident =3D 'Example VPD' ``` Michael Zeller who is also on the bhyve weekly called help me debug this a = bit (on SmartOS) We traced it to `Also the call to unregister_mem() ends up seeing an ENOENT from mmio_rb_lookup(&mmio_rb_root, memp->base, &entry);` mmio_rb_lookup is macro that we (illumos) copied from FreeBSD and it wasn't changed. So it would make sense that there is indeed a bug in there both FreeBSd and SmartOS bhyve users would hit the same error. --=20 You are receiving this mail because: You are the assignee for the bug.=