From owner-freebsd-virtualization@freebsd.org Sun Jun 16 20:18:01 2019 Return-Path: Delivered-To: freebsd-virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 6D98815C837E for ; Sun, 16 Jun 2019 20:18:01 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id EFBC78DF73 for ; Sun, 16 Jun 2019 20:18:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: by mailman.ysv.freebsd.org (Postfix) id A639D15C837B; Sun, 16 Jun 2019 20:18:00 +0000 (UTC) Delivered-To: virtualization@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7D2FB15C837A for ; Sun, 16 Jun 2019 20:18:00 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from mxrelay.ysv.freebsd.org (mxrelay.ysv.freebsd.org [IPv6:2001:1900:2254:206a::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.ysv.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DE8398DF6E for ; Sun, 16 Jun 2019 20:17:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org (kenobi.freebsd.org [IPv6:2001:1900:2254:206a::16:76]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.ysv.freebsd.org (Postfix) with ESMTPS id 0FE381C79D for ; Sun, 16 Jun 2019 20:17:59 +0000 (UTC) (envelope-from bugzilla-noreply@freebsd.org) Received: from kenobi.freebsd.org ([127.0.1.118]) by kenobi.freebsd.org (8.15.2/8.15.2) with ESMTP id x5GKHwKJ072584 for ; Sun, 16 Jun 2019 20:17:58 GMT (envelope-from bugzilla-noreply@freebsd.org) Received: (from www@localhost) by kenobi.freebsd.org (8.15.2/8.15.2/Submit) id x5GKHwvM072571 for virtualization@FreeBSD.org; Sun, 16 Jun 2019 20:17:58 GMT (envelope-from bugzilla-noreply@freebsd.org) X-Authentication-Warning: kenobi.freebsd.org: www set sender to bugzilla-noreply@freebsd.org using -f From: bugzilla-noreply@freebsd.org To: virtualization@FreeBSD.org Subject: [Bug 205549] bhyve pci passthru stops working after guest is restarted Date: Sun, 16 Jun 2019 20:17:56 +0000 X-Bugzilla-Reason: AssignedTo X-Bugzilla-Type: changed X-Bugzilla-Watch-Reason: None X-Bugzilla-Product: Base System X-Bugzilla-Component: bhyve X-Bugzilla-Version: CURRENT X-Bugzilla-Keywords: X-Bugzilla-Severity: Affects Some People X-Bugzilla-Who: julien.mstr320@gmail.com X-Bugzilla-Status: In Progress X-Bugzilla-Resolution: X-Bugzilla-Priority: --- X-Bugzilla-Assigned-To: virtualization@FreeBSD.org X-Bugzilla-Flags: X-Bugzilla-Changed-Fields: cc Message-ID: In-Reply-To: References: Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Bugzilla-URL: https://bugs.freebsd.org/bugzilla/ Auto-Submitted: auto-generated MIME-Version: 1.0 X-BeenThere: freebsd-virtualization@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Discussion of various virtualization techniques FreeBSD supports." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 16 Jun 2019 20:18:01 -0000 https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D205549 julien M changed: What |Removed |Added ---------------------------------------------------------------------------- CC| |julien.mstr320@gmail.com --- Comment #9 from julien M --- Having the same issue here with: FreeBSD 12.0-STABLE r349117 GENERIC amd64 (Generic kernel WITH xhci support) I patched stable with revisions 348778 and 348779 + D20623. FIRST TRY (vm create + vm install): BEFORE installing Ubuntu 18.04 on Bhyve: $ pciconf -r ppt4 0x3f 145c1022 00100007 0c033000 00800010 d9b00004 00000000 00000000 00000000 00000000 00000000 00000000 50071458 00000000 00000048 00000000 00000325 AFTER REBOOT: $ pciconf -r ppt4 0x3f 145c1022 00100407 0c033000 00800010 d9b00004 00000000 00000000 00000000 00000000 00000000 00000000 50071458 00000000 00000048 00000000 00000325 CHANGES: 00100007 =3D> 00100407 SECOND TRY (vm destroy, vm create, vm install): AFTER issuing: vm install linux linux.iso (while ubuntu is installing): $ pciconf -r ppt4 0x3f 145c1022 10100407 0c033000 00800010 d9b00004 00000000 00000000 00000000 00000000 00000000 00000000 50071458 00000000 00000048 00000000 00000325 CHANGES: 00100407 =3D> 10100407 THIRD TRY (vm destroy, vm create, vm install): AFTER issuing: vm install ...: $ pciconf -r ppt4 0x3f 145c1022 10100407 0c033000 00800010 d9b00004 00000000 00000000 00000000 00000000 00000000 00000000 50071458 00000000 00000048 00000000 00000325 CHANGES: NONE It seems that the FIRST install is breaking everything by changing 00100007= to 00100407. After a FreeBSD reboot, the 00100007 is back and passthrough is working when installing for the FIRST time linux in bhyve. This is the device to be passed: ppt2@pci0:8:0:0: class=3D0x130000 card=3D0x145a1022 chip=3D0x145a102= 2 rev=3D0x00 hdr=3D0x00 vendor =3D 'Advanced Micro Devices, Inc. [AMD]' device =3D 'Zeppelin/Raven/Raven2 PCIe Dummy Function' class =3D non-essential instrumentation ppt3@pci0:8:0:2: class=3D0x108000 card=3D0x14561022 chip=3D0x1456102= 2 rev=3D0x00 hdr=3D0x00 vendor =3D 'Advanced Micro Devices, Inc. [AMD]' device =3D 'Family 17h (Models 00h-0fh) Platform Security Processor' class =3D encrypt/decrypt ppt4@pci0:8:0:3: class=3D0x0c0330 card=3D0x50071458 chip=3D0x145c102= 2 rev=3D0x00 hdr=3D0x00 vendor =3D 'Advanced Micro Devices, Inc. [AMD]' device =3D 'Family 17h (Models 00h-0fh) USB 3.0 Host Controller' class =3D serial bus subclass =3D USB Configuration for bhyve-vm: ... passthru0=3D"8/0/0" passthru1=3D"8/0/2" passthru2=3D"8/0/3" ... I might be able to test this configuration in FreeBSD-CURRENT if that's of = any help? (compile times are low on a threadripper 1950x). --=20 You are receiving this mail because: You are the assignee for the bug.=