From owner-svn-src-all@freebsd.org Thu May 9 09:21:49 2019 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2C61215A313A; Thu, 9 May 2019 09:21:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 5CF3B86249; Thu, 9 May 2019 09:21:48 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id x499LepQ084980 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Thu, 9 May 2019 12:21:43 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua x499LepQ084980 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id x499LerO084978; Thu, 9 May 2019 12:21:40 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Thu, 9 May 2019 12:21:40 +0300 From: Konstantin Belousov To: John Baldwin Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: Re: svn commit: r347374 - stable/11/sys/amd64/vmm/io Message-ID: <20190509092140.GD2748@kib.kiev.ua> References: <201905082324.x48NOlcJ062685@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201905082324.x48NOlcJ062685@repo.freebsd.org> User-Agent: Mutt/1.11.4 (2019-03-13) X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on tom.home X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 May 2019 09:21:49 -0000 On Wed, May 08, 2019 at 11:24:47PM +0000, John Baldwin wrote: > Author: jhb > Date: Wed May 8 23:24:47 2019 > New Revision: 347374 > URL: https://svnweb.freebsd.org/changeset/base/347374 > > Log: > MFC 333174: Use PCI power-mgmt to reset a device if FLR fails. > > A large number of devices don't support PCIe FLR, in particular > graphics adapters. Use PCI power management to perform the > reset if FLR fails or isn't available, by cycling the device > through the D3 state. > > This has been tested by a number of users with Nvidia and AMD GPUs. Now stable/11 ppt_pci_reset() can be made identical to the HEAD version, I merged all reset changes some time ago. > > Modified: > stable/11/sys/amd64/vmm/io/ppt.c > Directory Properties: > stable/11/ (props changed) > > Modified: stable/11/sys/amd64/vmm/io/ppt.c > ============================================================================== > --- stable/11/sys/amd64/vmm/io/ppt.c Wed May 8 23:13:27 2019 (r347373) > +++ stable/11/sys/amd64/vmm/io/ppt.c Wed May 8 23:24:47 2019 (r347374) > @@ -351,6 +351,30 @@ ppt_is_mmio(struct vm *vm, vm_paddr_t gpa) > return (FALSE); > } > > +static void > +ppt_pci_reset(device_t dev) > +{ > + int ps; > + > + if (pcie_flr(dev, > + max(pcie_get_max_completion_timeout(dev) / 1000, 10), > + true)) > + return; > + > + /* > + * If FLR fails, attempt a power-management reset by cycling > + * the device in/out of D3 state. > + * PCI spec says we can only go into D3 state from D0 state. > + * Transition from D[12] into D0 before going to D3 state. > + */ > + ps = pci_get_powerstate(dev); > + if (ps != PCI_POWERSTATE_D0 && ps != PCI_POWERSTATE_D3) > + pci_set_powerstate(dev, PCI_POWERSTATE_D0); > + if (pci_get_powerstate(dev) != PCI_POWERSTATE_D3) > + pci_set_powerstate(dev, PCI_POWERSTATE_D3); > + pci_set_powerstate(dev, ps); > +} > + > int > ppt_assign_device(struct vm *vm, int bus, int slot, int func) > { > @@ -366,9 +390,7 @@ ppt_assign_device(struct vm *vm, int bus, int slot, in > return (EBUSY); > > pci_save_state(ppt->dev); > - pcie_flr(ppt->dev, > - max(pcie_get_max_completion_timeout(ppt->dev) / 1000, 10), > - true); > + ppt_pci_reset(ppt->dev); > pci_restore_state(ppt->dev); > ppt->vm = vm; > iommu_add_device(vm_iommu_domain(vm), pci_get_rid(ppt->dev)); > @@ -391,9 +413,7 @@ ppt_unassign_device(struct vm *vm, int bus, int slot, > return (EBUSY); > > pci_save_state(ppt->dev); > - pcie_flr(ppt->dev, > - max(pcie_get_max_completion_timeout(ppt->dev) / 1000, 10), > - true); > + ppt_pci_reset(ppt->dev); > pci_restore_state(ppt->dev); > ppt_unmap_mmio(vm, ppt); > ppt_teardown_msi(ppt);