From owner-svn-src-head@freebsd.org Wed May 2 18:19:11 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 854C4FB0C21; Wed, 2 May 2018 18:19:11 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (br1.CN84in.dnsmgr.net [69.59.192.140]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EDC6D7BA6A; Wed, 2 May 2018 18:19:10 +0000 (UTC) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: from pdx.rh.CN85.dnsmgr.net (localhost [127.0.0.1]) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3) with ESMTP id w42IJ1vC032638; Wed, 2 May 2018 11:19:01 -0700 (PDT) (envelope-from freebsd@pdx.rh.CN85.dnsmgr.net) Received: (from freebsd@localhost) by pdx.rh.CN85.dnsmgr.net (8.13.3/8.13.3/Submit) id w42IJ1uJ032637; Wed, 2 May 2018 11:19:01 -0700 (PDT) (envelope-from freebsd) From: "Rodney W. Grimes" Message-Id: <201805021819.w42IJ1uJ032637@pdx.rh.CN85.dnsmgr.net> Subject: Re: svn commit: r333174 - head/sys/amd64/vmm/io In-Reply-To: <201805021741.w42Hf06u078873@repo.freebsd.org> To: Peter Grehan Date: Wed, 2 May 2018 11:19:01 -0700 (PDT) CC: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Reply-To: rgrimes@freebsd.org X-Mailer: ELM [version 2.4ME+ PL121h (25)] MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 02 May 2018 18:19:11 -0000 [ Charset UTF-8 unsupported, converting... ] > Author: grehan > Date: Wed May 2 17:41:00 2018 > New Revision: 333174 > URL: https://svnweb.freebsd.org/changeset/base/333174 > > Log: > Use PCI power-mgmt to reset a device if FLR fails. > > A large number of devices don't support PCIe FLR, in particular > graphics adapters. Use PCI power management to perform the > reset if FLR fails or isn't available, by cycling the device > through the D3 state. > > This has been tested by a number of users with Nvidia and AMD GPUs. > > Submitted and tested by: Matt Macy > Reviewed by: jhb, imp, rgrimes > MFC after: 3 weeks That places the MFC right before the optional 11.2 Beta3, I would rather see this "low risk" change merged before May 11th, the code freeze date and Beta1, if it is desired in 11.2, for maximal test exposure. > Differential Revision: https://reviews.freebsd.org/D15268 > > Modified: > head/sys/amd64/vmm/io/ppt.c > > Modified: head/sys/amd64/vmm/io/ppt.c > ============================================================================== > --- head/sys/amd64/vmm/io/ppt.c Wed May 2 15:59:15 2018 (r333173) > +++ head/sys/amd64/vmm/io/ppt.c Wed May 2 17:41:00 2018 (r333174) > @@ -353,6 +353,30 @@ ppt_is_mmio(struct vm *vm, vm_paddr_t gpa) > return (FALSE); > } > > +static void > +ppt_pci_reset(device_t dev) > +{ > + int ps; > + > + if (pcie_flr(dev, > + max(pcie_get_max_completion_timeout(dev) / 1000, 10), > + true)) > + return; > + > + /* > + * If FLR fails, attempt a power-management reset by cycling > + * the device in/out of D3 state. > + * PCI spec says we can only go into D3 state from D0 state. > + * Transition from D[12] into D0 before going to D3 state. > + */ > + ps = pci_get_powerstate(dev); > + if (ps != PCI_POWERSTATE_D0 && ps != PCI_POWERSTATE_D3) > + pci_set_powerstate(dev, PCI_POWERSTATE_D0); > + if (pci_get_powerstate(dev) != PCI_POWERSTATE_D3) > + pci_set_powerstate(dev, PCI_POWERSTATE_D3); > + pci_set_powerstate(dev, ps); > +} > + > int > ppt_assign_device(struct vm *vm, int bus, int slot, int func) > { > @@ -368,9 +392,7 @@ ppt_assign_device(struct vm *vm, int bus, int slot, in > return (EBUSY); > > pci_save_state(ppt->dev); > - pcie_flr(ppt->dev, > - max(pcie_get_max_completion_timeout(ppt->dev) / 1000, 10), > - true); > + ppt_pci_reset(ppt->dev); > pci_restore_state(ppt->dev); > ppt->vm = vm; > iommu_add_device(vm_iommu_domain(vm), pci_get_rid(ppt->dev)); > @@ -393,9 +415,7 @@ ppt_unassign_device(struct vm *vm, int bus, int slot, > return (EBUSY); > > pci_save_state(ppt->dev); > - pcie_flr(ppt->dev, > - max(pcie_get_max_completion_timeout(ppt->dev) / 1000, 10), > - true); > + ppt_pci_reset(ppt->dev); > pci_restore_state(ppt->dev); > ppt_unmap_mmio(vm, ppt); > ppt_teardown_msi(ppt); > > -- Rod Grimes rgrimes@freebsd.org