From owner-svn-src-all@freebsd.org Tue May 3 19:45:26 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 3E8A2B2C109; Tue, 3 May 2016 19:45:26 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id F2C4010D2; Tue, 3 May 2016 19:45:25 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u43JjPnb061874; Tue, 3 May 2016 19:45:25 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u43JjODt061871; Tue, 3 May 2016 19:45:24 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201605031945.u43JjODt061871@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Tue, 3 May 2016 19:45:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r299002 - head/sys/dev/pci X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 May 2016 19:45:26 -0000 Author: jhb Date: Tue May 3 19:45:24 2016 New Revision: 299002 URL: https://svnweb.freebsd.org/changeset/base/299002 Log: Save and restore SRIOV-related config registers. Save the value of the IOV control and page size registers and restore them (along with the VF count) in pci_cfg_save/pci_cfg_restore. This ensures ARI remains enabled if a PF driver resets itself during the PCI_IOV_INIT callback. This might also properly restore SRIOV state across suspend/resume. Reviewed by: rstone, vangyzen Differential Revision: https://reviews.freebsd.org/D6192 Modified: head/sys/dev/pci/pci.c head/sys/dev/pci/pci_iov.c head/sys/dev/pci/pci_iov_private.h Modified: head/sys/dev/pci/pci.c ============================================================================== --- head/sys/dev/pci/pci.c Tue May 3 19:42:58 2016 (r299001) +++ head/sys/dev/pci/pci.c Tue May 3 19:45:24 2016 (r299002) @@ -5643,6 +5643,10 @@ pci_cfg_restore(device_t dev, struct pci pci_resume_msi(dev); if (dinfo->cfg.msix.msix_location != 0) pci_resume_msix(dev); + + if (dinfo->cfg.iov != NULL) + pci_iov_cfg_restore(dev, dinfo); + } static void @@ -5755,6 +5759,9 @@ pci_cfg_save(device_t dev, struct pci_de if (dinfo->cfg.pcix.pcix_location != 0) pci_cfg_save_pcix(dev, dinfo); + if (dinfo->cfg.iov != NULL) + pci_iov_cfg_save(dev, dinfo); + /* * don't set the state for display devices, base peripherals and * memory devices since bad things happen when they are powered down. Modified: head/sys/dev/pci/pci_iov.c ============================================================================== --- head/sys/dev/pci/pci_iov.c Tue May 3 19:42:58 2016 (r299001) +++ head/sys/dev/pci/pci_iov.c Tue May 3 19:45:24 2016 (r299002) @@ -770,6 +770,29 @@ out: return (error); } +void +pci_iov_cfg_restore(device_t dev, struct pci_devinfo *dinfo) +{ + struct pcicfg_iov *iov; + + iov = dinfo->cfg.iov; + + IOV_WRITE(dinfo, PCIR_SRIOV_PAGE_SIZE, iov->iov_page_size, 4); + IOV_WRITE(dinfo, PCIR_SRIOV_NUM_VFS, iov->iov_num_vfs, 2); + IOV_WRITE(dinfo, PCIR_SRIOV_CTL, iov->iov_ctl, 2); +} + +void +pci_iov_cfg_save(device_t dev, struct pci_devinfo *dinfo) +{ + struct pcicfg_iov *iov; + + iov = dinfo->cfg.iov; + + iov->iov_page_size = IOV_READ(dinfo, PCIR_SRIOV_PAGE_SIZE, 4); + iov->iov_ctl = IOV_READ(dinfo, PCIR_SRIOV_CTL, 2); +} + /* Return true if child is a VF of the given PF. */ static int pci_iov_is_child_vf(struct pcicfg_iov *pf, device_t child) Modified: head/sys/dev/pci/pci_iov_private.h ============================================================================== --- head/sys/dev/pci/pci_iov_private.h Tue May 3 19:42:58 2016 (r299001) +++ head/sys/dev/pci/pci_iov_private.h Tue May 3 19:45:24 2016 (r299002) @@ -47,10 +47,16 @@ struct pcicfg_iov { int iov_pos; int iov_num_vfs; uint32_t iov_flags; + + uint16_t iov_ctl; + uint32_t iov_page_size; }; #define IOV_RMAN_INITED 0x0001 #define IOV_BUSY 0x0002 +void pci_iov_cfg_restore(device_t dev, struct pci_devinfo *dinfo); +void pci_iov_cfg_save(device_t dev, struct pci_devinfo *dinfo); + #endif