From owner-dev-commits-src-branches@freebsd.org Wed May 5 00:58:42 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9E3A5629C4D; Wed, 5 May 2021 00:58:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FZdcy44X2z4hV5; Wed, 5 May 2021 00:58:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7F0AF1B830; Wed, 5 May 2021 00:58:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 1450wg4B028391; Wed, 5 May 2021 00:58:42 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 1450wgbE028390; Wed, 5 May 2021 00:58:42 GMT (envelope-from git) Date: Wed, 5 May 2021 00:58:42 GMT Message-Id: <202105050058.1450wgbE028390@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 96033da5c38e - stable/12 - Set PCIe device's Max_Payload_Size to match PCIe root's. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 96033da5c38e20c27e2ca9007cca8bfab7e350cc Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 May 2021 00:58:42 -0000 The branch stable/12 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=96033da5c38e20c27e2ca9007cca8bfab7e350cc commit 96033da5c38e20c27e2ca9007cca8bfab7e350cc Author: Alexander Motin AuthorDate: 2021-04-05 14:34:40 +0000 Commit: Alexander Motin CommitDate: 2021-05-05 00:57:07 +0000 Set PCIe device's Max_Payload_Size to match PCIe root's. Usually on boot the MPS is already configured by BIOS. But we've found that on hot-plug it is not true at least for our Supermicro X11 boards. As result, mismatch between root's configuration of 256 bytes and device's default of 128 bytes cause problems for some devices, while others seem to work fine. MFC after: 1 month Sponsored by: iXsystems, Inc. (cherry picked from commit 5a898b2b78ce04d608bbaaa0813424b11f921ae7) --- sys/dev/pci/pci.c | 40 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/sys/dev/pci/pci.c b/sys/dev/pci/pci.c index f3a0309c5202..e167465db9b3 100644 --- a/sys/dev/pci/pci.c +++ b/sys/dev/pci/pci.c @@ -4248,6 +4248,45 @@ pci_create_iov_child_method(device_t bus, device_t pf, uint16_t rid, } #endif +/* + * For PCIe device set Max_Payload_Size to match PCIe root's. + */ +static void +pcie_setup_mps(device_t dev) +{ + struct pci_devinfo *dinfo = device_get_ivars(dev); + device_t root; + uint16_t rmps, mmps, mps; + + if (dinfo->cfg.pcie.pcie_location == 0) + return; + root = pci_find_pcie_root_port(dev); + if (root == NULL) + return; + /* Check whether the MPS is already configured. */ + rmps = pcie_read_config(root, PCIER_DEVICE_CTL, 2) & + PCIEM_CTL_MAX_PAYLOAD; + mps = pcie_read_config(dev, PCIER_DEVICE_CTL, 2) & + PCIEM_CTL_MAX_PAYLOAD; + if (mps == rmps) + return; + /* Check whether the device is capable of the root's MPS. */ + mmps = (pcie_read_config(dev, PCIER_DEVICE_CAP, 2) & + PCIEM_CAP_MAX_PAYLOAD) << 5; + if (rmps > mmps) { + /* + * The device is unable to handle root's MPS. Limit root. + * XXX: We should traverse through all the tree, applying + * it to all the devices. + */ + pcie_adjust_config(root, PCIER_DEVICE_CTL, + PCIEM_CTL_MAX_PAYLOAD, mmps, 2); + } else { + pcie_adjust_config(dev, PCIER_DEVICE_CTL, + PCIEM_CTL_MAX_PAYLOAD, rmps, 2); + } +} + static void pci_add_child_clear_aer(device_t dev, struct pci_devinfo *dinfo) { @@ -4335,6 +4374,7 @@ pci_add_child(device_t bus, struct pci_devinfo *dinfo) pci_cfg_restore(dev, dinfo); pci_print_verbose(dinfo); pci_add_resources(bus, dev, 0, 0); + pcie_setup_mps(dev); pci_child_added(dinfo->cfg.dev); if (pci_clear_aer_on_attach)