From nobody Sat Mar 8 08:13:59 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Z8wsh0trqz5qcfQ; Sat, 08 Mar 2025 08:14:00 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Z8wsg3hkHz3l8s; Sat, 08 Mar 2025 08:13:59 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1741421639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QSxBy5P8jxGSI/dksEYr8/TYAiCb3k6xeCYpkrNKXb0=; b=Ri9rOVNS7u4ktG6w+WbVDZCig4necV/WD/pSHmenzrLN6MlU0EjVeLzSI485sBLLh9F1LW +Ipsm5kof07Tc9Ga9Mhxjktc2HLeRau54edxtvbqFY3J3gOUsJtqv3wEncx8KNEa6gRibI qxlyENfwj5TRDxuG1bKz/n5S462DEiNicHQJ+E7SBpF6lDrig1NmLHIT5o2R26NNJy98xM RETqOcF+Qzrhz5NjK13FeJBrpMX56mElT9P0O0zIjW59uzdqowabSCP1pl+zEBdh01Tu3Q XQtE0J2i3uuptWVOdlTjoW/4/AJAWXNyIaALmDylLyVj2ekiy3gU/KlESpmsig== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1741421639; a=rsa-sha256; cv=none; b=LY8M2ZAf4cU2MXzee/LZCUY7XVlzDw1Dd0Nlys9Kv5N4369xKnDh2EzggBRBDqUOCs7fjD GgFi0OE6zfi4z/43JdTtnPT/opCo6306dPdf79wGgi6Lyd0A+NQ9isQ/HmIYADxlOiHrKG JcTU3Xe7R6y/vVf1I6aM4HZrw2l7ZSlpnFUGc5w3hSvgJzTgRgUS9dom8JRyincSZY2d6+ uV+8zccdevMOz1GPgMSE6Y825AbqPQWN4m9qZS+cR0ANVYq80UCUvVsNt+jHedwlQADPtp JuDcuOTHKqfnDVj4eKjwoynw6VzjBYQKsK0XO/wS7ZVFH6b1slMWH7HPz0eXTg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1741421639; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=QSxBy5P8jxGSI/dksEYr8/TYAiCb3k6xeCYpkrNKXb0=; b=c7spYhq/QZlTe7qMy2Fb35YUiQTypBt4P/qaOHqaInFSB7kHufRDDJecX0f9VN5U4aYlnq OJ8j/1ViORkcWi6z4iIZ9992a377QUE0Z3l1daaPDQjgRUeVdEtJFNQiZ267vOlvX7eC+v /T5uhz88mVXNvu9FABKSdQPIeBXsgokjJob6BB4Ubjq52Zw3sn+T+PxpDStTFGv7pKi+XQ 2S2rUfGs/CoNfwBfztmNOEiXgAb3VGNX/Bv83o7V18cD32ODfmsoHv0O2JG0sYdKiQu4WA QZdUB9ZCnSMRN1ssKIikT8FQK829HOhoq7/YVZ1fwcOi9oz0y/09gCbvkpyQIg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Z8wsg32KXzdB; Sat, 08 Mar 2025 08:13:59 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 5288DxGk097725; Sat, 8 Mar 2025 08:13:59 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 5288Dxpl097722; Sat, 8 Mar 2025 08:13:59 GMT (envelope-from git) Date: Sat, 8 Mar 2025 08:13:59 GMT Message-Id: <202503080813.5288Dxpl097722@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: bec64e75a5a8 - stable/14 - bhyve: Avoid holding /dev/pci open unnecessarily List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: bec64e75a5a8f0c1fa27f5d6081f52266516f393 Auto-Submitted: auto-generated The branch stable/14 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=bec64e75a5a8f0c1fa27f5d6081f52266516f393 commit bec64e75a5a8f0c1fa27f5d6081f52266516f393 Author: Mark Johnston AuthorDate: 2025-02-14 15:25:08 +0000 Commit: Mark Johnston CommitDate: 2025-03-07 22:51:49 +0000 bhyve: Avoid holding /dev/pci open unnecessarily Some device models, LPC in particular, will call pci_host_read_config() when probing for devices. Currently this results in pcifd_init() opening /dev/pci, and thus bhyve holds the fd open even when it's not needed. Modify pci_host_{read,write}_config() to open /dev/pci independent of the global pcifd. This means that these routines can only be used during VM initialization, as capsicum will prevent further opens afterward. Introduce internal wrappers which use the global pcifd, intended for the passthru code. Reviewed by: jhb MFC after: 3 weeks Fixes: 563fd2240e13 ("bhyve: export funcs for read/write pci config") Differential Revision: https://reviews.freebsd.org/D48908 (cherry picked from commit 649a910e34b0314647d16a94f8af6de0f4cfd4b5) --- usr.sbin/bhyve/pci_passthru.c | 119 +++++++++++++++++++++++++++++------------- 1 file changed, 83 insertions(+), 36 deletions(-) diff --git a/usr.sbin/bhyve/pci_passthru.c b/usr.sbin/bhyve/pci_passthru.c index 345971641f8a..8506cab24a1b 100644 --- a/usr.sbin/bhyve/pci_passthru.c +++ b/usr.sbin/bhyve/pci_passthru.c @@ -121,13 +121,24 @@ msi_caplen(int msgctrl) } static int -pcifd_init(void) +pcifd_open(void) { - pcifd = open(_PATH_DEVPCI, O_RDWR, 0); - if (pcifd < 0) { + int fd; + + fd = open(_PATH_DEVPCI, O_RDWR, 0); + if (fd < 0) { warn("failed to open %s", _PATH_DEVPCI); - return (1); + return (-1); } + return (fd); +} + +static int +pcifd_init(void) +{ + pcifd = pcifd_open(); + if (pcifd < 0) + return (1); #ifndef WITHOUT_CAPSICUM cap_rights_t pcifd_rights; @@ -144,43 +155,75 @@ pcifd_init(void) return (0); } -uint32_t -pci_host_read_config(const struct pcisel *sel, long reg, int width) +static uint32_t +host_read_config(int fd, const struct pcisel *sel, long reg, int width) { struct pci_io pi; - if (pcifd < 0 && pcifd_init()) { - return (0); - } - bzero(&pi, sizeof(pi)); pi.pi_sel = *sel; pi.pi_reg = reg; pi.pi_width = width; - if (ioctl(pcifd, PCIOCREAD, &pi) < 0) - return (0); /* XXX */ + if (ioctl(fd, PCIOCREAD, &pi) < 0) + return (0); /* XXX */ else return (pi.pi_data); } -void -pci_host_write_config(const struct pcisel *sel, long reg, int width, +static uint32_t +passthru_read_config(const struct pcisel *sel, long reg, int width) +{ + return (host_read_config(pcifd, sel, reg, width)); +} + +uint32_t +pci_host_read_config(const struct pcisel *sel, long reg, int width) +{ + uint32_t ret; + int fd; + + fd = pcifd_open(); + if (fd < 0) + return (0); + ret = host_read_config(fd, sel, reg, width); + (void)close(fd); + return (ret); +} + +static void +host_write_config(int fd, const struct pcisel *sel, long reg, int width, uint32_t data) { struct pci_io pi; - if (pcifd < 0 && pcifd_init()) { - return; - } - bzero(&pi, sizeof(pi)); pi.pi_sel = *sel; pi.pi_reg = reg; pi.pi_width = width; pi.pi_data = data; - (void)ioctl(pcifd, PCIOCWRITE, &pi); /* XXX */ + (void)ioctl(fd, PCIOCWRITE, &pi); /* XXX */ +} + +static void +passthru_write_config(const struct pcisel *sel, long reg, int width, + uint32_t data) +{ + host_write_config(pcifd, sel, reg, width, data); +} + +void +pci_host_write_config(const struct pcisel *sel, long reg, int width, + uint32_t data) +{ + int fd; + + fd = pcifd_open(); + if (fd < 0) + return; + host_write_config(fd, sel, reg, width, data); + (void)close(fd); } #ifdef LEGACY_SUPPORT @@ -225,24 +268,24 @@ cfginitmsi(struct passthru_softc *sc) * Parse the capabilities and cache the location of the MSI * and MSI-X capabilities. */ - sts = pci_host_read_config(&sel, PCIR_STATUS, 2); + sts = passthru_read_config(&sel, PCIR_STATUS, 2); if (sts & PCIM_STATUS_CAPPRESENT) { - ptr = pci_host_read_config(&sel, PCIR_CAP_PTR, 1); + ptr = passthru_read_config(&sel, PCIR_CAP_PTR, 1); while (ptr != 0 && ptr != 0xff) { - cap = pci_host_read_config(&sel, ptr + PCICAP_ID, 1); + cap = passthru_read_config(&sel, ptr + PCICAP_ID, 1); if (cap == PCIY_MSI) { /* * Copy the MSI capability into the config * space of the emulated pci device */ sc->psc_msi.capoff = ptr; - sc->psc_msi.msgctrl = pci_host_read_config(&sel, - ptr + 2, 2); + sc->psc_msi.msgctrl = + passthru_read_config(&sel, ptr + 2, 2); sc->psc_msi.emulated = 0; caplen = msi_caplen(sc->psc_msi.msgctrl); capptr = ptr; while (caplen > 0) { - u32 = pci_host_read_config(&sel, capptr, + u32 = passthru_read_config(&sel, capptr, 4); pci_set_cfgdata32(pi, capptr, u32); caplen -= 4; @@ -257,7 +300,7 @@ cfginitmsi(struct passthru_softc *sc) msixcap_ptr = (char *)&msixcap; capptr = ptr; while (caplen > 0) { - u32 = pci_host_read_config(&sel, capptr, + u32 = passthru_read_config(&sel, capptr, 4); memcpy(msixcap_ptr, &u32, 4); pci_set_cfgdata32(pi, capptr, u32); @@ -266,7 +309,7 @@ cfginitmsi(struct passthru_softc *sc) msixcap_ptr += 4; } } - ptr = pci_host_read_config(&sel, ptr + PCICAP_NEXTPTR, + ptr = passthru_read_config(&sel, ptr + PCICAP_NEXTPTR, 1); } } @@ -302,7 +345,7 @@ cfginitmsi(struct passthru_softc *sc) */ if ((sts & PCIM_STATUS_CAPPRESENT) != 0 && sc->psc_msi.capoff == 0) { int origptr, msiptr; - origptr = pci_host_read_config(&sel, PCIR_CAP_PTR, 1); + origptr = passthru_read_config(&sel, PCIR_CAP_PTR, 1); msiptr = passthru_add_msicap(pi, 1, origptr); sc->psc_msi.capoff = msiptr; sc->psc_msi.msgctrl = pci_get_cfgdata16(pi, msiptr + 2); @@ -536,6 +579,8 @@ cfginitbar(struct passthru_softc *sc) * Initialize BAR registers */ for (i = 0; i <= PCI_BARMAX; i++) { + uint8_t lobits; + bzero(&bar, sizeof(bar)); bar.pbi_sel = sc->psc_sel; bar.pbi_reg = PCIR_BAR(i); @@ -581,8 +626,8 @@ cfginitbar(struct passthru_softc *sc) return (-1); /* Use same lobits as physical bar */ - uint8_t lobits = pci_host_read_config(&sc->psc_sel, PCIR_BAR(i), - 0x01); + lobits = (uint8_t)passthru_read_config(&sc->psc_sel, + PCIR_BAR(i), 0x01); if (bartype == PCIBAR_MEM32 || bartype == PCIBAR_MEM64) { lobits &= ~PCIM_BAR_MEM_BASE; } else { @@ -629,7 +674,7 @@ cfginit(struct pci_devinst *pi, int bus, int slot, int func) intpin = pci_get_cfgdata8(pi, PCIR_INTPIN); for (int i = 0; i <= PCIR_MAXLAT; i += 4) { pci_set_cfgdata32(pi, i, - pci_host_read_config(&sc->psc_sel, i, 4)); + passthru_read_config(&sc->psc_sel, i, 4)); } pci_set_cfgdata16(pi, PCIR_COMMAND, cmd); pci_set_cfgdata8(pi, PCIR_INTLINE, intline); @@ -988,15 +1033,17 @@ passthru_cfgread_default(struct passthru_softc *sc, * device's config space. */ if (coff == PCIR_COMMAND) { + uint32_t st; + if (bytes <= 2) return (-1); - *rv = pci_host_read_config(&sc->psc_sel, PCIR_STATUS, 2) << 16 | - pci_get_cfgdata16(pi, PCIR_COMMAND); + st = passthru_read_config(&sc->psc_sel, PCIR_STATUS, 2); + *rv = (st << 16) | pci_get_cfgdata16(pi, PCIR_COMMAND); return (0); } /* Everything else just read from the device's config space */ - *rv = pci_host_read_config(&sc->psc_sel, coff, bytes); + *rv = passthru_read_config(&sc->psc_sel, coff, bytes); return (0); } @@ -1079,7 +1126,7 @@ passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, return (-1); /* Update the physical status register. */ - pci_host_write_config(&sc->psc_sel, PCIR_STATUS, val >> 16, 2); + passthru_write_config(&sc->psc_sel, PCIR_STATUS, val >> 16, 2); /* Update the virtual command register. */ cmd_old = pci_get_cfgdata16(pi, PCIR_COMMAND); @@ -1088,7 +1135,7 @@ passthru_cfgwrite_default(struct passthru_softc *sc, struct pci_devinst *pi, return (0); } - pci_host_write_config(&sc->psc_sel, coff, bytes, val); + passthru_write_config(&sc->psc_sel, coff, bytes, val); return (0); }