From nobody Sat Jan 27 22:29:06 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TMq3k2MP3z593V1; Sat, 27 Jan 2024 22:29:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TMq3k1n8Tz4ChB; Sat, 27 Jan 2024 22:29:06 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706394546; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TtVSLkCqUEopS82JB3YYUXn5lvrFKbhSTniNRGQrxJE=; b=EtCGtXL6qq5hSv786Vxu5uBQfFik7yyEd+yK7WCFAz/fQeVjeAdgQxHF4XH0k7CB49Z1Ye Q0s+HXVoSlR7LKid6mZxNxs64yX4IPNJZ4hSR3+fRqtVn8KpbOvKBK/1BMreevmoGSy0Zg kVKA+pQjdbm2xkrupZI3qC1x9fxFJxTdTPDPpDaPnVn9XcGIe9igBM7TQlOcLCS4qqPqnn g+A7is7Ct4rRpEIPGmLdNeth2tks2qzH7YlOCuEaBw/QohRjajd+H/I9ckuzSZToqObk79 k9KdYlSqTiDTi/FZdBPUgvxE5x9QahhkOApX7QZWObRzdhND3iylIYLgpRxoew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1706394546; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=TtVSLkCqUEopS82JB3YYUXn5lvrFKbhSTniNRGQrxJE=; b=joGyQSoA20P1h58ZeDu97QTItjZL38xQ0B4x9DYsnEl2QvSv8yDvrb5sEjqIESaCj3yAIG a2TMBhjFwlgEaK0mY3mjGjlcI4yafPKRT171a77X+2EIyta1C9wb3P7SrQE2zEug7fYRlJ 61qM+ElbHBlFOKzIiHbE/WJ/NkbX8EfeHE/5hLQh+GAKR/AUvP0R3Cf8JK3oers7YB/M30 Uvva0WPImEBOMVOgz26yoliSbHw546MgnfjAGVv68WDdKTEjdT7g3EukxWpcEKOf3DkXC/ YRY4ah0dxaCPyPznZawoKlF5wSf0IxCdL97fYJp3TDWRtbLg/PLaOg8VARp8oA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1706394546; a=rsa-sha256; cv=none; b=uRQegTjjSpo/aHWwYX+avgi5fWmSzHVdeITursEJWLnzA641Mxt7CFgVtukFIEjs+tormR upqDIRVnT/wt7QNum4cDXTKbsAB3dT8uOgwP4mtGPQvRxPG/NMaXexI+ToUdRs2iDWaLt3 eQ2as8BMbLuw9nGgEagCxrPYSC3+mQL3EcZwKf2Aijp+MiJsga9IrVb5Q5cUpCpLkpLbxl tmJVbylfoshh3khLLqWi7K/D8TBarOJTBXeFN+iB+/RShq33a5+RcLhQbVsgPd9zMMG1dN GfsBwDCnLsfRu0S2K5G+7/e5XyNEneXt2loLUUkD1+VlrOFOdUAJaYj7HXMOAQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TMq3k0rnyzjbG; Sat, 27 Jan 2024 22:29:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 40RMT6xx043540; Sat, 27 Jan 2024 22:29:06 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 40RMT6Qu043537; Sat, 27 Jan 2024 22:29:06 GMT (envelope-from git) Date: Sat, 27 Jan 2024 22:29:06 GMT Message-Id: <202401272229.40RMT6Qu043537@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alexander Motin Subject: git: 3883c6fbf232 - main - ntb_hw_plx: Workaround read-only scratchpad registers List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3883c6fbf232452098ba6ea802ef1426d83d2d68 Auto-Submitted: auto-generated The branch main has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=3883c6fbf232452098ba6ea802ef1426d83d2d68 commit 3883c6fbf232452098ba6ea802ef1426d83d2d68 Author: Alexander Motin AuthorDate: 2024-01-27 21:51:46 +0000 Commit: Alexander Motin CommitDate: 2024-01-27 22:29:01 +0000 ntb_hw_plx: Workaround read-only scratchpad registers On several systems we've noticed that when NTB link goes down, the Physical Layer User Test Pattern registers we use as additional scratchpad registers (that is explicitly allowed by the chip specs) become read-only for about 100us. I see no explanation for this in the chip specs, neither why it was not seen before, may be a race. Since we do need these registers, workaround it by repeating writes until we succeed or 1ms timeout expire. MFC after: 1 week --- sys/dev/ntb/ntb.h | 2 +- sys/dev/ntb/ntb_hw/ntb_hw_plx.c | 26 +++++++++++++++++++++----- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/sys/dev/ntb/ntb.h b/sys/dev/ntb/ntb.h index f9119fa0aedc..e2af625fff94 100644 --- a/sys/dev/ntb/ntb.h +++ b/sys/dev/ntb/ntb.h @@ -279,7 +279,7 @@ int ntb_mw_set_wc(device_t ntb, unsigned mw_idx, vm_memattr_t mode); uint8_t ntb_spad_count(device_t ntb); /* - * ntb_get_max_spads() - zero local scratch registers + * ntb_spad_clear() - zero local scratch registers * @ntb: pointer to ntb_softc instance * * This functions overwrites all local scratchpad registers with zeroes. diff --git a/sys/dev/ntb/ntb_hw/ntb_hw_plx.c b/sys/dev/ntb/ntb_hw/ntb_hw_plx.c index 57e3265ee5b1..906660116067 100644 --- a/sys/dev/ntb/ntb_hw/ntb_hw_plx.c +++ b/sys/dev/ntb/ntb_hw/ntb_hw_plx.c @@ -879,17 +879,33 @@ static int ntb_plx_spad_write(device_t dev, unsigned int idx, uint32_t val) { struct ntb_plx_softc *sc = device_get_softc(dev); - u_int off; + u_int off, t; if (idx >= sc->spad_count1 + sc->spad_count2) return (EINVAL); - if (idx < sc->spad_count1) + if (idx < sc->spad_count1) { off = sc->spad_off1 + idx * 4; - else + bus_write_4(sc->conf_res, off, val); + return (0); + } else { off = sc->spad_off2 + (idx - sc->spad_count1) * 4; - bus_write_4(sc->conf_res, off, val); - return (0); + /* + * For some reason when link goes down Test Pattern registers + * we use as additional scratchpad become read-only for about + * 100us. I see no explanation in specs, so just wait a bit. + */ + for (t = 0; t <= 1000; t++) { + bus_write_4(sc->conf_res, off, val); + if (bus_read_4(sc->conf_res, off) == val) + return (0); + DELAY(1); + } + device_printf(dev, + "Can't write Physical Layer User Test Pattern (0x%x)\n", + off); + return (EIO); + } } static void