From nobody Mon Feb 5 20:39:24 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TTJC10b1jz58y7p; Mon, 5 Feb 2024 20:39:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TTJC040cMz4rLH; Mon, 5 Feb 2024 20:39:24 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707165564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lVxZYZgWk+VlsC8rEheJxpITmO6bmEzLR/U3GIDQl48=; b=PA06mqs0dY8YwLsWOUFk1Ee5iHmb5lylygSukM/a8BoM/Nx0vZcE5n2t/r48Vrf7Jjl16g fTBM6lSCxOecH81SA4DNja50Ut2M9pUS3tADkKY2HI/5tcfI4pD3e9tk7AjQ5PdIdn9sfm tqOQIBufurjggaOZTZHOLCCVqcYy+5DoQL+45Byyy1xV5J/bzRmkhBss0QpTjPB3iwqvrJ Y9j9pTqwfGndTVRkwBTidjS3fG73oc0FxuZfjFwWfmbo+1KE7b89J2WJdVhHS91A3SPqXC kXas1BaODVG9L1pPQcMn4X0xaMj52/hMoxdKhOhwSOuNt99yF5p4f5drpVF1dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707165564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=lVxZYZgWk+VlsC8rEheJxpITmO6bmEzLR/U3GIDQl48=; b=H4kgKiMOKWWmVAxhuz3CPc74AKHTdFUWAkI/eeU9PpUFtXmHtyD0lcHk2PGrIsmLcHxxbf nN3eJGJtNnVaHBBOxxJHWE6VLrChiSqNHKNNkxDFnXfyJ7TDXKjmGVZrGepGGMrA5yxw/g nWxubaaivl6LIXD2d3StpFL/X6/ssrZIaQeGT+Gfw/+KAm+QIuzlLTNNkXic2OtKEWhTmQ 8xoX9dRf9Madnct1UbViTUGnOLGbDkpraL8pExIUEREnE5CKyzMuhp3V61FNZ5kN5WoITd n6G9FH3sVcAO3DO5DotI5IAnHZ+B5fEBidqkU5uh02k943/5OERty613SB/Olg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707165564; a=rsa-sha256; cv=none; b=wIxROZn5/g6tWHpv8bVd6VrO3RkfXPjcfDdUnVbAbgV356uet+592d3jdUUhHVgUJmwvEa 0MYo7UxiDy9eZphj0r6gnWKED3ZOeuQmCLZrR/8z+dwzMbmBZ/w7JJXwG3M6iP/V5p+3sW 0IxDkd+9YPSbRDHYTfOsvsO7QeAS+73oovx3zCFcgn8rkNeqEBJXkmebNsd7ktOy5kntiv i4riQfLt8o5npid2uwhQxwSOUZgAitTHGFZwyFx5P/5Zz8sT3D6+NIO10s8E7yygw0nNds UoHxX+caGxKpmjtXIbq+A7b2tlc/GgDZWFv0+FK1HaFwSWUUHSHbtnXz6ViMsw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TTJC02lkBzvXb; Mon, 5 Feb 2024 20:39:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 415KdOus070138; Mon, 5 Feb 2024 20:39:24 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 415KdO9w070135; Mon, 5 Feb 2024 20:39:24 GMT (envelope-from git) Date: Mon, 5 Feb 2024 20:39:24 GMT Message-Id: <202402052039.415KdO9w070135@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alexander Motin Subject: git: 825b7c222a6b - stable/14 - ntb_hw_plx: Workaround read-only scratchpad registers List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: mav X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 825b7c222a6b47ebf5d8974d02e3a7d9b59f5a0d Auto-Submitted: auto-generated The branch stable/14 has been updated by mav: URL: https://cgit.FreeBSD.org/src/commit/?id=825b7c222a6b47ebf5d8974d02e3a7d9b59f5a0d commit 825b7c222a6b47ebf5d8974d02e3a7d9b59f5a0d Author: Alexander Motin AuthorDate: 2024-01-27 21:51:46 +0000 Commit: Alexander Motin CommitDate: 2024-02-05 20:37:15 +0000 ntb_hw_plx: Workaround read-only scratchpad registers On several systems we've noticed that when NTB link goes down, the Physical Layer User Test Pattern registers we use as additional scratchpad registers (that is explicitly allowed by the chip specs) become read-only for about 100us. I see no explanation for this in the chip specs, neither why it was not seen before, may be a race. Since we do need these registers, workaround it by repeating writes until we succeed or 1ms timeout expire. MFC after: 1 week (cherry picked from commit 3883c6fbf232452098ba6ea802ef1426d83d2d68) --- sys/dev/ntb/ntb.h | 2 +- sys/dev/ntb/ntb_hw/ntb_hw_plx.c | 26 +++++++++++++++++++++----- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/sys/dev/ntb/ntb.h b/sys/dev/ntb/ntb.h index f9119fa0aedc..e2af625fff94 100644 --- a/sys/dev/ntb/ntb.h +++ b/sys/dev/ntb/ntb.h @@ -279,7 +279,7 @@ int ntb_mw_set_wc(device_t ntb, unsigned mw_idx, vm_memattr_t mode); uint8_t ntb_spad_count(device_t ntb); /* - * ntb_get_max_spads() - zero local scratch registers + * ntb_spad_clear() - zero local scratch registers * @ntb: pointer to ntb_softc instance * * This functions overwrites all local scratchpad registers with zeroes. diff --git a/sys/dev/ntb/ntb_hw/ntb_hw_plx.c b/sys/dev/ntb/ntb_hw/ntb_hw_plx.c index c0de13d7357d..5d89297203b2 100644 --- a/sys/dev/ntb/ntb_hw/ntb_hw_plx.c +++ b/sys/dev/ntb/ntb_hw/ntb_hw_plx.c @@ -880,17 +880,33 @@ static int ntb_plx_spad_write(device_t dev, unsigned int idx, uint32_t val) { struct ntb_plx_softc *sc = device_get_softc(dev); - u_int off; + u_int off, t; if (idx >= sc->spad_count1 + sc->spad_count2) return (EINVAL); - if (idx < sc->spad_count1) + if (idx < sc->spad_count1) { off = sc->spad_off1 + idx * 4; - else + bus_write_4(sc->conf_res, off, val); + return (0); + } else { off = sc->spad_off2 + (idx - sc->spad_count1) * 4; - bus_write_4(sc->conf_res, off, val); - return (0); + /* + * For some reason when link goes down Test Pattern registers + * we use as additional scratchpad become read-only for about + * 100us. I see no explanation in specs, so just wait a bit. + */ + for (t = 0; t <= 1000; t++) { + bus_write_4(sc->conf_res, off, val); + if (bus_read_4(sc->conf_res, off) == val) + return (0); + DELAY(1); + } + device_printf(dev, + "Can't write Physical Layer User Test Pattern (0x%x)\n", + off); + return (EIO); + } } static void