From nobody Sun Jul 19 18:50:48 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h3CQl04clz6lQpG for ; Sun, 19 Jul 2026 18:50:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h3CQj5WQ8z3R9v for ; Sun, 19 Jul 2026 18:50:53 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784487053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=EAVToxBZY0FkL6svw7SBCEsrq3PvGkDbM6OnWyOd034=; b=tmDEgqRUonChfkRTpmrPocXhIS/fWfNgoq4DwZmjnsOUL9nbKF+j4BnrEqwnhFgsVpG2L8 M86FS135hMJokAjMLmF/7z+sBcL1bs911wAnTwgnx1xMonIlCuiJQ8X9w03WbYKmbWrt0z kkGoDXPVOyEzSsCj21HTARnI740h694eEyEVCGzy9zhylUs5ACidhzYDO7BWn2FbykcebN y/CbAD/zfdi+SiTHrR9pyb+oFZzzuK0KSZuzAoZ+62RlGxtbjLvxWYRgI5MnUZ4AKjDq4/ nybpvrt2Ty8l/M+60Gdjld+B9EgZ4L1CC6CeqLul2R9TbcVc1aBdh4ujPE3V4g== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1784487053; a=rsa-sha256; cv=none; b=ag0FLWg2G7N1NCsop4znWC8hM5t0dFKNoofsQD3aGS/zZTfS8Ohpr/3XHtiiRkZwEHfuJD hCUd2wSGwLc3ENVD1Iqzs9Qwe6rXsSZGFf/Hzy7NZMvHWhA7LSkjnHTKimKUsrvn4AsUqu oT3GvJrmeKYNQ/D8WAgTXyeHwI++EASyCJWi5/qNTaREgD+aDiYTqo2HwAX+ajoBW7+ODj hQNq6ropnRBT4DvS1GsR2Y1Tn91tZmABHdVq/IWKzVGJoRkcHsLs3B+APvyHzjwU8EYMvD 4Ry9iYhVV0L7N1TGbVc0mhtp4A/Q6icccXbXkHG07s6Tf7UAquDdVXn1pqpjAg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784487053; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=EAVToxBZY0FkL6svw7SBCEsrq3PvGkDbM6OnWyOd034=; b=Kg1TSRAo19zwznMAMPT3Jusm85PtE+BrCzth8qZuS2rLYz384ps21L1HWbmhvKgTMrQrdR 7tXDjKzm/RVjF7sFvAF2CYpd6nhaNfVroUr9BMVUus4d9BWhGtrHlUAn8oSV/g0CKAu931 G5nOANtZS3zbbfE6lhIjKIPetqnOeNN61V4cuAFz5+uBc8wWUju+rcys4egRbHHwPGxHuX URI097U0w3W2oY96uRHDCsnpXuvfdr2fKvWx/51xP1L+bw63vY4JTG4tDaHQldQakbH5h7 XI0HUNB/Eqa2DOAUywrsOjB8KOTypErFTNeA4SIb+N9rlabyYR/GHzZKQe7F7w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h3CQj4SjdzC1Y for ; Sun, 19 Jul 2026 18:50:53 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3f941 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Sun, 19 Jul 2026 18:50:48 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org Cc: =?utf-8?Q?J=C3=A9r=C3=A9mie?= Jourdin From: Adrian Chadd Subject: git: 38187938f522 - main - re(4): quiesce RTL8168G+ and reset before freeing buffers in re_stop() List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: adrian X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 38187938f52283143308dd219db35d889da55ea4 Auto-Submitted: auto-generated Date: Sun, 19 Jul 2026 18:50:48 +0000 Message-Id: <6a5d1c88.3f941.abdc7ed@gitrepo.freebsd.org> The branch main has been updated by adrian: URL: https://cgit.FreeBSD.org/src/commit/?id=38187938f52283143308dd219db35d889da55ea4 commit 38187938f52283143308dd219db35d889da55ea4 Author: Jérémie Jourdin AuthorDate: 2026-07-19 18:39:49 +0000 Commit: Adrian Chadd CommitDate: 2026-07-19 18:44:36 +0000 re(4): quiesce RTL8168G+ and reset before freeing buffers in re_stop() The STOPREQ command written by re_stop() is not defined for RTL8168G and later; issuing it can wedge the MAC. Replace it on those parts with the vendor-documented sequence: * settle delay * bounded poll for Tx queue empty * clear TE/RE * then bounded poll of the MCU command register (0xD3) FIFO-empty bits. Also reset the controller before the Rx/Tx buffer free: a controller that has not quiesced keeps DMAing stale, still-owned descriptors pointing at freed mbufs (use-after-free under INVARIANTS, cross-NIC mbuf corruption reported in the PR). Adds the RL_MCU_* register definitions. All waits are bounded; error paths only. * iperf3 --bidir at line rate against RTL8168H (XID 0x541); previously wedged the controller until power cycle, with the quiesce the reset path recovers. * Deployed in production on an RTL8168H fleet since 2026-07-01. Reviewed by: adrian Differential Revision: https://reviews.freebsd.org/D58276 PR: kern/166724 --- sys/dev/re/if_re.c | 41 ++++++++++++++++++++++++++++++++++++++++- sys/dev/rl/if_rlreg.h | 8 ++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) diff --git a/sys/dev/re/if_re.c b/sys/dev/re/if_re.c index 50e7e1cad16c..cf7ba90c9a5a 100644 --- a/sys/dev/re/if_re.c +++ b/sys/dev/re/if_re.c @@ -3630,7 +3630,37 @@ re_stop(struct rl_softc *sc) 0x00080000); } - if ((sc->rl_flags & RL_FLAG_WAIT_TXPOLL) != 0) { + if ((sc->rl_flags & RL_FLAG_8168G_PLUS) != 0) { + /* + * RTL8168G and later. The STOPREQ command is defined only for + * earlier controllers; issuing it on these parts can leave the + * MAC wedged. With the RXDV gate enabled above, drain the TX + * descriptor queue and the on-chip TX/RX FIFOs and clear the + * TX/RX enable bits so the DMA engine is idle before the reset + * and buffer free below. All waits are bounded. + */ + DELAY(2000); + for (i = RL_TIMEOUT; i > 0; i--) { + if ((CSR_READ_4(sc, RL_TXCFG) & + RL_TXCFG_QUEUE_EMPTY) != 0) + break; + DELAY(100); + } + if (i == 0) + device_printf(sc->rl_dev, "stopping TXQ timed out!\n"); + CSR_WRITE_1(sc, RL_COMMAND, CSR_READ_1(sc, RL_COMMAND) & + ~(RL_CMD_TX_ENB | RL_CMD_RX_ENB)); + for (i = RL_TIMEOUT * 3; i > 0; i--) { + if ((CSR_READ_1(sc, RL_MCU_CMD) & + (RL_MCU_TXFIFO_EMPTY | RL_MCU_RXFIFO_EMPTY)) == + (RL_MCU_TXFIFO_EMPTY | RL_MCU_RXFIFO_EMPTY)) + break; + DELAY(20); + } + if (i == 0) + device_printf(sc->rl_dev, + "TX/RX FIFO drain timed out!\n"); + } else if ((sc->rl_flags & RL_FLAG_WAIT_TXPOLL) != 0) { for (i = RL_TIMEOUT; i > 0; i--) { if ((CSR_READ_1(sc, sc->rl_txstart) & RL_TXSTART_START) == 0) @@ -3661,6 +3691,15 @@ re_stop(struct rl_softc *sc) CSR_WRITE_2(sc, RL_IMR, 0x0000); CSR_WRITE_2(sc, RL_ISR, 0xFFFF); + /* + * Reset the controller before freeing the DMA buffers below. A + * controller that has not fully quiesced can keep fetching stale, + * still-owned descriptors that point at about-to-be-freed mbufs. + * re_init_locked() resets again on the reinit path; the extra reset + * is idempotent and cheap. + */ + re_reset(sc); + if (sc->rl_head != NULL) { m_freem(sc->rl_head); sc->rl_head = sc->rl_tail = NULL; diff --git a/sys/dev/rl/if_rlreg.h b/sys/dev/rl/if_rlreg.h index 10e25fcb5373..0db4b2066ce6 100644 --- a/sys/dev/rl/if_rlreg.h +++ b/sys/dev/rl/if_rlreg.h @@ -145,6 +145,14 @@ #define RL_INTRMOD 0x00E2 /* 16 bits */ #define RL_MISC 0x00F0 +/* + * MCU command / FIFO status register (RTL8168G and later). Polled to + * confirm the on-chip TX/RX FIFOs have drained before a controller reset. + */ +#define RL_MCU_CMD 0x00D3 +#define RL_MCU_TXFIFO_EMPTY 0x20 /* bit 5 */ +#define RL_MCU_RXFIFO_EMPTY 0x10 /* bit 4 */ + /* * TX config register bits */