From nobody Tue Apr 14 18:50:36 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4fwCyj3QF6z6ZGfH for ; Tue, 14 Apr 2026 18:50:37 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4fwCyj1dFdz3qTY for ; Tue, 14 Apr 2026 18:50:37 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1776192637; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Gs/7TQWjCVTeKY9qb/dL4uQ2TXL3uMB/HpzykuFxAhc=; b=jJQauRF/HYxKJsH+R/Zv3gm0ELnFLlm0v7ZTcc/V8o4J2WlVqUW2dHexR4ryxlaT0GCZZI kke9mbYJhlrn6PZqCF7mhwb7Y/9VORLg/wHlK2ZoTSAu+0TqdKyFJkU/dCqJc4Hwf/6Fmx QzEbZSpmjdlPKOIkitn+nLdMp4rZoFjat38SQiy9PXnHGyMed1mrWEgmWjXk57oWKdElBH B4v97DQqvsbgilavwsfK5bKNV/+bLR2s29yZLlMtmoHvohPquUIgPPlYg/+Own7AHdjUA3 CecOga0FZDXVn+96BY+UNjAgaOnscEsmoOJf+f46Nd0y0SNXrAfgnS52jj66Eg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1776192637; a=rsa-sha256; cv=none; b=Aw+vXCDomPDDYuLEiEU/w0OraSyp15YgD7uGUFdH3VcnxESL7XxJg/xCzUMBANO+iDeQu0 /VIENfvYSKPE68a0IMyQvezfTqLhzMU/ebU6ld0CcnIPNGsGBn0TWP/d45LvwHgTsubAI1 sDlHTfLW4mmRB+6RZeTZnyVlJ4aXeiXa4wSq986aAl78gBa46qbY1RTJRyJqSKX5d7+33m qBXTq0nRvGlEENt3+xm2Jpuz0cI6ynUgsQ6hpH6g83AcD3x3kF9FQ0SgpemIKv8CxRfOZ1 n+OqDlEf5qb1Gu8+lmiR6LThGjrvl6MJR7HTt7iEBnRxP2bitA7mZE8MN6nCng== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1776192637; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Gs/7TQWjCVTeKY9qb/dL4uQ2TXL3uMB/HpzykuFxAhc=; b=vMV4U0H7UVIyVJgazZuvwdRyWt989kdIoNJos0NwqU7+c7uhcEmPT/8y8ea3TkGP+Z5yRa XwAYPeOFYTpKs3eJjIlx//rOhNRnQmJ8GC4oAffbNUCaLT9d7CeT/9CLDDmHmm3wk8Yz66 O2pNlJFYieqFJne2cCnUeGngr9SxskMcQ7g2xHbwWcL5kJSPM5NaDAymsuB8LPbl7RZ5Mt 9YnjNQjjTZ/+EcbrVdCcSzLn0Ij2eRAeJ981XsZxllYiKpBAL1ny7NsCxB5Xq/n/hdEooh J32PLsuqYyxS3MzKgw6AZy189Jx8CNZpTVLqicKzwvGNfpqqDPquSx6ukhXSLQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4fwCyh6Bkfz12b5 for ; Tue, 14 Apr 2026 18:50:36 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3d836 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Tue, 14 Apr 2026 18:50:36 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Gallatin Subject: git: f707cc00ed12 - main - lro: move pkt rejection checks to leafs to avoid queueing non-LRO'able pkts List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: gallatin X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: f707cc00ed124828e367b020d6b417842321f641 Auto-Submitted: auto-generated Date: Tue, 14 Apr 2026 18:50:36 +0000 Message-Id: <69de8c7c.3d836.56c79b47@gitrepo.freebsd.org> The branch main has been updated by gallatin: URL: https://cgit.FreeBSD.org/src/commit/?id=f707cc00ed124828e367b020d6b417842321f641 commit f707cc00ed124828e367b020d6b417842321f641 Author: Andrew Gallatin AuthorDate: 2026-04-13 21:33:17 +0000 Commit: Andrew Gallatin CommitDate: 2026-04-14 18:47:57 +0000 lro: move pkt rejection checks to leafs to avoid queueing non-LRO'able pkts When lro mbuf queuing is enabled, we should not queue easily reject-able packets. Queuing them does a bit of extra work (sorting, timestamps) and can potentially delay urgent packets such as LACP PDUs. This change moves simple rejection tests from lro_rx_common() into lro_rx and (more importantly) into tcp_lro_queue_mbuf(). Note this change only moves the easy checks on forwarding and packet metadata, where the rejection criteria is already hot in cache. It does not move parsing and looking inside the packet to verify the ether protocol, ip protocol, etc. This could be done, but we risk essentially doubling the cache misses per-packet by doing so. Differential Revision: https://reviews.freebsd.org/D56337 Reviewed by: rrs, tuexen Sponsored by: Netflix --- sys/netinet/tcp_lro.c | 73 ++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 52 insertions(+), 21 deletions(-) diff --git a/sys/netinet/tcp_lro.c b/sys/netinet/tcp_lro.c index 06280bce2279..5819fb5ceae0 100644 --- a/sys/netinet/tcp_lro.c +++ b/sys/netinet/tcp_lro.c @@ -1305,27 +1305,6 @@ tcp_lro_rx_common(struct lro_ctrl *lc, struct mbuf *m, uint32_t csum, bool use_h int error; uint16_t tcp_data_sum; -#ifdef INET - /* Quickly decide if packet cannot be LRO'ed */ - if (__predict_false(V_ipforwarding != 0)) - return (TCP_LRO_CANNOT); -#endif -#ifdef INET6 - /* Quickly decide if packet cannot be LRO'ed */ - if (__predict_false(V_ip6_forwarding != 0)) - return (TCP_LRO_CANNOT); -#endif - if (((m->m_pkthdr.csum_flags & (CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) != - ((CSUM_DATA_VALID | CSUM_PSEUDO_HDR))) || - (m->m_pkthdr.csum_data != 0xffff)) { - /* - * The checksum either did not have hardware offload - * or it was a bad checksum. We can't LRO such - * a packet. - */ - counter_u64_add(tcp_bad_csums, 1); - return (TCP_LRO_CANNOT); - } /* We expect a contiguous header [eh, ip, tcp]. */ pa = tcp_lro_parser(m, &po, &pi, true); if (__predict_false(pa == NULL)) @@ -1443,6 +1422,29 @@ tcp_lro_rx(struct lro_ctrl *lc, struct mbuf *m, uint32_t csum) { int error; +#ifdef INET + /* Quickly decide if packet cannot be LRO'ed */ + if (__predict_false(V_ipforwarding != 0)) + return (TCP_LRO_CANNOT); +#endif +#ifdef INET6 + /* Quickly decide if packet cannot be LRO'ed */ + if (__predict_false(V_ip6_forwarding != 0)) + return (TCP_LRO_CANNOT); +#endif + + if (((m->m_pkthdr.csum_flags & (CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) != + ((CSUM_DATA_VALID | CSUM_PSEUDO_HDR))) || + (m->m_pkthdr.csum_data != 0xffff)) { + /* + * The checksum either did not have hardware offload + * or it was a bad checksum. We can't LRO such + * a packet. + */ + counter_u64_add(tcp_bad_csums, 1); + return (TCP_LRO_CANNOT); + } + /* get current time */ binuptime(&lc->lro_last_queue_time); CURVNET_SET(lc->ifp->if_vnet); @@ -1472,6 +1474,22 @@ tcp_lro_queue_mbuf(struct lro_ctrl *lc, struct mbuf *mb) return; } +#ifdef INET + /* Quickly decide if packet cannot be LRO'ed */ + if (__predict_false(V_ipforwarding != 0)) { + /* input packet to network layer */ + (*lc->ifp->if_input) (lc->ifp, mb); + return; + } +#endif +#ifdef INET6 + /* Quickly decide if packet cannot be LRO'ed */ + if (__predict_false(V_ip6_forwarding != 0)) { + /* input packet to network layer */ + (*lc->ifp->if_input) (lc->ifp, mb); + return; + } +#endif /* check if packet is not LRO capable */ if (__predict_false((lc->ifp->if_capenable & IFCAP_LRO) == 0)) { /* input packet to network layer */ @@ -1479,6 +1497,19 @@ tcp_lro_queue_mbuf(struct lro_ctrl *lc, struct mbuf *mb) return; } + if (((mb->m_pkthdr.csum_flags & (CSUM_DATA_VALID | CSUM_PSEUDO_HDR)) != + ((CSUM_DATA_VALID | CSUM_PSEUDO_HDR))) || + (mb->m_pkthdr.csum_data != 0xffff)) { + /* + * The checksum either did not have hardware offload + * or it was a bad checksum. We can't LRO such + * a packet. + */ + counter_u64_add(tcp_bad_csums, 1); + (*lc->ifp->if_input) (lc->ifp, mb); + return; + } + /* If no hardware or arrival stamp on the packet add timestamp */ if ((tcplro_stacks_wanting_mbufq > 0) && (tcp_less_accurate_lro_ts == 0) &&