From owner-svn-src-head@freebsd.org Mon Nov 6 16:23:22 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5E1FFE622E5; Mon, 6 Nov 2017 16:23:22 +0000 (UTC) (envelope-from shurd@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 37FAB75CC0; Mon, 6 Nov 2017 16:23:22 +0000 (UTC) (envelope-from shurd@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id vA6GNLpi062640; Mon, 6 Nov 2017 16:23:21 GMT (envelope-from shurd@FreeBSD.org) Received: (from shurd@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id vA6GNLUB062639; Mon, 6 Nov 2017 16:23:21 GMT (envelope-from shurd@FreeBSD.org) Message-Id: <201711061623.vA6GNLUB062639@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: shurd set sender to shurd@FreeBSD.org using -f From: Stephen Hurd Date: Mon, 6 Nov 2017 16:23:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r325487 - head/sys/net X-SVN-Group: head X-SVN-Commit-Author: shurd X-SVN-Commit-Paths: head/sys/net X-SVN-Commit-Revision: 325487 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 06 Nov 2017 16:23:22 -0000 Author: shurd Date: Mon Nov 6 16:23:21 2017 New Revision: 325487 URL: https://svnweb.freebsd.org/changeset/base/325487 Log: Only chain non-LRO mbufs when LRO is not possible Preserve packet order between tcp_lro_rx() and if_input() to avoid creating extra corner cases. If no packets can be LROed, combine them into one chain for submission via if_input(). If any packet can potentially be LROed however, retain old behaviour and call if_input() for each packet. This should keep the 12% improvement for small packet forwarding intact, but mostly avoids impacting the LRO case. Reviewed by: cem, sbruno Approved by: sbruno (mentor) Sponsored by: Limelight Networks Differential Revision: https://reviews.freebsd.org/D12876 Modified: head/sys/net/iflib.c Modified: head/sys/net/iflib.c ============================================================================== --- head/sys/net/iflib.c Mon Nov 6 15:29:33 2017 (r325486) +++ head/sys/net/iflib.c Mon Nov 6 16:23:21 2017 (r325487) @@ -59,6 +59,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -68,6 +69,8 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include +#include #include #include @@ -2463,7 +2466,48 @@ iflib_rxd_pkt_get(iflib_rxq_t rxq, if_rxd_info_t ri) return (m); } +#if defined(INET6) || defined(INET) +/* + * Returns true if it's possible this packet could be LROed. + * if it returns false, it is guaranteed that tcp_lro_rx() + * would not return zero. + */ static bool +iflib_check_lro_possible(struct lro_ctrl *lc, struct mbuf *m) +{ + struct ether_header *eh; + uint16_t eh_type; + + eh = mtod(m, struct ether_header *); + eh_type = ntohs(eh->ether_type); + switch (eh_type) { + case ETHERTYPE_IPV6: + { + CURVNET_SET(lc->ifp->if_vnet); + if (VNET(ip6_forwarding) == 0) { + CURVNET_RESTORE(); + return true; + } + CURVNET_RESTORE(); + break; + } + case ETHERTYPE_IP: + { + CURVNET_SET(lc->ifp->if_vnet); + if (VNET(ipforwarding) == 0) { + CURVNET_RESTORE(); + return true; + } + CURVNET_RESTORE(); + break; + } + } + + return false; +} +#endif + +static bool iflib_rxeof(iflib_rxq_t rxq, qidx_t budget) { if_ctx_t ctx = rxq->ifr_ctx; @@ -2476,6 +2520,7 @@ iflib_rxeof(iflib_rxq_t rxq, qidx_t budget) iflib_fl_t fl; struct ifnet *ifp; int lro_enabled; + bool lro_possible = false; /* * XXX early demux data packets so that if_input processing only handles @@ -2555,8 +2600,6 @@ iflib_rxeof(iflib_rxq_t rxq, qidx_t budget) mt = mf = NULL; while (mh != NULL) { m = mh; - if (mf == NULL) - mf = m; mh = mh->m_nextpkt; m->m_nextpkt = NULL; #ifndef __NO_STRICT_ALIGNMENT @@ -2566,12 +2609,27 @@ iflib_rxeof(iflib_rxq_t rxq, qidx_t budget) rx_bytes += m->m_pkthdr.len; rx_pkts++; #if defined(INET6) || defined(INET) - if (lro_enabled && tcp_lro_rx(&rxq->ifr_lc, m, 0) == 0) { - if (mf == m) - mf = NULL; - continue; + if (lro_enabled) { + if (!lro_possible) { + lro_possible = iflib_check_lro_possible(&rxq->ifr_lc, m); + if (lro_possible && mf != NULL) { + ifp->if_input(ifp, mf); + DBG_COUNTER_INC(rx_if_input); + mt = mf = NULL; + } + } + if (lro_possible && tcp_lro_rx(&rxq->ifr_lc, m, 0) == 0) + continue; } #endif + if (lro_possible) { + ifp->if_input(ifp, m); + DBG_COUNTER_INC(rx_if_input); + continue; + } + + if (mf == NULL) + mf = m; if (mt != NULL) mt->m_nextpkt = m; mt = m;