From owner-svn-src-head@freebsd.org Thu Jun 21 21:03:59 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 08146100043E; Thu, 21 Jun 2018 21:03:59 +0000 (UTC) (envelope-from rrs@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id B1ED88E26A; Thu, 21 Jun 2018 21:03:58 +0000 (UTC) (envelope-from rrs@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 93120172A1; Thu, 21 Jun 2018 21:03:58 +0000 (UTC) (envelope-from rrs@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w5LL3wM4051762; Thu, 21 Jun 2018 21:03:58 GMT (envelope-from rrs@FreeBSD.org) Received: (from rrs@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w5LL3wlo051761; Thu, 21 Jun 2018 21:03:58 GMT (envelope-from rrs@FreeBSD.org) Message-Id: <201806212103.w5LL3wlo051761@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: rrs set sender to rrs@FreeBSD.org using -f From: Randall Stewart Date: Thu, 21 Jun 2018 21:03:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r335502 - head/sys/netinet X-SVN-Group: head X-SVN-Commit-Author: rrs X-SVN-Commit-Paths: head/sys/netinet X-SVN-Commit-Revision: 335502 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 Jun 2018 21:03:59 -0000 Author: rrs Date: Thu Jun 21 21:03:58 2018 New Revision: 335502 URL: https://svnweb.freebsd.org/changeset/base/335502 Log: This adds in an optimization so that we only walk one time through the mbuf chain during copy and TSO limiting. It is used by both Rack and now the FreeBSD stack. Sponsored by: Netflix Inc Differential Revision: https://reviews.freebsd.org/D15937 Modified: head/sys/netinet/tcp_output.c Modified: head/sys/netinet/tcp_output.c ============================================================================== --- head/sys/netinet/tcp_output.c Thu Jun 21 20:18:23 2018 (r335501) +++ head/sys/netinet/tcp_output.c Thu Jun 21 21:03:58 2018 (r335502) @@ -209,6 +209,8 @@ tcp_output(struct tcpcb *tp) int32_t len; uint32_t recwin, sendwin; int off, flags, error = 0; /* Keep compiler happy */ + u_int if_hw_tsomaxsegcount = 0; + u_int if_hw_tsomaxsegsize; struct mbuf *m; struct ip *ip = NULL; #ifdef TCPDEBUG @@ -879,9 +881,6 @@ send: if (tso) { u_int if_hw_tsomax; - u_int if_hw_tsomaxsegcount; - u_int if_hw_tsomaxsegsize; - struct mbuf *mb; u_int moff; int max_len; @@ -913,66 +912,7 @@ send: len = max_len; } } - /* - * Check if we should limit by maximum segment - * size and count: - */ - if (if_hw_tsomaxsegcount != 0 && - if_hw_tsomaxsegsize != 0) { - /* - * Subtract one segment for the LINK - * and TCP/IP headers mbuf that will - * be prepended to this mbuf chain - * after the code in this section - * limits the number of mbufs in the - * chain to if_hw_tsomaxsegcount. - */ - if_hw_tsomaxsegcount -= 1; - max_len = 0; - mb = sbsndmbuf(&so->so_snd, off, &moff); - - while (mb != NULL && max_len < len) { - u_int mlen; - u_int frags; - - /* - * Get length of mbuf fragment - * and how many hardware frags, - * rounded up, it would use: - */ - mlen = (mb->m_len - moff); - frags = howmany(mlen, - if_hw_tsomaxsegsize); - - /* Handle special case: Zero Length Mbuf */ - if (frags == 0) - frags = 1; - - /* - * Check if the fragment limit - * will be reached or exceeded: - */ - if (frags >= if_hw_tsomaxsegcount) { - max_len += min(mlen, - if_hw_tsomaxsegcount * - if_hw_tsomaxsegsize); - break; - } - max_len += mlen; - if_hw_tsomaxsegcount -= frags; - moff = 0; - mb = mb->m_next; - } - if (max_len <= 0) { - len = 0; - } else if (len > max_len) { - sendalot = 1; - len = max_len; - } - } - - /* * Prevent the last segment from being * fractional unless the send sockbuf can be * emptied: @@ -1006,7 +946,6 @@ send: */ if (tp->t_flags & TF_NEEDFIN) sendalot = 1; - } else { len = tp->t_maxseg - optlen - ipoptlen; sendalot = 1; @@ -1041,6 +980,7 @@ send: */ if (len) { struct mbuf *mb; + struct sockbuf *msb; u_int moff; if ((tp->t_flags & TF_FORCEDATA) && len == 1) @@ -1074,14 +1014,30 @@ send: * Start the m_copy functions from the closest mbuf * to the offset in the socket buffer chain. */ - mb = sbsndptr(&so->so_snd, off, len, &moff); - + mb = sbsndptr_noadv(&so->so_snd, off, &moff); if (len <= MHLEN - hdrlen - max_linkhdr) { m_copydata(mb, moff, len, mtod(m, caddr_t) + hdrlen); + if (SEQ_LT(tp->snd_nxt, tp->snd_max)) + sbsndptr_adv(&so->so_snd, mb, len); m->m_len += len; } else { - m->m_next = m_copym(mb, moff, len, M_NOWAIT); + if (SEQ_LT(tp->snd_nxt, tp->snd_max)) + msb = NULL; + else + msb = &so->so_snd; + m->m_next = tcp_m_copym(mb, moff, + &len, if_hw_tsomaxsegcount, + if_hw_tsomaxsegsize, msb); + if (len <= (tp->t_maxseg - optlen)) { + /* + * Must have ran out of mbufs for the copy + * shorten it to no longer need tso. Lets + * not put on sendalot since we are low on + * mbufs. + */ + tso = 0; + } if (m->m_next == NULL) { SOCKBUF_UNLOCK(&so->so_snd); (void) m_free(m);