From nobody Mon Apr 10 20:34:05 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4PwLKn6vPnz44lwc; Mon, 10 Apr 2023 20:34:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4PwLKn5fglz41Dl; Mon, 10 Apr 2023 20:34:05 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681158845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZUr2+F3jCW4Vq4oiIIwZXbU8PtdROVIzyG1uzowJhRM=; b=JRCw/N/Ic43f30ydrBa2T0vCyc0DtzEUtrdVQLzmu1ADV+Ox1QTllcxADi9OKAUK5Z5vKG leDyBq3rhZdmm3WcQQGUbU6ahA1Ex7taCDWfRoDI81Zl5ORWbGEyt4J25ynzcC/xugs43C YvA28AG1OJZZwIQRUhBZZ7bR2nasdYCjmxhelKeI46KptNgxRV6kVK73L+DU1bJ0dHIOeA kYyrdYf21klFEsmrNYLh8ZzpIP75bWhMQTr8cb0trohl8KC7C3+RzVOC1einV0E/tvZlML IBhrJjF+CjHBXMq9Kg99zJhU+fkMmJAZcNfDvraCJKqg3JIqjl9jXbX/RkQ95Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1681158845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ZUr2+F3jCW4Vq4oiIIwZXbU8PtdROVIzyG1uzowJhRM=; b=Fl0Tuy9idzA0b8mPLLir6tlJc0YS63RFP+2ATBrjWpHV6X12HPOu1W0JfuxQGmmPo8Yjd0 l76yL8N5zEROgWLE6WljNwLL+hn1IMtakupewIZzAq3aY+ZvX46GIPlxJIwUUdoXUNfDKz I9j5MKPk0pI3j1lNOvINSS3psryh2bu0MUOVzxWBKWwYcy77MxEC2VHGjV9jdf4o/TzPdk EilFm4ZIIvSrRRMqYriFRjXgTxcOz/snvby3/eONoGfdv+bIoTOqaNGcLU2JmMkB0ss0my J6qF53ubGi28eScOO6jFIhNxC3l2de5nRA2HE23ynhteE2YticfzU069InAKcQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1681158845; a=rsa-sha256; cv=none; b=lvthXFO/hye5JjdGossJMMNYEj8oZDnXU/P5VPeS0mBiMshJnWU3gw6jEc/KJNqWicIbwf pBnFqc4OKGFT9qLYOR1r3mDvIqaNQptQs3j6q9Qi4jNMwFdTU5Chl6+dJavGlHIoBIGi7I 7XAn8yipxaRH+tlvZMpHoFL27mBxeGbo1KbfknBNKmSoEYjNRUnvNVOons1eA0dc+TKo0F 31Q6HpjKlU5EBmnK7jQRj4OkMTjxbbtwoKYp3NWQDynDD76qBmsRLdcQjcpvdynTcgtffx tKgFD2sVUNV4S3ZD7sTGV5Y8Xn4SqfwB5aubqe5lbMT6XOED1o8Ej6o6Ft1E2Q== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4PwLKn4byFz14tk; Mon, 10 Apr 2023 20:34:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 33AKY5Re054143; Mon, 10 Apr 2023 20:34:05 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 33AKY513054142; Mon, 10 Apr 2023 20:34:05 GMT (envelope-from git) Date: Mon, 10 Apr 2023 20:34:05 GMT Message-Id: <202304102034.33AKY513054142@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Randall Stewart Subject: git: a2b33c9a7a95 - main - tcp: Rack - in the absence of LRO fixed rate pacing (loopback or interfaces with no LRO) does not work correctly. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rrs X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a2b33c9a7a952ff35d5061ae1dc76980d40320d3 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by rrs: URL: https://cgit.FreeBSD.org/src/commit/?id=a2b33c9a7a952ff35d5061ae1dc76980d40320d3 commit a2b33c9a7a952ff35d5061ae1dc76980d40320d3 Author: Randall Stewart AuthorDate: 2023-04-10 20:33:56 +0000 Commit: Randall Stewart CommitDate: 2023-04-10 20:33:56 +0000 tcp: Rack - in the absence of LRO fixed rate pacing (loopback or interfaces with no LRO) does not work correctly. Rack is capable of fixed rate or dynamic rate pacing. Both of these can get mixed up when LRO is not available. This is because LRO will hold off waking up the tcp connection to processing the inbound packets until the pacing timer is up. Without LRO the pacing only sort-of works. Sometimes we pace correctly, other times not so much. This set of changes will make it so pacing works properly in the absence of LRO. Reviewed by: tuexen Sponsored by: Netflix Inc Differential Revision:https://reviews.freebsd.org/D39494 --- sys/netinet/tcp_stacks/rack.c | 54 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/sys/netinet/tcp_stacks/rack.c b/sys/netinet/tcp_stacks/rack.c index a1dda9889549..1bab253818b5 100644 --- a/sys/netinet/tcp_stacks/rack.c +++ b/sys/netinet/tcp_stacks/rack.c @@ -16433,6 +16433,9 @@ rack_do_compressed_ack_processing(struct tcpcb *tp, struct socket *so, struct mb return (0); } +#define TCP_LRO_TS_OPTION \ + ntohl((TCPOPT_NOP << 24) | (TCPOPT_NOP << 16) | \ + (TCPOPT_TIMESTAMP << 8) | TCPOLEN_TIMESTAMP) static int rack_do_segment_nounlock(struct tcpcb *tp, struct mbuf *m, struct tcphdr *th, @@ -16458,6 +16461,8 @@ rack_do_segment_nounlock(struct tcpcb *tp, struct mbuf *m, struct tcphdr *th, struct tcp_rack *rack; struct rack_sendmap *rsm; int32_t prev_state = 0; + int no_output = 0; + int slot_remaining = 0; #ifdef TCP_ACCOUNTING int ack_val_set = 0xf; #endif @@ -16480,6 +16485,43 @@ rack_do_segment_nounlock(struct tcpcb *tp, struct mbuf *m, struct tcphdr *th, */ rack_deferred_init(tp, rack); } + /* + * Check to see if we need to skip any output plans. This + * can happen in the non-LRO path where we are pacing and + * must process the ack coming in but need to defer sending + * anything becase a pacing timer is running. + */ + us_cts = tcp_tv_to_usectick(tv); + if ((rack->rc_always_pace == 1) && + (rack->r_ctl.rc_hpts_flags & PACE_PKT_OUTPUT) && + (TSTMP_LT(us_cts, rack->r_ctl.rc_last_output_to))) { + /* + * Ok conditions are right for queuing the packets + * but we do have to check the flags in the inp, it + * could be, if a sack is present, we want to be awoken and + * so should process the packets. + */ + slot_remaining = rack->r_ctl.rc_last_output_to - us_cts; + if (rack->rc_inp->inp_flags2 & INP_DONT_SACK_QUEUE) { + no_output = 1; + } else { + /* + * If there is no options, or just a + * timestamp option, we will want to queue + * the packets. This is the same that LRO does + * and will need to change with accurate ECN. + */ + uint32_t *ts_ptr; + int optlen; + + optlen = (th->th_off << 2) - sizeof(struct tcphdr); + ts_ptr = (uint32_t *)(th + 1); + if ((optlen == 0) || + ((optlen == TCPOLEN_TSTAMP_APPA) && + (*ts_ptr == TCP_LRO_TS_OPTION))) + no_output = 1; + } + } if (m->m_flags & M_ACKCMP) { /* * All compressed ack's are ack's by definition so @@ -16914,7 +16956,7 @@ rack_do_segment_nounlock(struct tcpcb *tp, struct mbuf *m, struct tcphdr *th, } } #endif - if (nxt_pkt == 0) { + if ((nxt_pkt == 0) && (no_output == 0)) { if ((rack->r_wanted_output != 0) || (rack->r_fast_output != 0)) { do_output_now: if (tcp_output(tp) < 0) { @@ -16927,6 +16969,16 @@ do_output_now: } rack_start_hpts_timer(rack, tp, cts, 0, 0, 0); rack_free_trim(rack); + } else if ((no_output == 1) && + (nxt_pkt == 0) && + (tcp_in_hpts(rack->rc_inp) == 0)) { + /* + * We are not in hpts and we had a pacing timer up. Use + * the remaining time (slot_remaining) to restart the timer. + */ + KASSERT ((slot_remaining != 0), ("slot remaining is zero for rack:%p tp:%p", rack, tp)); + rack_start_hpts_timer(rack, tp, cts, slot_remaining, 0, 0); + rack_free_trim(rack); } /* Update any rounds needed */ if (rack_verbose_logging && tcp_bblogging_on(rack->rc_tp))