From owner-dev-commits-src-all@freebsd.org Fri Oct 1 14:36:39 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3A79D6B0A10; Fri, 1 Oct 2021 14:36:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HLXjz1CQgz4WQB; Fri, 1 Oct 2021 14:36:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 09F4921268; Fri, 1 Oct 2021 14:36:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 191Eacwa017420; Fri, 1 Oct 2021 14:36:38 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 191Eackp017419; Fri, 1 Oct 2021 14:36:38 GMT (envelope-from git) Date: Fri, 1 Oct 2021 14:36:38 GMT Message-Id: <202110011436.191Eackp017419@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Randall Stewart Subject: git: a36230f75e0a - main - tcp: Make dsack stats available in netstat and also make sure its aware of TLP's. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: rrs X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: a36230f75e0ae5090e4ef47393536255b436cca6 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Oct 2021 14:36:39 -0000 The branch main has been updated by rrs: URL: https://cgit.FreeBSD.org/src/commit/?id=a36230f75e0ae5090e4ef47393536255b436cca6 commit a36230f75e0ae5090e4ef47393536255b436cca6 Author: Randall Stewart AuthorDate: 2021-10-01 14:32:30 +0000 Commit: Randall Stewart CommitDate: 2021-10-01 14:36:27 +0000 tcp: Make dsack stats available in netstat and also make sure its aware of TLP's. DSACK accounting has been for quite some time under a NETFLIX_STATS ifdef. Statistics on DSACKs however are very useful in figuring out how much bad retransmissions you are doing. This is further complicated, however, by stacks that do TLP. A TLP when discovering a lost ack in the reverse path will cause the generation of a DSACK. For this situation we introduce a new dsack-tlp-bytes as well as the more traditional dsack-bytes and dsack-packets. These will now all display in netstat -p tcp -s. This also updates all stacks that are currently built to keep track of these stats. Reviewed by: tuexen Sponsored by: Netflix Inc. Differential Revision: https://reviews.freebsd.org/D32158 --- sys/netinet/tcp_sack.c | 6 ++++++ sys/netinet/tcp_stacks/bbr.c | 4 +--- sys/netinet/tcp_stacks/rack.c | 14 +++++++++----- sys/netinet/tcp_subr.c | 27 +++++++++++++++++++++++++++ sys/netinet/tcp_var.h | 16 ++++++++++++++-- usr.bin/netstat/inet.c | 6 ++++++ 6 files changed, 63 insertions(+), 10 deletions(-) diff --git a/sys/netinet/tcp_sack.c b/sys/netinet/tcp_sack.c index 9753536926d5..25eb633fbbd4 100644 --- a/sys/netinet/tcp_sack.c +++ b/sys/netinet/tcp_sack.c @@ -604,6 +604,12 @@ tcp_sack_doack(struct tcpcb *tp, struct tcpopt *to, tcp_seq th_ack) SEQ_GT(sack.end, tp->snd_una) && SEQ_LEQ(sack.end, tp->snd_max)) { sack_blocks[num_sack_blks++] = sack; + } else if (SEQ_LEQ(sack.start, th_ack) && + SEQ_LEQ(sack.end, th_ack)) { + /* + * Its a D-SACK block. + */ + tcp_record_dsack(tp, sack.start, sack.end, 0); } } } diff --git a/sys/netinet/tcp_stacks/bbr.c b/sys/netinet/tcp_stacks/bbr.c index c96fec07b6c9..3c4cf0f54d97 100644 --- a/sys/netinet/tcp_stacks/bbr.c +++ b/sys/netinet/tcp_stacks/bbr.c @@ -7593,14 +7593,12 @@ proc_sack: } sack_blocks[num_sack_blks] = sack; num_sack_blks++; -#ifdef NETFLIX_STATS } else if (SEQ_LEQ(sack.start, th_ack) && SEQ_LEQ(sack.end, th_ack)) { /* * Its a D-SACK block. */ - tcp_record_dsack(sack.start, sack.end); -#endif + tcp_record_dsack(tp, sack.start, sack.end, 0); } } if (num_sack_blks == 0) diff --git a/sys/netinet/tcp_stacks/rack.c b/sys/netinet/tcp_stacks/rack.c index 2369a1c368bf..00f830caf217 100644 --- a/sys/netinet/tcp_stacks/rack.c +++ b/sys/netinet/tcp_stacks/rack.c @@ -9402,11 +9402,12 @@ rack_do_detection(struct tcpcb *tp, struct tcp_rack *rack, uint32_t bytes_this_ } #endif -static void +static int rack_note_dsack(struct tcp_rack *rack, tcp_seq start, tcp_seq end) { uint32_t am, l_end; + int was_tlp = 0; if (SEQ_GT(end, start)) am = end - start; @@ -9422,6 +9423,7 @@ rack_note_dsack(struct tcp_rack *rack, tcp_seq start, tcp_seq end) * our previous retransmit TLP. */ rack_log_dsack_event(rack, 7, __LINE__, start, end); + was_tlp = 1; goto skip_dsack_round; } if (rack->rc_last_sent_tlp_seq_valid) { @@ -9433,6 +9435,7 @@ rack_note_dsack(struct tcp_rack *rack, tcp_seq start, tcp_seq end) * for reordering purposes. */ rack_log_dsack_event(rack, 7, __LINE__, start, end); + was_tlp = 1; goto skip_dsack_round; } } @@ -9462,6 +9465,7 @@ skip_dsack_round: rack->r_ctl.retran_during_recovery = 0; rack->r_ctl.dsack_byte_cnt = 0; } + return (was_tlp); } static void @@ -9614,13 +9618,13 @@ rack_log_ack(struct tcpcb *tp, struct tcpopt *to, struct tcphdr *th, int entered num_sack_blks++; } else if (SEQ_LEQ(sack.start, th_ack) && SEQ_LEQ(sack.end, th_ack)) { -#ifdef NETFLIX_STATS + int was_tlp; + + was_tlp = rack_note_dsack(rack, sack.start, sack.end); /* * Its a D-SACK block. */ - tcp_record_dsack(sack.start, sack.end); -#endif - rack_note_dsack(rack, sack.start, sack.end); + tcp_record_dsack(tp, sack.start, sack.end, was_tlp); } } if (rack->rc_dsack_round_seen) { diff --git a/sys/netinet/tcp_subr.c b/sys/netinet/tcp_subr.c index 697ae7d3270b..9d66086a383b 100644 --- a/sys/netinet/tcp_subr.c +++ b/sys/netinet/tcp_subr.c @@ -390,6 +390,30 @@ static int tcp_fb_cnt = 0; struct tcp_funchead t_functions; static struct tcp_function_block *tcp_func_set_ptr = &tcp_def_funcblk; +void +tcp_record_dsack(struct tcpcb *tp, tcp_seq start, tcp_seq end, int tlp) +{ + TCPSTAT_INC(tcps_dsack_count); + tp->t_dsack_pack++; + if (tlp == 0) { + if (SEQ_GT(end, start)) { + tp->t_dsack_bytes += (end - start); + TCPSTAT_ADD(tcps_dsack_bytes, (end - start)); + } else { + tp->t_dsack_tlp_bytes += (start - end); + TCPSTAT_ADD(tcps_dsack_bytes, (start - end)); + } + } else { + if (SEQ_GT(end, start)) { + tp->t_dsack_bytes += (end - start); + TCPSTAT_ADD(tcps_dsack_tlp_bytes, (end - start)); + } else { + tp->t_dsack_tlp_bytes += (start - end); + TCPSTAT_ADD(tcps_dsack_tlp_bytes, (start - end)); + } + } +} + static struct tcp_function_block * find_tcp_functions_locked(struct tcp_function_set *fs) { @@ -4003,6 +4027,9 @@ tcp_inptoxtp(const struct inpcb *inp, struct xtcpcb *xt) xt->t_snd_wnd = tp->snd_wnd; xt->t_snd_cwnd = tp->snd_cwnd; xt->t_snd_ssthresh = tp->snd_ssthresh; + xt->t_dsack_bytes = tp->t_dsack_bytes; + xt->t_dsack_tlp_bytes = tp->t_dsack_tlp_bytes; + xt->t_dsack_pack = tp->t_dsack_pack; xt->t_maxseg = tp->t_maxseg; xt->xt_ecn = (tp->t_flags2 & TF2_ECN_PERMIT) ? 1 : 0 + (tp->t_flags2 & TF2_ACE_PERMIT) ? 2 : 0; diff --git a/sys/netinet/tcp_var.h b/sys/netinet/tcp_var.h index c26f503f4a1d..d5b2963ef4dc 100644 --- a/sys/netinet/tcp_var.h +++ b/sys/netinet/tcp_var.h @@ -265,6 +265,9 @@ struct tcpcb { uint64_t t_sndtlpbyte; /* total tail loss probe bytes sent */ uint64_t t_sndbytes; /* total bytes sent */ uint64_t t_snd_rxt_bytes; /* total bytes retransmitted */ + uint32_t t_dsack_bytes; /* Total number of dsack bytes we have received */ + uint32_t t_dsack_tlp_bytes; /* Total number of dsack bytes we have received for TLPs sent */ + uint32_t t_dsack_pack; /* Total dsack packets we have recieved */ uint8_t t_tfo_client_cookie_len; /* TCP Fast Open client cookie length */ uint32_t t_end_info_status; /* Status flag of end info */ @@ -703,7 +706,12 @@ struct tcpstat { uint64_t tcps_tunneled_pkts; /* Packets encap's in UDP received */ uint64_t tcps_tunneled_errs; /* Packets that had errors that were UDP encaped */ - uint64_t _pad[9]; /* 6 UTO, 3 TBD */ + /* Dsack related stats */ + uint64_t tcps_dsack_count; /* Number of ACKs arriving with DSACKs */ + uint64_t tcps_dsack_bytes; /* Number of bytes DSACK'ed no TLP */ + uint64_t tcps_dsack_tlp_bytes; /* Number of bytes DSACK'ed due to TLPs */ + + uint64_t _pad[6]; /* 3 UTO, 3 TBD */ }; #define tcps_rcvmemdrop tcps_rcvreassfull /* compat */ @@ -801,9 +809,12 @@ struct xtcpcb { uint32_t t_rcv_wnd; /* (s) */ uint32_t t_snd_wnd; /* (s) */ uint32_t xt_ecn; /* (s) */ + uint32_t t_dsack_bytes; /* (n) */ + uint32_t t_dsack_tlp_bytes; /* (n) */ + uint32_t t_dsack_pack; /* (n) */ uint16_t xt_encaps_port; /* (s) */ int16_t spare16; - int32_t spare32[25]; + int32_t spare32[22]; } __aligned(8); #ifdef _KERNEL @@ -1064,6 +1075,7 @@ int tcp_twcheck(struct inpcb *, struct tcpopt *, struct tcphdr *, struct mbuf *, int); void tcp_setpersist(struct tcpcb *); void tcp_slowtimo(void); +void tcp_record_dsack(struct tcpcb *tp, tcp_seq start, tcp_seq end, int tlp); struct tcptemp * tcpip_maketemplate(struct inpcb *); void tcpip_fillheaders(struct inpcb *, uint16_t, void *, void *); diff --git a/usr.bin/netstat/inet.c b/usr.bin/netstat/inet.c index 45c5f05e60b8..2d3a4bb10d52 100644 --- a/usr.bin/netstat/inet.c +++ b/usr.bin/netstat/inet.c @@ -695,6 +695,12 @@ tcp_stats(u_long off, const char *name, int af1 __unused, int proto __unused) "{N:/window probe%s}\n"); p(tcps_rcvwinupd, "\t\t{:receive-window-update-packets/%ju} " "{N:/window update packet%s}\n"); + p(tcps_dsack_count, "\t\t{:received-with-dsack-packets/%ju} " + "{N:/packet%s received with dsack}\n"); + p(tcps_dsack_bytes, "\t\t{:received-with-dsack-bytes/%ju} " + "{N:/dsack byte%s received (no TLP involved)}\n"); + p(tcps_dsack_tlp_bytes, "\t\t{:received-with-dsack-bytes-tlp/%ju} " + "{N:/dsack byte%s received (TLP responsible)}\n"); p(tcps_rcvafterclose, "\t\t{:received-after-close-packets/%ju} " "{N:/packet%s received after close}\n"); p(tcps_rcvbadsum, "\t\t{:discard-bad-checksum/%ju} "