From owner-freebsd-transport@freebsd.org Wed Oct 28 23:27:38 2015 Return-Path: Delivered-To: freebsd-transport@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 69CFFA20A5B for ; Wed, 28 Oct 2015 23:27:38 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 4E827198E for ; Wed, 28 Oct 2015 23:27:38 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: by mailman.ysv.freebsd.org (Postfix) id 4B3B7A20A5A; Wed, 28 Oct 2015 23:27:38 +0000 (UTC) Delivered-To: transport@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 310F5A20A59 for ; Wed, 28 Oct 2015 23:27:38 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from mail.strugglingcoder.info (strugglingcoder.info [65.19.130.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.strugglingcoder.info", Issuer "mail.strugglingcoder.info" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 14C23198D for ; Wed, 28 Oct 2015 23:27:37 +0000 (UTC) (envelope-from hiren@strugglingcoder.info) Received: from localhost (unknown [10.1.1.3]) (Authenticated sender: hiren@strugglingcoder.info) by mail.strugglingcoder.info (Postfix) with ESMTPA id 5785F10B120 for ; Wed, 28 Oct 2015 16:27:37 -0700 (PDT) Date: Wed, 28 Oct 2015 16:27:37 -0700 From: hiren panchasara To: transport@FreeBSD.org Subject: Re: Correct inflight/pipe calculation Message-ID: <20151028232737.GG5261@strugglingcoder.info> References: <20151007172610.GA42742@strugglingcoder.info> <20151021232210.GL28288@strugglingcoder.info> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="yQbNiKLmgenwUfTN" Content-Disposition: inline In-Reply-To: <20151021232210.GL28288@strugglingcoder.info> User-Agent: Mutt/1.5.23 (2014-03-12) X-BeenThere: freebsd-transport@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Discussions of transport level network protocols in FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Oct 2015 23:27:38 -0000 --yQbNiKLmgenwUfTN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 10/21/15 at 04:22P, hiren panchasara wrote: > On 10/07/15 at 10:26P, hiren panchasara wrote: > > Randall and I have been poking at different ways to improve FreeBSD > > tcp's reaction to loss. One of the major issue we found is that we do > > not use information provided by SACK efficiently even though we do keep > > the SACK scoreboard well in shape. Knowing amount of data in flight can > > be really crucial and can help us use available capacity of the path > > more efficiently. We currently do not have an accurate way of knowing > > this information. > >=20 > > For example, inside tcp_do_segment(), while processing duplicate acks, > > we try to compute amount of data inflight with: > > awnd =3D (tp->snd_nxt - tp->snd_fack) + > > tp->sackhint.sack_bytes_rexmit; > >=20 > > Which is incorrect as it doesn't take into account whats been already > > sacked by the receiver. > > There are definitely other places in the stack where we do this > > incorrectly. > >=20 > > RFC 6675 provides guidance on how to implement calculations for > > bytes in flight at any point in time. Randall and I came to a conclusion > > that following can provide us inflight information almost(!) accurately > > with least amount of code changes: > >=20 > > pipe =3D snd_max - snd_una - sackhint.sacked_bytes + sackhint.sack_byte= s_rexmit; > >=20 > > here, > > snd_max: highest sequence number sent > > snd_una: lowest sequence number sent but not yet cumulatively acked > > sacked_bytes: total bytes sacked by receiver reported via SACK holes > > sack_bytes_rexmit: total bytes retransmitted from holes in this recovery > > period > >=20 > > Only missing piece in FreeBSD is sackd_bytes. This is basically total > > bytes sacked by the receiver and it can be extracted from SACK holes > > reported by the receiver. The approach we've decided to take is pretty > > simple: we already process each ACK with sack holes in=20 > > tcp_sack_doack() and extract sack blocks out of it. We'd now also track > > this new variable there which keeps track of total sacked bytes > > reported. > >=20 > > The downside with this approach is: > > There is no persistent information about sacked bytes. We recalculate > > it every time we get an ACK with sack holes in it. So if, for any > > reason, receiver decides to drop sack info than we get incorrect > > value for inflight. This may be also true when there are more holes but > > receiver can only report 3 at a time. > >=20 > > I have actual code that I've been testing and if people see no major pr= oblem > > with this approach, I can put it up for review in phabricator. >=20 > Well, I didn't receive any replies so here it is: > https://reviews.freebsd.org/D3971 >=20 > Please take a look at that. Closing the loop. This has been committed: https://svnweb.freebsd.org/changeset/base/290122 Cheers, Hiren --yQbNiKLmgenwUfTN Content-Type: application/pgp-signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQF8BAABCgBmBQJWMVnlXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXRBNEUyMEZBMUQ4Nzg4RjNGMTdFNjZGMDI4 QjkyNTBFMTU2M0VERkU1AAoJEIuSUOFWPt/lAsAH/R4CcgPuMBguWagONCc542Fq 7FgQncCFfcy4VOaH1F0uudILJPFdMKd/LbtWYyYRL0Eec5eIF2GbqCq/DXvZjzhe YosyytQ9ctH11ihlcIq23OmERUr/dH3aQWbgSzP2Tp0rZpoGSbnJN8Xdc0ak/Tnz 8dBi1EzF+sslCwO1QtZV4kCb8/203KjRsVtgdJTX8H7eK9si/x4DS7gYBnFCbGUh vsiDainq4eNYlFFCsCZF8Ik5BtsSUxqajRXoH2w5L1SdtdCP4GGRZLz4ldAMcimj Sq7A8X8narUsVz4sgE5BBnLXmXPGHSYZA/pWx6j60VJdhF9kLWYQ3yR3nq69yis= =sRbu -----END PGP SIGNATURE----- --yQbNiKLmgenwUfTN--