From owner-svn-src-head@freebsd.org Fri Jun 8 00:58:26 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0837A100A3A9 for ; Fri, 8 Jun 2018 00:58:26 +0000 (UTC) (envelope-from rrs@netflix.com) Received: from mail-qt0-x232.google.com (mail-qt0-x232.google.com [IPv6:2607:f8b0:400d:c0d::232]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 8B8856961E for ; Fri, 8 Jun 2018 00:58:25 +0000 (UTC) (envelope-from rrs@netflix.com) Received: by mail-qt0-x232.google.com with SMTP id s9-v6so11903953qtg.2 for ; Thu, 07 Jun 2018 17:58:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=ZGK5DS+n8zG/JwvLnebKqP9pFKFH2t37zP192E44Pw0=; b=ZA05YrfngbjPtYURm+q7ChR02QPj7EmoOZJ4fbB77eyhQu+jwn7kix4CSHTeGj0iwo P5szRFBSFFaxS6F4htf/fxKdrrKYiOk6H3KcmBbJQdR+/oLkwjpaY2CIS5UFuhAKZREJ nyQwkz5hZ6KEYSrKXXMPAvbdEv6QkPbxon3qDt3DpTxBITEH5dkKexHNw3CUWgCHd02L uVF4t/s4UPTBbEQB0ck+brjCzcRmseBpzv0fykN+cefr/8/qtweWTOnDjyfPDVcJWQHa C82B1+z91L8JB47eu5951jzkvBLJGWUiZ4bt1Kx3jwB6y2q7TQVWhGqUASGFTv1bpbaK zI7A== X-Gm-Message-State: APt69E0DxEn9hmaMoo87jUPt9tnlrh22+TKpk5qIOFNcxFXaE/Tz0azH +6t6X7uAX7p9S0RNCTFDQvC4Fw== X-Google-Smtp-Source: ADUXVKJ6pvMkIozr3l5zE+XDGmvy0T856fjYwn/TyQtNSuS9upjcdl1GkVi1bFgBt697uUmNGVNtig== X-Received: by 2002:aed:242a:: with SMTP id r39-v6mr4003307qtc.207.1528419504911; Thu, 07 Jun 2018 17:58:24 -0700 (PDT) Received: from [172.20.1.52] ([38.64.177.126]) by smtp.gmail.com with ESMTPSA id p80-v6sm19829655qka.83.2018.06.07.17.58.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 07 Jun 2018 17:58:23 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 11.3 \(3445.6.18\)) Subject: Re: svn commit: r334804 - in head/sys: kern modules/tcp modules/tcp/rack netinet netinet/tcp_stacks sys From: Randall Stewart In-Reply-To: <20180607220121.GC62424@strugglingcoder.info> Date: Thu, 7 Jun 2018 20:58:21 -0400 Cc: Randall Ray Stewart , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <40CCC8B4-5667-4D4B-A4B4-7AD5779FE011@netflix.com> References: <201806071818.w57IIENp080093@repo.freebsd.org> <20180607220121.GC62424@strugglingcoder.info> To: hiren panchasara X-Mailer: Apple Mail (2.3445.6.18) X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Jun 2018 00:58:26 -0000 > On Jun 7, 2018, at 6:01 PM, hiren panchasara = wrote: >=20 > On 06/07/18 at 06:18P, Randall Stewart wrote: >> Author: rrs >> Date: Thu Jun 7 18:18:13 2018 >> New Revision: 334804 >> URL: https://svnweb.freebsd.org/changeset/base/334804 >>=20 >> Log: >> This commit brings in a new refactored TCP stack called Rack. >> Rack includes the following features: >> - A different SACK processing scheme (the old sack structures are = not used). >> - RACK (Recent acknowledgment) where counting dup-acks is no longer = done >> instead time is used to knwo when to retransmit. (see the = I-D) >> - TLP (Tail Loss Probe) where we will probe for tail-losses to = attempt >> to try not to take a retransmit time-out. (see the I-D) >> - Burst mitigation using TCPHTPS >> - PRR (partial rate reduction) see the RFC. >>=20 >> Once built into your kernel, you can select this stack by either >> socket option with the name of the stack is "rack" or by setting >> the global sysctl so the default is rack. >>=20 >> Note that any connection that does not support SACK will be kicked >> back to the "default" base FreeBSD stack (currently known as = "default"). >>=20 >> To build this into your kernel you will need to enable in your >> kernel: >> makeoptions WITH_EXTRA_TCP_STACKS=3D1 >> options TCPHPTS >>=20 >> Sponsored by: Netflix Inc. >> Differential Revision: = https://reviews.freebsd.org/D15525 >>=20 >> Added: >> head/sys/modules/tcp/rack/ >> head/sys/modules/tcp/rack/Makefile (contents, props changed) >> head/sys/netinet/tcp_stacks/rack.c (contents, props changed) >> head/sys/netinet/tcp_stacks/rack_bbr_common.h (contents, props = changed) >> head/sys/netinet/tcp_stacks/sack_filter.c (contents, props = changed) >> head/sys/netinet/tcp_stacks/sack_filter.h (contents, props = changed) >> head/sys/netinet/tcp_stacks/tcp_rack.h (contents, props changed) >> Modified: >> head/sys/kern/uipc_sockbuf.c >> head/sys/modules/tcp/Makefile >> head/sys/netinet/tcp.h >> head/sys/netinet/tcp_log_buf.h >> head/sys/netinet/tcp_output.c >> head/sys/netinet/tcp_stacks/fastpath.c >> head/sys/netinet/tcp_timer.c >> head/sys/netinet/tcp_timer.h >> head/sys/netinet/tcp_var.h >> head/sys/sys/mbuf.h >> head/sys/sys/queue.h >> head/sys/sys/sockbuf.h >> head/sys/sys/time.h >=20 > I thought we'd have more time to review/test this. Looks like BSDCan > commit-spree in effect. :-) The Phabricator review has been up since May 22nd. Thats over 2.5 weeks, this was also discussed on the Thursday conference calls. >=20 > A few questions: > 1) Does RACK work reliably without HPTS? If yes, has that config been > tested? >=20 No it requires the pacer. > 2) It looks like PRR is tied to RACK. Why did we go that route? > Shouldn't it be easily used with the 'default' stack also? >=20 It is what I developed.. and I had no desire to work with the default = stack. That is a fifth rail that no one wants touched. > 3) Can new SACK be used with the traditional stack? Well if you want to rework the base stack you might be able to do that = :) It would be quite some effort.. I think Robert wants eventually the old stack to be de-composed and then slowly work at getting more common code between them until eventually you can have a diff and somehow figure out how to integrate the two. >=20 > 4) Where should manpage like info for RACK go? a new man-page or > extending tcp(4)? Info like how to enable system-wide or per socket > should go here. >=20 The enable/disable or per-socket I think is in with the pluggable stack stuff. We might want a Rack man page.. have to think about it. > 5) Any perf numbers to go along with this commit? Synthetic or > production numbers showing improvements in transfer speed or any other > impact on CPU usage (specially with HPTS) that you can share? >=20 CPU will be more but we see close to a drop in rebuffers by about 12% I = am told. > 6) In your testing, have you found cases where RACK does poorly = compared > to the 'default' stack? Any recommendations on when should RACK be > enabled? (Something like this could go in the manpage.) Nope.=20 R >=20 > Glad to finally see this in -head! >=20 > Cheers, > Hiren -------- Randall Stewart rrs@netflix.com 803-317-4952