Date: Thu, 22 Jan 2015 15:04:37 -0800 From: "K. Macy" <kmacy@freebsd.org> To: javocado <javocado@gmail.com> Cc: "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Tuning net.inet.tcp.sendbuf_max Message-ID: <CAHM0Q_PLXY1LS17XZPCMMyKSLn-eFr%2BJJK5uzZHiM11ZpHgvhg@mail.gmail.com> In-Reply-To: <CAP1HOmRbOX7RGp-WZBhFZ7P4Os791c3W4oaj2hr6iQQUH%2BYfTA@mail.gmail.com> References: <CAP1HOmRbOX7RGp-WZBhFZ7P4Os791c3W4oaj2hr6iQQUH%2BYfTA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
TCP host cache? See netinet/tcp_hostcache.c for any fiddling that
needs doing. Let me know if there are any values that should be
exported as sysctls.
-K
SYSCTL_INT(_net_inet_tcp_hostcache, OID_AUTO, expire, CTLFLAG_VNET | CTLFLAG_RW,
&VNET_NAME(tcp_hostcache.expire), 0,
"Expire time of TCP hostcache entries");
SYSCTL_INT(_net_inet_tcp_hostcache, OID_AUTO, prune, CTLFLAG_VNET | CTLFLAG_RW,
&VNET_NAME(tcp_hostcache.prune), 0,
"Time between purge runs");
SYSCTL_INT(_net_inet_tcp_hostcache, OID_AUTO, purge, CTLFLAG_VNET | CTLFLAG_RW,
&VNET_NAME(tcp_hostcache.purgeall), 0,
"Expire all entires on next purge run");
/*
* The tcp_hostcache moves the tcp-specific cached metrics from the routing
* table to a dedicated structure indexed by the remote IP address. It keeps
* information on the measured TCP parameters of past TCP sessions to allow
* better initial start values to be used with later connections to/from the
* same source. Depending on the network parameters (delay, bandwidth, max
* MTU, congestion window) between local and remote sites, this can lead to
* significant speed-ups for new TCP connections after the first one.
*
* Due to the tcp_hostcache, all TCP-specific metrics information in the
* routing table have been removed. The inpcb no longer keeps a pointer to
* the routing entry, and protocol-initiated route cloning has been removed
* as well. With these changes, the routing table has gone back to being
* more lightwight and only carries information related to packet forwarding.
*
* tcp_hostcache is designed for multiple concurrent access in SMP
* environments and high contention. All bucket rows have their own lock and
* thus multiple lookups and modifies can be done at the same time as long as
* they are in different bucket rows. If a request for insertion of a new
* record can't be satisfied, it simply returns an empty structure. Nobody
* and nothing outside of tcp_hostcache.c will ever point directly to any
* entry in the tcp_hostcache. All communication is done in an
* object-oriented way and only functions of tcp_hostcache will manipulate
* hostcache entries. Otherwise, we are unable to achieve good behaviour in
* concurrent access situations. Since tcp_hostcache is only caching
* information, there are no fatal consequences if we either can't satisfy
* any particular request or have to drop/overwrite an existing entry because
* of bucket limit memory constrains.
*/
On Wed, Jan 21, 2015 at 11:04 AM, javocado <javocado@gmail.com> wrote:
> System: FreeBSD 8.3 amd64
>
> I've been trying to tune my system to a long route (RTT 180ms) and I've
> made 2 modifications and seeing some results which I cannot explain or
> understand.
>
> 1. kern.ipc.maxsockbuf: 262144 -> 4194304
>
> Speed improves from 85 Mbps -> 100 Mbps
>
> I can run the test: iperf -c tserv20.hkg1.ipv6.he.net -t10 -P 10
>
> all day long and consistently get 100Mbps
>
>
> 2. net.inet.tcp.sendbuf_max: 262144 -> 524288
>
> 1st run: 156Mbps
>
> BUT, on subsequent runs, just moment later, I see the speed drop off
> quickly with each successive run:
>
> 73 ... 50Mbps
>
>
> Simply returning sendbuf_max to the original value of 262144 does not
> return me to 100Mbps. Why is this?
> Setting net.inet.tcp.sendbuf_max, net.inet.tcp.recvbuf_max
> and kern.ipc.maxsockbuf to 4194304 also makes no difference.
>
> I noticed that the speed returned to 100Mbps when I set it back to 262144
> and re-ran the test ~12hrs later. Is there some kind of buffer that needs
> to be cleared (or clears with enough time) so these changes have immediate
> effect when returning these sysctls to original values?
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHM0Q_PLXY1LS17XZPCMMyKSLn-eFr%2BJJK5uzZHiM11ZpHgvhg>