Date: Tue, 01 Oct 2002 17:27:34 -0700 From: Eli Dart <dart@nersc.gov> To: net@FreeBSD.ORG Subject: Re: limit to data in flight Message-ID: <20021002002734.4AF843B1AE@gemini.nersc.gov> In-Reply-To: Message from Eli Dart <dart@nersc.gov> of "Tue, 01 Oct 2002 12:44:55 PDT." <20021001194455.552A33B1AE@gemini.nersc.gov>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --]
Sorry to reply to my own post, but I have some updated info...
It appears that a host route is being cloned from the default route.
The host route has an ssthresh value associated with it, which is
picked up by subsequent connections that use this route.
We attempted to lock the ssthresh value at 0 (which would prevent its
being used) by configuring the default route with an ssthresh of 0
and locking ssthresh. (route add -lock -ssthresh 0 default <rtr_addr> )
The first connection created a host route to the destination with
rtt, ssthresh, etc identical to the default route, including the lock
on ssthresh. Subsequent connections modified the value of ssthresh,
even though there is a lock on it!
After first test:
# route get portal.astro.sunysb.edu
route to: portal.astro.sunysb.edu
destination: portal.astro.sunysb.edu
gateway: ernersc-128
interface: sk0
flags: <UP,GATEWAY,HOST,DONE,WASCLONED>
recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire
0 0 0L 0 0 0 1500 0
# route monitor
(now run the second test)
got message of size 196 on Tue Oct 1 16:12:08 2002
RTM_GET: Report Metrics: len 196, pid: 184, seq 1, errno 0,
flags:<UP,GATEWAY,HOST,DONE,WASCLONED>
locks: <ssthresh> inits:
sockaddrs: <DST,GATEWAY,IFP,IFA>
portal.astro.sunysb.edu ernersc-128 sk0:0.0.5a.99.73.64 iperf
^C
# route get portal.astro.sunysb.edu
route to: portal.astro.sunysb.edu
destination: portal.astro.sunysb.edu
gateway: ernersc-128
interface: sk0
flags: <UP,GATEWAY,HOST,DONE,WASCLONED>
recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire
0 0 221712L 72 2 0 1500 0
So, it appears that the lock is not being honored. Do I have this
right, or am I missing something somewhere? We've found a workaround
for now: edit /usr/src/sys/netinet/tcp_input.c thusly:
*** tcp_input.c.cya Tue Oct 1 15:57:35 2002
--- tcp_input.c Tue Oct 1 16:17:55 2002
***************
*** 2649,2664 ****
else
tp->snd_cwnd = mss * ss_fltsz;
! if (rt->rt_rmx.rmx_ssthresh) {
/*
* There's some sort of gateway or interface
* buffer limit on the path. Use this to set
* the slow start threshhold, but set the
* threshold to no less than 2*mss.
*/
! tp->snd_ssthresh = max(2 * mss, rt->rt_rmx.rmx_ssthresh);
! tcpstat.tcps_usedssthresh++;
! }
}
/*
--- 2649,2664 ----
else
tp->snd_cwnd = mss * ss_fltsz;
! /* if (rt->rt_rmx.rmx_ssthresh) { */
/*
* There's some sort of gateway or interface
* buffer limit on the path. Use this to set
* the slow start threshhold, but set the
* threshold to no less than 2*mss.
*/
! /* tp->snd_ssthresh = max(2 * mss, rt->rt_rmx.rmx_ssthresh); */
! /* tcpstat.tcps_usedssthresh++; */
! /* } */
}
/*
All this does is make the tcp socket ignore any ssthresh value set on
the route....it's like cutting chopsticks with a chainsaw....
Should the lock be preventing updates to ssthresh for that route?
Or, am I misinterpreting something?
--eli
In reply to Eli Dart <dart@nersc.gov> :
>
> --==_Exmh_1318550241P
> Content-Type: text/plain; charset=us-ascii
>
> Hi all,
>
> I'm seeing something strange here... I have a freebsd box running
> iperf (4.6-RELEASE-p1, iperf 1.6.2 with pthreads patches). When
> attempting to use a 1MB tcp window, the box won't put more than 256kB
> in flight after the first connection to a given host.
>
> I seem to remember hearing/reading/whatever that freebsd keeps track
> of congestion stats for a route in the kernel routing table and
> primes the congestion window for new sockets to the same destination
> with the previous values (thus eliminating a congestion avoidance
> cycle on each new socket). High-bandwidth connections between the
> hosts in question (the other is a linux box of indeterminate recent
> version) do hit congestion the first time.
>
> However, in this particular case (since this is a test machine that
> we use to diagnose network problems) I'd like to be able to turn it
> off. I didn't see anything in sysctl that looked obvious, but I'm
> perfectly willing to believe I missed it.
>
> So, can this be turned off? Also, what is the timeout on this data
> in the kernel?
>
> Thanks!
>
> --eli
[-- Attachment #2 --]
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (FreeBSD)
Comment: This is a comment.
iD8DBQE9mj12LTFEeF+CsrMRAkU1AKDeLrtOve/DxN9eR+JU//HGIjjp9ACaA0Al
+YwC+hNDo7zxWvi/OT8ZDUU=
=f0oJ
-----END PGP SIGNATURE-----
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20021002002734.4AF843B1AE>