Date: Tue, 25 Aug 2015 18:03:23 -0700 From: John-Mark Gurney <jmg@funkthat.com> To: Chris Stankevitz <chris@stankevitz.com> Cc: freebsd-net@freebsd.org Subject: Re: ssh over WAN: TCP window too small Message-ID: <20150826010323.GN33167@funkthat.com> In-Reply-To: <55DCF080.7080208@stankevitz.com> References: <55DCF080.7080208@stankevitz.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Chris Stankevitz wrote this message on Tue, Aug 25, 2015 at 15:47 -0700: > # cat /dev/urandom | ssh root@host 'cat > /dev/null' Don't use this for testing... use /dev/zero or some other device that can produce data faster than this... > I use the above ssh command over a high-BDP WAN link (80 ms @ 100 Mbps). > tcpdump shows I am TCP window limited to 64 KBytes (yielding 5 Mbps). > iperf with default options gets the window opened to 500 KBytes > (yielding 35 Mbps). > > Both sides of the connection: FreeBSD 10.1 w/default sshd options > (except I permit root login). In particular, HPN is not disabled. > > Can anyone explain my abysmally small TCP window? Looks like ssh is propbably hard setting the send/recv buffers to values that are too small... So, our SSH does have the HPN patches: https://www.psc.edu/index.php/hpn-ssh and the README says: BUFFER SIZES: - if HPN is disabled the receive buffer size will be set to the OpenSSH default of 64K. You can read more at: https://svnweb.freebsd.org/base/stable/10/crypto/openssh/README.hpn?annotate=256281 Looks like there are undocumented options like TCPRcvBuf that you can use to adjust the recv buffer window... It looks like OpenSSH hard sets the buffer sizes for some reason... On FreeBSD, these should never be set unless the option is provided and you know what you are doing.. We have code that will auto grow buffer sizes properly so that slow connections won't use up too much buffer space... > Can anyone recommend some tools/tricks to figure out what in FreeBSD > and/or base SSH is limiting the send/recv buffer and/or TCP window? Seems like from looking at the code, things should "just work", so not sure why you are seeing the smaller window size... In a quick test of mine, I'm seeing a buffer size of ~520k from my MacOSX box, and ~776k from my 9.2-R box... Server in both cases is a June -CURRENT... netstat -xAanfinet is helpful on this... Hope this helps! -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20150826010323.GN33167>