Date: Sun, 20 Mar 2011 15:22:51 +1100 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Jeff Roberson <jroberson@jroberson.net> Cc: Alexander Leidinger <Alexander@Leidinger.net>, George Neville-Neil <gnn@neville-neil.com>, arch@FreeBSD.org Subject: Re: Updating our TCP and socket sysctl values... Message-ID: <20110320151003.A939@besplex.bde.org> In-Reply-To: <alpine.BSF.2.00.1103191157010.1480@desktop> References: <132388F1-44D9-45C9-AE05-1799A7A2DCD9@neville-neil.com> <20110319160400.000043f5@unknown> <alpine.BSF.2.00.1103191157010.1480@desktop>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 19 Mar 2011, Jeff Roberson wrote: > On Sat, 19 Mar 2011, Alexander Leidinger wrote: > >> On Sat, 19 Mar 2011 15:37:47 +0900 George Neville-Neil >> <gnn@neville-neil.com> wrote: >>> >>> I believe it's time for us to upgrade our sysctl values for TCP >>> sockets so that they are more in line with the modern world. At the >>> moment we have these limits on our buffering: >>> >>> kern.ipc.maxsockbuf: 262144 >>> net.inet.tcp.recvbuf_max: 262144 >>> net.inet.tcp.sendbuf_max: 262144 >>> >>> I believe it's time to up these values to something that's in line >>> with higher speed local networks, such as 10G. Perhaps it's time to >>> move these to 2MB instead of 256K. All hard-coded limits are bogus. The same limit for a machine that has 8MB memory is nonense for a machine that has 8GB. In FreeBSD, AFAIK only the vm system has _very_ good auto-tuning of parameters and limits thanks to dyson's work 10-15 years ago. It has almost no user-settable parameters or limits like the above. >> I suggest to read >> http://www.bufferbloat.net/projects/bloat/wiki/Bufferbloat >> and do a before/after test to make sure we do not suffer from the >> described problem. Jim Getty has test descriptions: >> http://gettys.wordpress.com/category/bufferbloat/ > > Are they not talking about buffers in non-endpoint devices? Or perhaps even > overly large rx queues in endpoints, but not local socket receive buffers? > It seems that they are describing situations where excessive buffering masks > network conditions until it's too late. I don't know, but there is an mostly-unrelated bufferbloat problem that is purely local. If you have a buffer that is larger than an Ln cache (or about half than), then actually using just a single buffer of that size guarantees thrashing of the Ln cache, so that almost every memory access is an Ln cache miss. Even with current hardware, a buffer of size 256K will thrash most L1 caches and a buffer of size a few MB will thrash most L2 caches. Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110320151003.A939>