Date: Wed, 28 Aug 2013 17:55:20 -0400 From: Garrett Wollman <wollman@bimajority.org> To: Rick Macklem <rmacklem@uoguelph.ca> Cc: freebsd-fs@freebsd.org Subject: Re: NFS on ZFS pure SSD pool Message-ID: <21022.29128.557471.157078@hergotha.csail.mit.edu> In-Reply-To: <1342658741.14983067.1377722983208.JavaMail.root@uoguelph.ca> References: <CAM=5oeAWbV1wzscnTHHH1=FFQ6DYjB%2BpriGowe1WcBfM=SsPXg@mail.gmail.com> <1342658741.14983067.1377722983208.JavaMail.root@uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
<<On Wed, 28 Aug 2013 16:49:43 -0400 (EDT), Rick Macklem <rmacklem@uoguelph.ca> said: > Eric Browning wrote: >> Sam and I applied the patch (kernel now at r254983M) and set >> vfs.nfsd.tcphighwater=5000 in sysctl.conf and my CPU is still >> slammed. SHould I up it to 10000? >> > You can try. I have no insight into where this goes, since I can't > produce the kind of server/load where it makes any difference. (I have > single core i386 (P4 or similar) to test with and I don't use ZFS at all.) > I've cc'd Garrett Wollman, since he runs rather large servers and may > have some insight into appropriate tuning, etc. 10,000 is probably way too small. We run high-peformance servers with vfs.nfsd.tcphighwater set between 100k and 150k, and we crank vfs.nfsd.tcpcachetimeo down to five minutes or less. Just to give you an idea of how rarely this cache is actually hit: my two main production file servers have both been up for about three months now, and have answered billions of requests (enough for the 32-bit signed statistics counters to wrap). One server shows 63 hits, with a peak TCP cache size of 150k and the other shows zero, with a peak cache size of 64k. Another server, which serves scratch space, has been up for a little more than a month, and in nearly two billion accesses has yet to see a single cache hit (peak cache size 131k, which was actually hitting the configured limit, which I've since raised). -GAWollman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?21022.29128.557471.157078>