From owner-freebsd-performance@FreeBSD.ORG Wed Jun 11 13:55:50 2003 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D76D737B401 for ; Wed, 11 Jun 2003 13:55:50 -0700 (PDT) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5606343F75 for ; Wed, 11 Jun 2003 13:55:50 -0700 (PDT) (envelope-from sean@nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id 928F421058; Wed, 11 Jun 2003 13:55:49 -0700 (PDT) Date: Wed, 11 Jun 2003 13:55:49 -0700 From: Sean Chittenden To: Terry Lambert Message-ID: <20030611205549.GY65470@perrin.int.nxad.com> References: <20030609211526.58641.qmail@web14908.mail.yahoo.com> <3EE4FAED.6090603@centtech.com> <3EE595D2.B223CA19@mindspring.com> <3EE5F8DE.30001@centtech.com> <20030610195632.GQ65470@perrin.int.nxad.com> <3EE6F918.1C1FF28E@mindspring.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <3EE6F918.1C1FF28E@mindspring.com> X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341 X-Web-Homepage: http://sean.chittenden.org/ User-Agent: Mutt/1.5.4i cc: freebsd-performance@freebsd.org cc: Eric Anderson Subject: Re: Slow disk write speeds over network X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Jun 2003 20:55:51 -0000 > > net.inet.tcp.sendspace=65536 > > Double the default. Might not be a good idea, unless you have a ton > of memory. You will potentially use 64K send + 64K receive times > number of sockets. Assuming 4G and near-perfect tuning, you will be > limited to 16384 simultaneous connections fully packed before memory > pressure causes your machine to crash. I tend to like smaller > buffers and more connections. If you only have 512M, drop this > number to 2048 simultaneous connections if all buffers are full. 4GB of memory in this box and it's not directly available to the Internet: a very important point to take into note when tuning. > > kern.maxfiles=65536 > > Seems kind of overkill for the number of connections you can support > without overcommit, and the number of client machines you say you > have. This machine has a busy database on it too. :) > > net.inet.udp.recvspace=65536 > > net.inet.udp.maxdgram=57344 > > These are important for UDP NFS. I do not reccomend it. Other than I do use UDP NFS, so I do recommend these values. :) The relevance of the UDP vs TCP NFS should be kept to the archives, we don't need to re-open that bikeshed. :) > > vfs.nfs.async=1 > > This is very dangerous, if you care about your data. It permits > NFS to ACK writes before they have been committed to stable > storage. With a large enough window size, this should not be > necessary. It's just as dangerous as write caching, same argument for/against write caching, this is no worse. > > net.inet.udp.log_in_vain=1 > > This is just overhead; I reccomend turning it off. Ehh.... there's something floating around someplace where I've been getting errant NFS UDP packets and I haven't figured out what's going on... this is useful for debugging in the absence of a firewall. > > net.inet.icmp.icmplim=20000 > > This is only useful for TCP; but it can be very useful. Basically, > this is "connection rate limiting". If you have a ton of clients, > or trying to "netbench" the system, then set this number up. For > 100 NFS clients, it likely does not matter. I beg to differ... any ideas on how you throttle UDP connections without ICMP? A busy UDP NFS server will require this to be set higher. -sc -- Sean Chittenden