From owner-freebsd-questions Tue Nov 5 1:40:36 2002 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 01C2937B401 for ; Tue, 5 Nov 2002 01:40:34 -0800 (PST) Received: from mail.duth.gr (mail.duth.gr [192.108.114.110]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9835543E7B for ; Tue, 5 Nov 2002 01:40:32 -0800 (PST) (envelope-from bigbrother@bonbon.net) Received: from bigb3server.bbcluster.gr (b9-149.xan.duth.gr [193.92.211.149]) by mail.duth.gr (8.12.6/8.12.6) with ESMTP id gA59eTGh055593 (version=TLSv1/SSLv3 cipher=EDH-DSS-DES-CBC3-SHA bits=168 verify=NO) for ; Tue, 5 Nov 2002 11:40:29 +0200 (EET) (envelope-from bigbrother@bonbon.net) Received: from bigb3server.bbcluster.gr (localhost.bbcluster.gr [127.0.0.1]) by bigb3server.bbcluster.gr (8.12.3/8.12.3) with ESMTP id gA59eEJK090161 for ; Tue, 5 Nov 2002 11:40:14 +0200 (EET) (envelope-from bigbrother@bonbon.net) Received: from localhost (bigbrother@localhost) by bigb3server.bbcluster.gr (8.12.3/8.12.3/Submit) with ESMTP id gA59eDHI090149 for ; Tue, 5 Nov 2002 11:40:14 +0200 (EET) X-Authentication-Warning: bigb3server.bbcluster.gr: bigbrother owned process doing -bs Date: Tue, 5 Nov 2002 11:40:13 +0200 (EET) From: BigBrother X-X-Sender: bigbrother@bigb3server.bbcluster.gr To: questions@FreeBSD.ORG Subject: Re: NFS Performance woes In-Reply-To: Message-ID: <20021105112912.Q69960-100000@bigb3server.bbcluster.gr> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Spam-Status: No, hits=-3.4 required=10.0 tests=IN_REP_TO,SPAM_PHRASE_03_05,X_AUTH_WARNING version=2.41 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG >I recently did some research into NFS performance tuning and came across >the suggestion in an article on onlamp.com by Michael Lucas, that 32768 >is a good value for the read and write buffers. His suggestion is these >flags: > >tcp,intr,nfsv3,-r=32768,-w=32768 > >I used these options (I found tcp was mandatory, as we have multiple IPs >and UDP was refusing to play nice), also adding dumbtimer to avoid the >log messages about server not responding. According to my experience UDP is much preffered for NFS transport protocols. Also try to have the NFSIOD daemon being executed on every machine by putting in the /etc/rc.conf nfs_client_enable="YES" nfs_client_flags="-n 10" [u may put more than 10 instances if u suspect that more than 10 simultaneous transactions will happen] Also use the -w=32768,-r=32768 switch only on the machines that have a fast cpu and a good network card [e.g. nestat -w 1, doesnt show errors under heavy load] On all the other machines dont put any w,r values [which will default to 8k blocks] In some machines of mine I have even used blocks of -r=4096,-w=4096 because they were old machines that could not keep up with the traffic and they were complaining about mbufs [they run out of mbufs and after some time they crashed]..(and because they machines were diskless it was unable to change the value of mbufss, after the kernel loading the value is readonly and cannot be changed). Use good networking hardware...scrappy hardware will certainly put you into great trouble. If you use TCP for NFS on a 1GB network you will sure have problems on your machines and they will not be able to keep up. TCP causes a great overhead. UDP doesnt. So bottom line: a) Use UDP b) Run a lot of NFSIOD - the more the better c) Examine what is the best block size for every host idividually! (dont assume that 32k block is good for every host) Hopes it does your job......I was searching for over 3 months when I once dealt with this....Read also from the 'Sun' site the 'Optimizing and Tunning NFS' guide which is a nice PDF document that you can download for free, and has a lot of interesting things similar with FreeBSD! To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message