From owner-freebsd-questions Wed Nov 6 17:28:57 2002 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E75ED37B401 for ; Wed, 6 Nov 2002 17:28:54 -0800 (PST) Received: from mail.au.darkbluesea.com (mail.au.darkbluesea.com [203.185.208.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1BE6643E42 for ; Wed, 6 Nov 2002 17:28:53 -0800 (PST) (envelope-from d.anker@au.darkbluesea.com) Received: (qmail 42262 invoked by uid 82); 7 Nov 2002 01:27:21 -0000 Received: from unknown (HELO ?10.0.0.188?) (10.0.0.188) by mail.au.darkbluesea.com with SMTP; 7 Nov 2002 01:27:21 -0000 Subject: Re: NFS Performance woes From: Duncan Anker To: BigBrother Cc: questions@FreeBSD.ORG In-Reply-To: <20021106114019.W69960-100000@bigb3server.bbcluster.gr> References: <20021106114019.W69960-100000@bigb3server.bbcluster.gr> Content-Type: text/plain Content-Transfer-Encoding: 7bit X-Mailer: Ximian Evolution 1.0.8 Date: 07 Nov 2002 11:28:42 +1000 Message-Id: <1036632522.25128.27.camel@duncan> Mime-Version: 1.0 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Wed, 2002-11-06 at 19:52, BigBrother wrote: > > > Although the man page says this, I *think* that the communication is done > like this > > CLIENT <=> NFSIOD(CLIENT) <=> NFSIOD (SERVER) <=> NFSD > > which menas that NFSIOD 'speak' with each other and then they pass the > requests to NFS. > > Of course if u dont need to have on the server too many NFSIOD. So in my > case I just have 8 nfsiod on server running and most of them are idle, and > besides, they only take 1.5MB of memory which I can afford. So I think > having *some* NFSIOD also on server is not a bad idea. Of course on the > server u should have a lot of NFSD. > > in other words, running NFSIOD on server is not a bad idea.. NFSIOD is enabled by putting nfs_client_enable="YES" into rc.conf. Empirical evidence suggests that NFSIOD is not used server-side. I ran 4 nfsiod daemons on the server and checked their usage time. All were 0:00. If the server does any client NFS stuff it would make a difference, and it may be different under other OS. Certainly it does no harm to have them running. > > Also monitor the mbufs on all your machines (especially the server). > > do from time to time a 'netstat -m' and see the peak value of mbuf > and mbuf clusters...if it is close to the max limit then you will suffer > from mbuff exhaustion will will eventually make the machine unreachable > from network. > > > u can change mbuff max value before kernel load ...see tuning (7) kern.nmbclusters="" in /boot/loader.conf, if anyone needs to do this. Have to reboot for this one :-( > > > Also if u have mbuf exhastion try to use smaller block size in NFS mounts. Now this is interesting. I had thought mbuf cluster exhaustion was due to a high number of connections. Although I guess a high number of connections * large buffer size would do it too. Thank you for your response and suggestions vis the other NFS stuff - I managed to get our server talking UDP. The wildcard binding was the problem, and the -h flag to nfsd fixed it. Network usage graphs are showing the differential between incoming and outgoing traffic to be much less now, so I would say there was a lot of overhead in there, as well as retransmissions. I am still playing with buffer sizes but chances are in this case the FreeBSD default is best. Regards -- Duncan Anker Senior Systems Administrator Dark Blue Sea To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message