From owner-freebsd-stable@FreeBSD.ORG Sat May 20 17:58:17 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7D8DC16A424 for ; Sat, 20 May 2006 17:58:17 +0000 (UTC) (envelope-from howard@leadmon.net) Received: from ibm.leadmon.net (ibm.leadmon.net [207.114.24.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id DADA643D46 for ; Sat, 20 May 2006 17:58:14 +0000 (GMT) (envelope-from howard@leadmon.net) Received: from gamer (gamer.leadmon.net [207.114.24.7]) (authenticated bits=0) by ibm.leadmon.net (8.13.6/8.13.6/LNSG+ORDB+SCOP+NJABL+SBL+DSBL+SORBS+CBL+RHSBL) with ESMTP id k4KHv5ej084186; Sat, 20 May 2006 13:58:13 -0400 (EDT) (envelope-from howard@leadmon.net) Authentication-Results: ibm.leadmon.net from=howard@leadmon.net; sender-id=pass; spf=pass X-SenderID: Sendmail Sender-ID Filter v0.2.12 ibm.leadmon.net k4KHv5ej084186 From: "Howard Leadmon" To: "'Kris Kennaway'" Date: Sat, 20 May 2006 13:57:00 -0400 Organization: Leadmon Networking Message-ID: <001e01c67c36$f5126720$071872cf@Leadmon.local> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit X-Mailer: Microsoft Office Outlook 11 Thread-Index: AcZ3ylKgpFVrMfGMQVG/NXupI1LitgEa03Pg X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.2869 In-Reply-To: <20060515024958.GA99002@xor.obsecurity.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0 (ibm.leadmon.net [207.114.24.13]); Sat, 20 May 2006 13:58:13 -0400 (EDT) X-Virus-Scanned: ClamAV 0.88.2/1472/Sat May 20 04:11:04 2006 on ibm.leadmon.net X-Virus-Status: Clean Cc: freebsd-stable@freebsd.org Subject: RE: Trouble with NFSd under 6.1-Stable, any ideas? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 20 May 2006 17:58:17 -0000 Sorry for delay, ended up sick.. :( You say use tcpdump, is there something I should be looking out for? As NFS is serving files, even more strange is if I kill off the nfsd process it's zippy fast for a moment and then the CPU load goes through the roof, and it starts serving files slowly. So it's actually working, outside of it consumes all available CPU and brings the machine to it knees quickly. Doesn't matter if I access it from my Solaris box, my other FBSD boxes, and so on, it still dogs down terribly and never used. Anything anyone can think of config wise that might cause that, it would be nice to know. I have the following that I can think of that affects NFS configs: # # NFS # nfs_client_enable="NO" # This host is an NFS client (or NO). nfs_access_cache="2" # Client cache timeout in seconds nfs_server_enable="YES" # This host is an NFS server (or NO). nfs_server_flags="-u -t -n 5" # Flags to nfsd (if enabled). mountd_enable="YES" # Run mountd (or NO). mountd_flags="-r" # Flags to mountd (if NFS server enabled). weak_mountd_authentication="NO" # Allow non-root mount requests to be served. nfs_reserved_port_only="YES" # Provide NFS only on secure port (or NO). nfs_bufpackets="" # bufspace (in packets) for client rpc_lockd_enable="YES" # Run NFS rpc.lockd needed for client/server. rpc_statd_enable="YES" # Run NFS rpc.statd needed for client/server. rpcbind_enable="YES" # Run the portmapper service (YES/NO). rpcbind_program="/usr/sbin/rpcbind" # path to rpcbind, if you want a differe rpcbind_flags="" # Flags to rpcbind (if enabled). I can't think of anything that should have changed, unless mergemaster updating the default files might have changed something that would have an effect. --- Howard Leadmon http://www.leadmon.net > -----Original Message----- > From: owner-freebsd-stable@freebsd.org > [mailto:owner-freebsd-stable@freebsd.org] On Behalf Of Kris Kennaway > Sent: Sunday, May 14, 2006 10:50 PM > To: Howard Leadmon > Cc: freebsd-stable@freebsd.org > Subject: Re: Trouble with NFSd under 6.1-Stable, any ideas? > > On Sun, May 14, 2006 at 02:28:55PM -0400, Howard Leadmon wrote: > > > > Hello All, > > > > I have been running FBSD a long while, and actually > running since the 5.x > > releases on the server I am having troubles with. I > basically have a small > > network and just use NIS/NFS to link my various FBSD and Solaris > > machines together. > > > > This has all been running fine up till a few days ago, > when all of a > > sudden NFS came to a crawl, and CPU usage so high the box > appears to freeze almost. > > When I had 6.1-RC running all seemed well, then came the > announcement > > for the official 6.1 release, so I did the cvs updates, made world, > > kernel, and ran mergemaster to get everything up to the 6.1 > stable version. > > > > Now after doing this, something is wrong with NFS. It > works, it will return > > information and open files, just it's very very slow, and while > > performing a request the CPU spike is astounding. A simple > du of my > > home directory can take minutes, and machine all but locks > up if the request is done over NFS. > > Here is top snip: > > > > PID USERNAME THR PRI NICE SIZE RES STATE C TIME > WCPU COMMAND > > 497 root 1 4 0 1252K 780K - 2 50:42 > 188.48% nfsd > > > > > > This is a nice IBM eServer with dual P4-XEON's and a > couple GB or RAM > > on a disk array, and locally is screams, heck NFS used to > scream till > > I updated. I am not really sure what info would be useful in > > debugging, so won't post tons of misc junk in this eMail, but if > > anyone has any ideas as to how best to figure out and > resolve this issue it would sure be appreicated... > > Use tcpdump and related tools to find out what traffic is being sent. > > Also verify that you did not change your system configuration in any > way: there have been no changes to NFS since the release, so > it is unclear why an update would cause the problem to suddenly occur. > > Kris > >