From owner-freebsd-stable@FreeBSD.ORG Tue May 23 14:27:03 2006 Return-Path: X-Original-To: freebsd-stable@freebsd.org Delivered-To: freebsd-stable@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EB44416A4C5 for ; Tue, 23 May 2006 14:27:03 +0000 (UTC) (envelope-from grafan@gmail.com) Received: from nf-out-0910.google.com (nf-out-0910.google.com [64.233.182.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id D9B5243D62 for ; Tue, 23 May 2006 14:27:02 +0000 (GMT) (envelope-from grafan@gmail.com) Received: by nf-out-0910.google.com with SMTP id k27so780688nfc for ; Tue, 23 May 2006 07:27:01 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=okjZofwjG9cxd7UmPD/K2WiiE6sTAfbDU9fXodEYKBmqsx0AgKQmdcHjd1XORRHusaNsNdV325EfaTmthwD8LL/27zmZh5+c5GEgVPsnv9E8+oMxe8KHACKfQL08dcWQfsU0QJk0iYVF1L0dCLE3t7pXEIkElgr7iwGkm+P0BtI= Received: by 10.48.225.3 with SMTP id x3mr499804nfg; Tue, 23 May 2006 07:18:09 -0700 (PDT) Received: by 10.48.30.7 with HTTP; Tue, 23 May 2006 07:18:44 -0700 (PDT) Message-ID: <6eb82e0605230718l337a58efmc85637afdec4fffb@mail.gmail.com> Date: Tue, 23 May 2006 10:18:44 -0400 From: "Rong-en Fan" To: "Howard Leadmon" In-Reply-To: <013b01c67e71$23aaacf0$071872cf@Leadmon.local> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <6eb82e0605230556n31b86e55y1b07a2ef6ad9ca14@mail.gmail.com> <013b01c67e71$23aaacf0$071872cf@Leadmon.local> Cc: Konstantin Belousov , freebsd-stable@freebsd.org, Kris Kennaway Subject: Re: Trouble with NFSd under 6.1-Stable, any ideas? X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 May 2006 14:27:09 -0000 On 5/23/06, Howard Leadmon wrote: > > > > > > If there are any thing I can provide to help tracking this down. > > > > Please let me know. By the way, I tried with truss/kdump > > to see what > > > > happens when nfsd eats lot of CPUs, but in vain. They do > > not return anything. > > > > > > > I tried your recipe on 7-CURRENT with locally exported fs, > > remounted > > > over nfs. I did not get the behaviour your described. > > > > As noted in my previous thread, I have another 6.1-RELEASE > > nfs server, which does not have this problem. > > > > > Could you, please, provide the backtrace for the nfsd that eats the > > > CPU (from the ddb). I think it would be helpful to get several > > > backtraces (i.e., bt , cont, bt ...) > > to see where > > > it running. > > > > I'm afraid that I can not do that. Last time I tried breaking > > into ddb (on 5.x), it hangs my serial console and the server > > is miles away :-( . Perhaps we can ask Howard to do that? > > I am more than willing to do that, as this machine runs here with me, so= if > needed I can easily get on a console, or perform a reboot. Can one of yo= u > shed a little light on exactly what I need to do, and how to do this? I = ask > as I have never used this ddb stuff, so not clue one on how to go about > getting the information your looking to find. Guess I have been lucky, a= nd > just never had an issue that took things to this level. At least you have to add the following to your kernel: options KDB options DDB Recompile it, reboot. You would better to setup a serial console so you can easily copy thing from ddb output. To do it, you have to put "device sio" in your kernel configuration and some files below: /boot.config -Dh /boot/loader.conf comconsole_speed=3D115200 machdep.conspeed=3D115200 /etc/ttys ttyd0 "/usr/libexec/getty std.115200" cons25 on secure On the other machine, /etc/remote: com1:dv=3D/dev/cuad0:br#115200:pa=3Dnone: Then, use "tip com1" to attach the nfs server. The above settings assume your serial console on nfs server is on COM1 and on the client side is also COM1. If that's not the case, please follow Handbook for howto setup a serial console other than COM1. To break into ddb, either use ctrl+alt+esc or send a BREAK (I think ^b will do) via serial line. After that, you should see db> Then you first use "ps" to find out the nfsd pid (better to remember the pid which eats lots of cpu before enter ddb). After that, do what Konstantin suggests. I have never tried "cont" in db. I guess that will return the execution back to kernel and you need to break into ddb again to do another "bt ". By the way, could you verify that backing out vfs_lookup.c rev 1.90 helps in your situation? If not, maybe we are seeing different problems, and then I have to figure out how to make my serial console work here. Thanks, Rong-En Fan