From owner-freebsd-fs@FreeBSD.ORG Thu Jul 11 23:30:53 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1A0CAB7D; Thu, 11 Jul 2013 23:30:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id BF79F1D86; Thu, 11 Jul 2013 23:30:52 +0000 (UTC) X-Cloudmark-SP-Filtered: true X-Cloudmark-SP-Result: v=1.1 cv=u+Bwc9JL7tMNtl/i9xObSTPSFclN5AOtXcIZY5dPsHA= c=1 sm=2 a=2CN1efILQXEA:10 a=FKkrIqjQGGEA:10 a=l0nrKk16v60A:10 a=IkcTkHD0fZMA:10 a=6I5d2MoRAAAA:8 a=tG8P3wK5P364QUJtx80A:9 a=QEXdDO2ut3YA:10 a=SV7veod9ZcQA:10 a=HCYscHdTzkEnlTMq:21 a=Hg57A2LVB9d5kEav:21 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AqQEAIs/31GDaFve/2dsb2JhbABaFoMkT4MGvlCBHXSCIwEBAQMBAQEBICsgCxsYAgINGQIpAQkmBggHBAEcAQOHaAYMpiaRO4Emi2qBDxB+NAeCVoEfA5UVg3GIeYcrgViBVSAygQM3 X-IronPort-AV: E=Sophos;i="4.89,648,1367985600"; d="scan'208";a="40006670" Received: from muskoka.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.222]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 11 Jul 2013 19:30:45 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 51E5F79204; Thu, 11 Jul 2013 19:30:45 -0400 (EDT) Date: Thu, 11 Jul 2013 19:30:45 -0400 (EDT) From: Rick Macklem To: Bryan Drewery Message-ID: <672055679.467398.1373585445319.JavaMail.root@uoguelph.ca> In-Reply-To: <51DD3C1F.1000609@shatow.net> Subject: Re: NFS panic: newnfs_copycred: negative nfsc_ngroups (client HEAD r253033, server 9.1-R) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.91.201] X-Mailer: Zimbra 7.2.1_GA_2790 (ZimbraWebClient - FF3.0 (Win)/7.2.1_GA_2790) Cc: freebsd-fs@FreeBSD.org, FreeBSD Current X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Jul 2013 23:30:53 -0000 Bryan Drewery wrote: > I received this panic on the client while doing heavy parallel > reads/writes over NFS. I only recently moved these files to NFS, so I > don't know whether or not it's a recent regression. > > Client: HEAD r253033 > Server: 9.1-R > > core.txt: http://people.freebsd.org/~bdrewery/nfs.txt > > fstab of related paths: > > > tank:/tank/distfiles/freebsd /mnt/distfiles > > nfs > > rw,bg,noatime,intr,rsize=65536,wsize=65536,readahead=8,nfsv4 > > 0 0 > > tank:/usr/packages/ > > /mnt/all-packages nfs > > rw,bg,noatime,soft,retrycnt=3,rsize=65536,wsize=65536,readahead=8,nfsv4 > > 0 0 The mount options "soft" and "intr" should never be used for NFSv4. If an RPC fails with ETIMEDOUT or EINTR it can leave the open state in an undefined state. If you still get one of these crashes with all hard mounts, email again, since that would imply a client bug. (This is documented in the BUGS sections of mount_nfs(1), but not very well.;-) I'm not sure if this undefined open state could cause the crash, but it seems plausible, since the crash indicates garbage for the credentials in the open state structure. rick > > Server: params on these paths: -maproot=root -network 10.10.0.0/16 > > tcpdump at the time: > > > 21:43:05.396585 IP 10.10.0.7.4180315003 > 10.10.0.5.2049: 168 > > getattr fh 0,4/2 > > 21:43:05.396589 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48265029:48266477, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396603 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48266477:48267925, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396605 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack > > 48266477, win 3916, options [nop,nop,TS val 596674 ecr > > 1950216660], length 0 > > 21:43:05.396608 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48267925:48269373, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396621 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48269373:48270821, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396624 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack > > 48269373, win 3870, options [nop,nop,TS val 596674 ecr > > 1950216660], length 0 > > 21:43:05.396641 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48270821:48272269, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396653 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48272269:48273717, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396656 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack > > 48272269, win 3825, options [nop,nop,TS val 596674 ecr > > 1950216660], length 0 > > 21:43:05.396659 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48273717:48275165, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396671 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48275165:48276613, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396674 IP 10.10.0.7.946 > 10.10.0.5.2049: Flags [.], ack > > 48275165, win 3780, options [nop,nop,TS val 596674 ecr > > 1950216660], length 0 > > 21:43:05.396676 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48276613:48278061, ack 4394885, win 29124, options [nop,nop,TS val > > 1950216660 ecr 596674], length 1448 > > 21:43:05.396689 IP 10.10.0.5.2049 > 10.10.0.7.946: Flags [.], seq > > 48278061:48279509, ack 4394885, win 29124, options [nop,nop,TS val > > Write failed: Broken pipe > > I have nfsuserd running on both client/server. nfscbd is running. > nfs_client_enable=yes in rc.conf. > > User lookups seem to work fine: > > > -rw-r--r-- 1 root bryan 1554804 Jul 6 10:50 > > /mnt/distfiles/pkg-1.1.4.tar.xz > > I ran a find -ls on these paths and all files return a user/group. I > am > guessing there is a race condition with files being written and > looking > up the associated groups. > > -- > Regards, > Bryan Drewery > bdrewery@freenode/EFNet > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >