From owner-freebsd-fs@freebsd.org Sat Jan 9 15:30:54 2016 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 9D55CA68E19 for ; Sat, 9 Jan 2016 15:30:54 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 4094D1FC6; Sat, 9 Jan 2016 15:30:53 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) IronPort-PHdr: 9a23:2Kcf3RDBXtbKMG4xUtEXUyQJP3N1i/DPJgcQr6AfoPdwSP79r8bcNUDSrc9gkEXOFd2CrakU1ayO6+jJYi8p39WoiDg6aptCVhsI2409vjcLJ4q7M3D9N+PgdCcgHc5PBxdP9nC/NlVJSo6lPwWB6kO74TNaIBjjLw09fr2zQd6MyZzvn8mJuLTtICxwzAKnZr1zKBjk5S7wjeIxxbVYF6Aq1xHSqWFJcekFjUlhJFaUggqurpzopM0r221qtvkg789NV7nhN+R9FOQATWduD2dgzcHxtBDFBSCP73cQGjEfngBJCg7t4gv3U53qvm39rOUriweAOsijd7E/WnyH5qxoTBLtwHMdMjcy82Xaj+Rti61GrRa5p1p0ytiHM8muKPNic/aFLpshTm1bU5MUDnQZDw== X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: A2DQAQAXJpFW/61jaINehAxtBohTs04BDYFmGAqFI0oCgU8UAQEBAQEBAQGBCYItgggBAQQBAQEgBCcgCxACAQgYAgINGQICJwEJJgEBBAgHBAEcBIgNDrA7kAcBAQEBAQEBAwEBAQEBAQEBFwSBAYVVhH+ENwEBBYM1gUkFjjaIXYVDgnOCN5Fpjk8CIAEBQoQoIDQHhBg6gQgBAQE X-IronPort-AV: E=Sophos;i="5.20,544,1444708800"; d="scan'208";a="260428638" Received: from nipigon.cs.uoguelph.ca (HELO zcs1.mail.uoguelph.ca) ([131.104.99.173]) by esa-jnhn.mail.uoguelph.ca with ESMTP; 09 Jan 2016 10:30:52 -0500 Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id 9A0B915F55D; Sat, 9 Jan 2016 10:30:52 -0500 (EST) Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id v7k1wSdIoW01; Sat, 9 Jan 2016 10:30:52 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id EE44215F565; Sat, 9 Jan 2016 10:30:51 -0500 (EST) X-Virus-Scanned: amavisd-new at zcs1.mail.uoguelph.ca Received: from zcs1.mail.uoguelph.ca ([127.0.0.1]) by localhost (zcs1.mail.uoguelph.ca [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id ne0y4JofCE8o; Sat, 9 Jan 2016 10:30:51 -0500 (EST) Received: from zcs1.mail.uoguelph.ca (zcs1.mail.uoguelph.ca [172.17.95.18]) by zcs1.mail.uoguelph.ca (Postfix) with ESMTP id D1B8715F55D; Sat, 9 Jan 2016 10:30:51 -0500 (EST) Date: Sat, 9 Jan 2016 10:30:51 -0500 (EST) From: Rick Macklem To: Adrian Chadd Cc: FreeBSD Filesystems Message-ID: <2114482125.154452234.1452353451676.JavaMail.zimbra@uoguelph.ca> In-Reply-To: References: Subject: Re: LOR on 10.2-REL-p8 z/ ZFS + NFS? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [172.17.95.10] X-Mailer: Zimbra 8.0.9_GA_6191 (ZimbraWebClient - FF43 (Win)/8.0.9_GA_6191) Thread-Topic: LOR on 10.2-REL-p8 z/ ZFS + NFS? Thread-Index: A9idCivPcAZ6HQOPtPcXNxn3jLY/ZA== X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Jan 2016 15:30:54 -0000 Adrian Chadd wrote: > Hiya, > > Someone asked me about this. It's happening on a 10.2-REL-p8 box > serving ZFS via NFS. > > lock order reversal: > 1st 0xfffff801ed92ca28 zfs (zfs) @ > /usr/src/sys/fs/nfsserver/nfs_nfsdsocket.c:967 > 2nd 0xfffff8015f98a068 ufs (ufs) @ /usr/src/sys/kern/vfs_vnops.c:534 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame > 0xfffffe0467b8bb30 > kdb_backtrace() at kdb_backtrace+0x39/frame 0xfffffe0467b8bbe0 > witness_checkorder() at witness_checkorder+0xe24/frame 0xfffffe0467b8bc70 > __lockmgr_args() at __lockmgr_args+0x9d9/frame 0xfffffe0467b8bdb0 > ffs_lock() at ffs_lock+0x92/frame 0xfffffe0467b8be00 > VOP_LOCK1_APV() at VOP_LOCK1_APV+0xfc/frame 0xfffffe0467b8be30 > _vn_lock() at _vn_lock+0xd2/frame 0xfffffe0467b8bea0 > vn_rdwr() at vn_rdwr+0x1c1/frame 0xfffffe0467b8bf80 > nfsrv_writestable() at nfsrv_writestable+0xbd/frame 0xfffffe0467b8bff0 > nfsrv_openupdate() at nfsrv_openupdate+0x557/frame 0xfffffe0467b8c480 > nfsrvd_openconfirm() at nfsrvd_openconfirm+0x175/frame 0xfffffe0467b8c560 > nfsrvd_dorpc() at nfsrvd_dorpc+0xf66/frame 0xfffffe0467b8c720 > nfssvc_program() at nfssvc_program+0x4e6/frame 0xfffffe0467b8c8d0 > svc_run_internal() at svc_run_internal+0xbb7/frame 0xfffffe0467b8ca60 > svc_thread_start() at svc_thread_start+0xb/frame 0xfffffe0467b8ca70 > fork_exit() at fork_exit+0x84/frame 0xfffffe0467b8cab0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0467b8cab0 > --- trap 0xc, rip = 0x80089242a, rsp = 0x7fffffffe5d8, rbp = 0x7fffffffe880 > --- > > Any ideas? > This shouldn't cause a deadlock in practice. The second one is locking a vnode for a single file used exclusively by the nfs server for NFSv4 called "stablerestart". It shouldn't normally be on an exported volume and shouldn't be accessed by anything other than the nfsd. (It is only updated when a new NFSv4 client creates a clientid, which basically means "once per client" or "once per client mount" depending on the client.) Also, for the above, it exists on a UFS volume, so there is zero chance of a deadlock against a vnode on ZFS. I suppose if "stablerestart" existed on an exported volume and was accessed erroneously by a client as the first file opened after mounting, a deadlock is conceivable. --> I'll take a look and maybe the code can check to see if the nfsd thread already has a lock on the vnode to avoid this unlikely scenario. In summary, I may commit a fix for this someday, but I don't think it will ever cause a deadlock in practice. Thanks for reporting it, rick > > > -a > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >