From owner-freebsd-fs@freebsd.org Sat Dec 12 17:51:12 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 68B714BBAAE for ; Sat, 12 Dec 2020 17:51:12 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4CtZvg3zZ6z3jPv for ; Sat, 12 Dec 2020 17:51:11 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.16.1/8.16.1) with ESMTPS id 0BCHowXE099686 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NO); Sat, 12 Dec 2020 19:51:01 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua 0BCHowXE099686 Received: (from kostik@localhost) by tom.home (8.16.1/8.16.1/Submit) id 0BCHowtX099685; Sat, 12 Dec 2020 19:50:58 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sat, 12 Dec 2020 19:50:58 +0200 From: Konstantin Belousov To: Rick Macklem Cc: J David , "freebsd-fs@freebsd.org" Subject: Re: Major issues with nfsv4 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FORGED_GMAIL_RCVD,FREEMAIL_FROM, NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.4 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on tom.home X-Rspamd-Queue-Id: 4CtZvg3zZ6z3jPv X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=gmail.com (policy=none); spf=softfail (mx1.freebsd.org: 2001:470:d5e7:1::1 is neither permitted nor denied by domain of kostikbel@gmail.com) smtp.mailfrom=kostikbel@gmail.com X-Spamd-Result: default: False [0.85 / 15.00]; TO_DN_EQ_ADDR_SOME(0.00)[]; FREEMAIL_CC(0.00)[gmail.com,freebsd.org]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; HAS_XAW(0.00)[]; R_SPF_SOFTFAIL(0.00)[~all]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RBL_DBL_DONT_QUERY_IPS(0.00)[2001:470:d5e7:1::1:from]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:6939, ipnet:2001:470::/32, country:US]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; NEURAL_SPAM_SHORT(0.85)[0.849]; TAGGED_RCPT(0.00)[]; R_DKIM_NA(0.00)[]; MIME_GOOD(-0.10)[text/plain]; SPAMHAUS_ZRD(0.00)[2001:470:d5e7:1::1:from:127.0.2.255]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_SPAM_LONG(1.00)[1.000]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-fs]; DMARC_POLICY_SOFTFAIL(0.10)[gmail.com : No valid SPF, No valid DKIM,none] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Dec 2020 17:51:12 -0000 On Sat, Dec 12, 2020 at 05:33:06PM +0000, Rick Macklem wrote: > Konstantin Belousov wrote: > [stuff snipped] > >Nullfs vnodes keep a reference on the lower vnode. When nullfs vnode > >caching is enabled, nullfs vnodes survive after a vfs syscall is finished. > > > >NFSv4 mount automatically sets flag MNTK_NULL_NOCACHE that disables nullfs > >vnode cache. > Thanks Kostik, I see that. (And I recall discussions about disabling the nullfs caching.) > > Now, if I understand it correctly, if vrele() is called with the vnode shared locked, > then VOP_INACTIVE() won't be called. > --> It is VOP_INACTIVE()/VOP_RECLAIM() in the NFSv4 client that does the closes. > Normally the NFS client calls vput() when the vnode is locked, but is there a case > where nullfs might cause vrele() to be called on the lower vp when it is shared > locked? (Delaying closes until VOP_RECLAIM() would cause problems, I think?) If vput() is called with the vnode shared-locked, it tries to upgrade to exclusive with LK_NOWAIT. If sleep-less upgrade fails, invalidation is postponed. In practice, the failure to upgrade the lock is not too common, because caller drops the last use reference on the vnode, but it still happens. That said, I do not believe that this situation causes the problems OP described. I want to see what is going on in the exiting process. > > Note that, until we see the "nfsstat -c -E" we won't know if lots of opens > are an issue anyhow. > > Thanks for the help, rick >